Feat/verification run all checks#197
Conversation
TestSuiteGate.verify ran pytest via a hard-coded python3 interpreter with cwd set to file_path.parent. For nested files this misses the repo's pyproject.toml / pytest.ini / conftest.py and may use a different interpreter/venv than the host process, causing false positives or undiscovered tests and CI-vs-local mismatch. - Run pytest via sys.executable instead of python3. - Set cwd to the resolved project root (project_root or file dir). - Prepend the project root to PYTHONPATH for layouts that rely on it. - Add tests asserting the interpreter, cwd, and PYTHONPATH. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Warning Rate limit exceeded
You’ve run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (9)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Two related improvements to the verification pipeline. Machine-readable JSON output: CI gates, bots, and parent tooling previously had to scrape Rich-formatted terminal text. This adds to_json_dict() helpers on the VerificationResult and CheckResult data contracts (backed by a JSON_SCHEMA_VERSION constant so the schema can evolve safely), a format_verification_result_json() formatter, and a new `refactron verify` CLI command with a --json flag that emits a stable, versioned JSON object and returns exit code 0 when safe to apply or 1 when blocked. CLI subcommand registration in main.py is also hardened to catch any exception rather than only ImportError, so a broken optional dependency degrades one subcommand gracefully instead of taking down the whole CLI. Optional run-all verification checks: VerificationEngine.verify previously stopped after the first failing check, hiding stacked problems. A short_circuit flag (default True, preserving existing behaviour) is added; when False the full pipeline runs so every failure category surfaces in one run, with checks_failed aggregating all failures while blocking_reason stays the first failure for stable prioritization. A --all-checks CLI flag wires this through. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
shrutu0929
force-pushed
the
feat/verification-run-all-checks
branch
from
330eea1 to
b93c4ec
Compare
1 participant
solve #190
This change makes verification failure reporting more complete by letting callers run the full check pipeline instead of stopping at the first failure. Previously VerificationEngine.verify short-circuited as soon as one check failed and recorded the remaining checks as skipped, which is good for latency but hides stacked problems — for example, an author would never learn the test-suite status until after import errors were fixed, forcing multiple verify-fix cycles. A new short_circuit parameter is added to verify(), defaulting to True so existing behaviour is fully preserved; when set to False, every check runs regardless of earlier failures, nothing is recorded in skipped_checks, and checks_failed aggregates all failures while blocking_reason continues to report the first failure for stable prioritization. A corresponding --all-checks flag is added to the refactron verify CLI command, which simply wires short_circuit=not all_checks into the engine. New tests cover the run-all behaviour and confirm that multiple failures are aggregated correctly in a single run.