In this piece
Alibaba Open Code Review returned no comments. Which files did it check?
A billing change made each retry of the same invoice produce a different payment key. The developer also edited the test. In our two-file example, Alibaba Open Code Review selected the Go implementation and left the test out by default. The missing file contained the clearest description of what a retry was supposed to do.
Which files does Alibaba Open Code Review select?
Use --preview on the pull request’s Git range. It lists which changed paths are eligible for review, along with the reason for each exclusion, without calling a model. We ran version 1.13.8 on 13 November 2026. You can repeat the selection check before configuring an API key. Alibaba’s v1.13.8 release
git diff --name-only main...HEAD
ocr review --from main --to HEAD --preview --format jsonBoth commands used the same two commits in our example. On main, attempts 1 and 2 for invoice inv-42 shared a key. On the branch, the keys became inv-42-1 and inv-42-2. The edited test now required different keys. If a payment processor uses that key to identify a repeat request, a retry after a timeout could charge the invoice twice. The edited test would pass.
| Changed file | Default selection | With the test explicitly included |
|---|---|---|
billing/retry.go | Selected | Selected |
billing/retry_test.go | Excluded (default_path) | Selected |
For the second preview, we added one entry to the local .opencodereview/rule.json and kept the comparison commits fixed. The eligible count went from one file to two. A team can include a particular test this way and keep its other default exclusions. File selection and rule precedence in v1.13.8
{"include":["billing/retry_test.go"]}Download the shell script for the two-file experiment. Run it with Bash and a local v1.13.8 binary. The script creates its own Git repository and checks both previews. We measured file selection only. Whether the model would spot the changed payment contract is a separate experiment.
What does an empty comments array mean in CI?
Read the final JSON report’s status and any warnings before counting comments. A skipped review can return "comments": []. Exit code 0 can also accompany nonfatal warnings. Compare the final report with the expected files from Git and the preview for the exact revision queued for merge. CLI reference for JSON status, warnings and exit codes
We also ran a full CLI invocation on a branch that changed only the excluded test. The process exited 0 with status: skipped, comments: [] and zero files reviewed. A local endpoint recorded zero model requests. If your saved report does contain findings but GitHub shows none, check the action’s posting settings and logs. GitHub Action reporting options
| Report or process result | Next step before merging |
|---|---|
skipped | Check the comparison base and the filters. If an important path was excluded, include it and rerun, or assign someone to review it. |
partial, completed_with_warnings or completed_with_errors | Inspect the affected paths and warnings. Recover missing work or give it to a reviewer. Keep valid findings from the completed work. |
failed, fatal exit or missing report | Treat the review as unavailable. Check stderr and confirm that any saved report belongs to this run. |
complete or success, expected files covered, zero comments | Record that the reviewer raised no findings on that scope. For billing, still exercise the timeout and retry against the one-charge requirement. |
When a manifest is present, compare manifest.coverage.selected with the paths listed under completed, reused, failed and waived. A waived file still needs a reason you accept. Record the base and head commit IDs alongside the result. File coverage in the v1.13.8 run manifest
What does the reviewer do after it selects a file?
Alibaba’s Go CLI handles file selection, rule matching, execution limits and positioning comments on changed lines. Related files can be reviewed together and groups can run in parallel. The agent can read a whole file, follow a caller and search the repository before deciding whether a change is worth a comment. A file excluded as a review target may still be read for context. Alibaba’s architecture, integrations and installation guide
Alibaba says its internal predecessor served tens of thousands of developers over two years and identified millions of defects. The public project, licensed under Apache-2.0, has passed 50,000 GitHub stars. For a team evaluating it, the relevant question is whether it can find the kind of regression that makes it through that team’s existing checks. Open Code Review on GitHub
What did Open Code Review catch compared with Claude Code?
In Alibaba’s 200-pull-request benchmark, Open Code Review produced a higher share of comments that matched annotated defects. Claude Code found more of the known defects. The comparison below holds the model constant. It uses the authors’ August evaluation across 50 repositories and 10 languages, with 1,505 annotated issues. OpenCodeReview paper, evaluation setup and Table 3
| Metric | Open Code Review 1.3.1 | Claude Code 2.1.169 |
|---|---|---|
| Precision | 33.90% | 7.23% |
| Recall | 20.00% | 28.90% |
| F1 | 25.10% | 11.57% |
| Average tokens per PR | 385,000 | 5,664,000 |
On that run, Open Code Review matched 301 of the 1,505 annotated issues. Claude Code matched 435. Higher precision makes a surfaced comment more promising to investigate. Lower recall gives you a reason to keep an independent test for a costly mistake. Neither figure tells you how often today’s CLI will catch a double charge in your product.
What would we require before merging the billing change?
Have someone who owns payments confirm what a retry means. If the first charge succeeded but its response timed out, a second attempt for the same invoice should leave one charge in the ledger. Run that scenario and inspect the resulting ledger entries. The edited unit test only establishes that the code returns two different strings.
If you are introducing the reviewer, pick a previously fixed retry defect and a similar clean change. Hide the fix from the reviewer, record which paths it covered, and ask an engineer to verify the findings. Our guide to testing AI-generated code before release covers the broader QA decision. For the cost of escalating reviews on a hosted service, see our Greptile review-budget comparison.
Open Code Review CLI questions
How do I install the version used in the example?
npm install -g @alibaba-group/[email protected] and check ocr version. The downloaded fixture needs Bash, Git and jq as well as the local OCR binary. A preview can run before you configure a model.How do I review one commit instead of an entire pull request?
--commit with the commit SHA to ocr review. For a pull request, use --from and --to on the two fetched revisions. The range starts at their merge base, so it captures what the branch introduced.How should I review a pull request that changes only tests?
Can the CLI review a whole directory without a diff?
ocr scan --path billing for a full-file scan of a billing directory. Choose this when you want to examine existing code. To check what a particular pull request changed, keep its commit range so the result can be traced to that change.Can it use my coding agent’s existing model?
Independence and trademarks: Wavect publishes this page and is itself a provider, so we have a commercial interest in it. We are not affiliated with, endorsed by or partnered with the other companies named here, and all third-party company names, brands and trademarks are the property of their respective owners. Statements about other providers are taken from publicly available sources, primarily their own published pages, as of the review date shown on this page, and may have changed since. Please verify them directly before you decide. This page was written to the best of our knowledge and with the intent to remain objective. If you believe anything here is inaccurate or unfair, write to us and we will correct it: [email protected]