Open-source AI code review
Find bugs in AI-generated code. Reproduce them with tests.
arnx investigates code changes, generates and runs tests for
suspected bugs, and lets you keep confirmed findings as regression
checks.
Your agent writes the code.
arnx checks the behavior.
⌖ From a real code review
×A failure worth investigating
Python + Postgres / real review
A page limit dropped callouts while the polling state advanced.
The generated tests reproduced the failure. The owner accepted
the finding and fixed the code.
Read the case study ↗
Summary of a recorded run; not terminal output.
Show me
the evidence.
Scan, then review
Your own model key, or arnx cloud (the first 100 accounts get $5 of
credit). macOS or Linux; on Windows, WSL 2.
Install with npm (Node 20 or later):
npm install -g @arnx-dev/arnx
1. Scan: what may be broken, in under a minute.
No Docker, no setup, nothing run.
2. Review: prove the suspects with tests. You
need Docker running.
A scan costs a few cents and lists suspects, ranked: where, what could
break, why, and an input to try. Each one is unproven until a test
fails. No model key? Run arnx login first: the scan runs
on arnx cloud, and arnx says so.
The first review sets up what it needs in a few minutes (it stops at
10, or 20 for Go and Rust). No separate setup step.
FOR CODING AGENTS
Let your agent run it.
arnx skill install gives Claude Code, Codex and other
agents that read Agent Skills the review and fix loop. Triage,
accept and commits stay yours.
arnx for coding agents ↗
REPRODUCIBLE DEMO
Try it on a planted bug.
Clone
arnx-dev/checkout-demo, switch to discount-codes, then run
arnx scan, arnx init and
arnx review. In our run the scan cost $0.04 and the
review $0.12, setup included.
The demo, step by step ↗
SUPPORTED RUNNERSPython / pytestJavaScript / TypeScriptRust / cargo-nextest · earlyGo / go test · earlyMore languages are planned
Want to see more here? ↗
Rust is early and our least-tested runner. First setup compiles
dependencies and test targets into an image that can take several GB. Go
is our newest language: generated tests reach exported names only.
Try your crate or module and tell us what breaks ↗
01 / Public evidence
Evidence from real repositories
Open source
Reviews of public code, each with the test, the upstream pull request
and what arnx got wrong. These projects are examples, not customers or
partners.
SQLFLUFF · MERGED
Valid ClickHouse SQL was unparsable.
A generated test showed projections without
ORDER BY or GROUP BY were rejected. The
fix was merged upstream.
Read the SQLFluff case ↗
MARK · MERGED · GO
A second Close returned too early.
A concurrency bug in a Go tool, found with a development build
before Go shipped in 0.1.14. The fix was merged upstream.
Read the mark case ↗
BEETS · PULL REQUEST OPEN
Shell completion crashed with one plugin on.
A regression test caught an option registered twice. The fix and
test are in an open pull request.
Read the beets case ↗
HUMMINGBOT · PULL REQUEST OPEN
Order statuses paired with the wrong order.
One finding became a fix we submitted. Others matched fixes the
maintainers had already made on their own.
Read the Hummingbot case ↗
ARNX · SELF-REVIEW
A delete that could leave the repository.
arnx reviewing its own remove command: the test, the
failure, the fix and the check it still runs.
Read the self-review ↗
How often do findings hold up on code we do not know? We graded 65 of
them.
Read the counts.
02 / A private review, start to finish
A finding. A fix.
The same tests.
Python + Postgres
On a Python pipeline backed by Postgres, generated tests
reproduced an inverted calculation, lost callouts beyond a page
limit, and stale prices used in evaluation.
The owner marked 18 findings real. Their coding agent changed the
product code; accepted test files stayed unchanged after
acceptance.
Read the full case study ↗
FINAL ACCEPTED-TEST CHECK
54passed
0failing
1skipped
One owner-run case on a tuned development branch. Two
known issues were missed. The counts describe the final test run, not
bug-detection accuracy. Private source and raw execution logs are not
published.
03 / How arnx works
Investigate the change.
Keep the checks that matter.
-
01
Scan
Models read the change and list ranked suspects in under a minute.
No Docker, no setup, nothing run. Each suspect is unproven until a
review tests it.
$ arnx scan
-
02
Review
Initialize once, then generate targeted tests and run them in
Docker. The first review sets up only the projects the change
touches. Read the failures, open questions, and coverage gaps.
$ arnx init
$ arnx review
-
03
Accept
Triage real / intended / noise / rewrite, then accept useful
tests. Use arnx prompt to hand the fix to your coding agent.
$ arnx triage
$ arnx accept
$ arnx prompt
-
04
Check
Rerun accepted regression tests in Docker without new model calls.
Review and commit the tests with the fix.
$ arnx check
Read the complete CLI workflow ↗
04 / Why executable evidence matters
A suspicion needs a test.
A failing test needs your judgment.
Inspect the behavior
A review comment identifies a possible problem. An executed test
gives you inputs, an expected result, and a failure you can
investigate.
Validate the assumption
A test can misunderstand intent, depend on a brittle fixture, or
fail at setup. Repetition and a second model do not replace
developer review.
Check it again
Accepted tests live in .arnx/tests/, pinned by
sha256 in .arnx/accepted-tests.json. Your own suite
never collects them; only arnx check runs them in a throwaway
copy. Read this model-written code first. Use
arnx check after the fix and on later changes, with
no additional model inference cost.
Understand the product and its boundaries ↗
Read the guides ↗
What a finding means
Repeat the failure
arnx runs each test 3 times, reruns failures on the code before
the change, and has a second model confirm or dispute it. A
failing test shows an assertion fails, not that the code is wrong.
Model replies vary: two reviews of one change can differ.
Compare before and after
Before/after outcomes are regression, pre-existing, new-code,
changed or unknown. Only regression is evidence about the change:
the test passes before and fails after, if its expectation is
right.
Make your decision
Findings with one cause are grouped into an issue, most severe
first, explained as input, actual, expected and why it matters.
Issues backed by a confirmed test come pre-marked real; change
what you disagree with. Marks are real, intended, noise or
rewrite.
Read the coverage gaps
Tests that pass after the change but fail before it appear under
“The change fixes”. Changed code no test touched is listed by
name, and the review says “partly covered”. A passing check covers
only the accepted tests.
A test from arnx’s own review
arnx reviewing its own arnx remove command, before 0.1.
The test checks whether a recorded path can escape the repository.
Test and report quoted exactly from the README.
test("S2 accepted-test paths escaping the repository are never deleted", async () => {
const { base, repo, env } = setup();
const victim = join(base, "victim.txt");
put(victim, "keep me");
put(join(repo, ".arnx", "accepted-tests.json"), record({ "../victim.txt": {} }));
const cli = new Cli({ cwd: repo, env });
await withDocker("/nonexistent/docker", () => cli.run("remove", "--yes", "--tests"));
expect(existsSync(victim)).toBe(true);
});
### Issue 1 · remove --yes can delete files outside the repository
- Actual: In S2, `../victim.txt` outside the repository was deleted. …
- Failing tests: F2, F4
### F2 · regression · confirmed by the second opinion
- Failure: `expect(received).toBe(expected)`
- Before the change (regression): passes on the code before the change (3 runs), fails with it
- Second opinion: planRemove takes accepted-tests.json keys verbatim and applyRemove rmSync's join(repo, p), …
The test failed in all 3 runs, passed before the change, and the
second model agreed. We marked it real, kept it with arnx accept,
fixed the code from the fix prompt and saw it pass in arnx check.
Not every finding is like this one: in our own reviews, some
confirmed issues tested behaviour we intended.
05 / Choose your setup
Your keys. Our Cloud. Your requirements.
BYOM · OPEN SOURCE
Bring your own model.
Use supported provider API keys and pay providers directly. Tests
run in your local Docker environment.
arnx Open Source is licensed under Apache License 2.0. CLI version 0.1.15.
Open-source quickstart ↗
ARNX CLOUD
Let us manage the models.
Managed model access and centralized billing, without bringing
your own provider keys. The first 100 accounts get $5 of Cloud credit.
New project: arnx init --cloud
Already using arnx? arnx cloud on
Get started with Cloud ↗
ENTERPRISE · LET’S TALK
Need a private setup?
Talk directly with us about private deployment requirements, data
boundaries, and dedicated support for your team.
We’ll work through your requirements and agree on scope together.
Email us about Enterprise ↗
Tool calls, Docker tests, the before/after check, and report generation run on your machine. arnx cloud receives the diff, the files the agent reads, the tests it writes and their output, then passes them to its model providers.
Read the data and execution boundaries.
Keep the evidence
Run your first review.
Keep the tests that matter.