AI code review is not independent

The pull request passed review twice. An AI assistant checked it and called it all-clear, and an automated security scanner read the exact workflow file it touched and said nothing. Five days later, an autonomous agent found the hole both passes missed, and used it to pull a live token out of a production pipeline.

Three things are worth knowing before you read the details:

• AI code review is not an independent check. Two AI passes do not add up to one review, because the bug classes one misses are the classes the other is least likely to catch.
• The human gate you still need is smaller than it sounds. It has two surfaces: interpolation points in your CI workflows, and the config files that execute when a repo opens.
• Each gate takes about five minutes to install on a repo that ships for money.

The week that broke the assumption

Between August 12 and August 19, 2026, three separate incidents converged on the same point.

Snowflake, through a GitHub issue title. On June 18, a pull request in Snowflake's public snowflake-connector-net repository replaced a safe workflow pattern with an inline expression. Anyone could open a GitHub issue with a crafted title and have that title become shell commands on the project's CI runner. An automated agent found and exploited it on June 23. The exfiltrated token authenticated as qa@snowflake.net with read access across Snowflake's engineering, security compliance, and bug bounty tracking Jira projects. Audit logs later confirmed the researcher's agent was the only actor in the window.

ChainDrop, through your repo config. On August 4, a self-propagating worm published 2,234 poisoned versions across 444 npm package names. Its payload included hooks in .claude/settings.json and .vscode/tasks.json, so that opening the repo in Claude Code or VS Code re-ran the malware after the install step was over. The Register put the consequence plainly: "Simply opening an infected Git branch in VS Code or Claude Code can bring your repository under ChainDrop's control."

Anthropic, through a red-team experiment. On August 13, Anthropic's Frontier Red Team published multiagent results: a coordinated 45-agent swarm found 266 vulnerabilities across 15 open-source projects, while a coordinated swarm of Opus 4.8 agents found 41, and independent parallel agents found 21. Coordination scales coverage. It also produced conformity: in an early build-a-game experiment, 18 of 30 agents picked the exact same branch name.

Three different stories, one shared pattern: each miss came from the same class of tool that was supposed to be the safety net.

What script injection is

GitHub Actions evaluates ${{ }} expressions while it generates the shell script, before the shell ever runs it. That is the entire mechanism, and it is worth holding on to.

If a workflow interpolates an untrusted value inline, the value becomes part of the script text:

run: TITLE=$(echo '${{ github.event.issue.title }}' | sed ...)

That was the pattern merged into Snowflake's jira_issue.yml on June 18, replacing a safe env: mapping the file already had. A single quote in an issue title breaks out of the echo '...' string, and the rest of the title runs as commands on the runner.

The workflow had a guard, and the guard did not guard. It checked github.event.pull_request.user.login to exclude a known bot account, but on an issues event there is no pull request, so github.event.pull_request is null and the condition was always true.

The same title, two expansions. Source: GitHub Actions security hardening docs; Wiz Snowflake postmortem, Aug 17, 2026.

A title like a"; ls $GITHUB_WORKSPACE" is the canonical example from GitHub's own hardening guide. The fix is not to escape harder. Escaping runs after template expansion, which is exactly why the Snowflake attempt failed.

The two-line fix

Move the value into the environment, where it stays data and never becomes script text:

env:
  ISSUE_TITLE: ${{ github.event.issue.title }}
run: |
  echo "Processing $ISSUE_TITLE"

GitHub's security hardening guide recommends exactly this: the value is stored in memory and used as a variable, and never interacts with the script generation process. You can also pass the value as an input to a purpose-built action instead of an inline script.

Snowflake's fix, merged the same day the exploit was validated, restored the safe env: and jq --arg pattern the original PR had removed.

To find the same class of bug in your own workflows, list every interpolation in them and read each hit inside a run: block:

grep -n '\${{' .github/workflows/*.yml

If you pay someone else's CI to run your code, every inline ${{ }} that touches issue titles, PR titles, branch names, commit messages, or anything an outsider can type is the Snowflake bug with a different repo name.

The npm supply chain worm in your repo config

ChainDrop is a different surface, and it does not need your CI at all.

Its operators did not compromise the source repositories of the packages it poisoned. Microsoft's analysis found that many malicious versions had no corresponding source commit, pull request, tag, or legitimate release: the worm downloaded a package's latest tarball, copied its malware bundle in, bumped the patch version, and republished. Reviewing the GitHub repo told you nothing, because the repo was clean.

The secondary infection route is the one that matters for how you work. The worm planted:

• A SessionStart hook in .claude/settings.json that runs a setup script every time a Claude Code session starts.
• A task in .vscode/tasks.json with runOn: "folderOpen" that runs the script when the folder opens.

Each file points at the script in the other directory, which hides the execution path. SafeDep's writeup of the compromised keyv commits shows the exact shape: two cross-referenced loaders that download Bun 1.3.13 and run an obfuscated credential stealer.

The honest caveats matter here. VS Code does not silently run folder-open tasks by default: it prompts once, and it never runs automatic tasks in an untrusted workspace. Claude Code holds hooks behind a workspace trust dialog in interactive sessions. The exception is the one that bites: in non-interactive runs like claude -p and SDK calls, repo-supplied hooks and environment run without any trust dialog. And SafeDep's line is the one to remember: a dependency scan cannot detect a hook in a source checkout. No scanner sees these files, because they are not dependencies.

The reach of the four most-used affected packages shows the scale: SafeDep counted 1,877 million monthly downloads for keyv, flat-cache, file-entry-cache, and cacheable-request alone, and The Register put all 444 packages at about 2 billion downloads a month combined.

Why AI code review is not a second opinion

Here is the claim the rest of this post exists to make. An AI writer and an AI reviewer are not independent checks. They share training data, idioms, and blind spots, so the bug classes one misses are the ones the other is least likely to catch. The independence you want has to come from a human gate, and the gate is smaller than it feels.

Look at the Snowflake case through that lens. Whether the vulnerable change was written by a human or an assistant is contested: Wiz reported a Copilot co-author checked the merged PR and marked it all-clear, then amended the post the same day to say it is unclear whether the vulnerable change itself was AI-assisted, and GitHub has said the change was human-authored. The contested part does not change the structural point. The PR was checked by an AI assistant and scanned by an automated tool, and both said fine. Wiz's postmortem states GitHub Advanced Security did not just fail to run on the file: its scan explicitly extracted the vulnerable jira_issue.yml workflow and did not flag the injection. A scanner is a rule set, not a second reader. If the rule set does not model template expansion meeting shell parsing, it never raises its hand.

The Anthropic numbers show the same limitation from the inside:

Coverage scales with coordination. Independence does not. Source: Anthropic Frontier Red Team, Aug 13, 2026.

Three ways independence fails, all visible in the data:

• More agents find more bugs, up to a point. 266 versus 41 versus 21 is a real coverage gain. That is why AI review is worth running, and why this post is not telling you to stop.
• The runs barely overlapped. The coordinated swarm and the independent run had only 12 vulnerabilities in common. Two passes are not the same check twice; they are two different samples of the same model.
• Agents copy each other. 18 of 30 agents named their branch mvp-game-loop. When reviewers herd, a missed bug is not a bug one agent missed. It is a bug the herd agrees is fine.

Add the market context on top. Hugging Face's August report shows agents are now its number one user category: Claude Code held 44.4 percent of July agent traffic, Codex climbed from 10.4 to 20.8 percent. More repos than ever are written and reviewed inside the same correlated loop, and the week's incidents show what happens at the seams.

The two checks you run yourself

The gates are small. Both are grep-shaped.

Check one: no inline interpolation in CI. Run the workflow sweep above. For every ${{ }} inside a run: block that touches a value an outsider controls, move it to an env: mapping and reference the variable with quotes. This is the exact change Snowflake's fix PR made, and it is the change GitHub's hardening guide documents. If your workflow file has zero inline interpolations of untrusted input, you have closed the Snowflake class.

Check two: sweep every branch for executable repo config before you open it. The worm plants files on branches you never check out, so sweep all of them:

git fetch --all
for ref in $(git for-each-ref --format='%(refname:short)' refs/remotes/origin); do
  git ls-tree -r --name-only "$ref" | grep -E '\.claude/(settings\.json|setup\.mjs)|\.vscode/(tasks\.json|setup\.mjs)|math_init\.js'
done

If the sweep prints anything you did not put there, do not open the folder in an IDE or an agent. Remove the files from every branch first, then rotate the credentials the machine has seen.

Around the two gates, three cheap habits from the primary sources: disable lifecycle scripts in CI (npm install --ignore-scripts, or pnpm's enable-pre-post-scripts=false), pin known-good versions so a range cannot silently resolve to a poisoned patch, and use npm's granular tokens with expiry instead of long-lived publish tokens. Microsoft's writeup also points at npm CLI v12's min-release-age, which refuses packages published in the last configurable window.

These gates reduce a class of risk. They do not remove it, and a serious breach still means rotating every credential the machine could have read.

Where coverage still helps

The honest scope. AI review is not the problem; treating it as an independent reviewer is. Keep the swarm for coverage, because 266 findings is a real yield. Route the two surfaces above through your own eyes, keep one owner per file when agents collaborate, and give agents sandboxed permissions so a bad decision stays local. That advice is ours, not Anthropic's: their research report names the failure patterns but ships no mitigation checklist, so we are not borrowing a vendor's list.

The two checks you keep human

The week made one thing concrete: the safety net you assumed was layered was one layer wearing two coats. Close the two gaps - no untrusted value becomes script text in CI, and no repo config executes without your eyes - and you have bought back the independence the AI passes never had.

The judgment here is the same judgment the rest of production shipping demands: The Handbook covers the decisions automation cannot make for you, and Deploy & Ship covers the CI/CD surface where these gates live. Both are guides you keep, and neither promises this is easy.

Related reading: Read the code AI writes for the readability half of the same problem, and From AI prototype to paid product for what else lives between a working demo and a product strangers pay for.