The Problem with AI Pull Request Reviews

Definition

The pull request review process was designed for human-to-human collaboration. It assumes an author who can defend their decisions, explain trade-offs, and answer questions when a reviewer is skeptical. When the author is an AI agent that no longer exists after the code is written, this fundamental assumption breaks. Reviewers are left staring at a diff with no access to intent, no record of rejected alternatives, and no understanding of the constraints that shaped the output. The process doesn't fail cleanly; it degrades into rubber-stamping or excessive skepticism, neither of which is effective governance.

Why This Matters

Let's walk through a typical PR review with AI-generated code.

A reviewer pulls up a PR titled "Implement user session cleanup." The diff shows:

• New background job that expires sessions older than 24 hours
• Scheduled to run every 15 minutes
• Uses soft delete (sets an inactive flag rather than removing rows)

The reviewer has questions:

• Why 24 hours? Is that an SLA decision, or did the AI just guess?
• Why every 15 minutes? Does the load justify that frequency, or could it be hourly?
• Why soft delete instead of hard delete? Is there a compliance requirement, or is the AI being conservative?

In a normal PR review, the reviewer would comment: "Why soft delete?" The author would explain: "We have audit requirements that need records for 90 days, so hard delete breaks compliance."

With AI code, there's no author. The reviewer has three options:

Option 1: Assume the AI knew what it was doing The reviewer rubber-stamps the code, assuming the AI had good reasons for every decision. This is dangerous because the AI might have made a random choice or a mistaken assumption.

Option 2: Reverse-engineer the reasoning The reviewer digs into the codebase trying to understand why these specific values were chosen. Do other parts of the system use 24-hour windows? Is there a compliance doc that mentions soft deletes? This takes hours and often leads to educated guesses, not certainty.

Option 3: Over-scrutinize out of caution The reviewer doesn't trust the code because they can't ask the author, so they question every detail. They request exhaustive testing, demand documentation, ask for alternatives. This creates massive review bottlenecks because they're applying skepticism that would only be warranted if they knew the reasoning was sound.

None of these options are good. Option 1 is a governance failure (no actual review). Option 2 is inefficient (expensive to reverse-engineer). Option 3 is a bottleneck (expensive to over-scrutinize).

The root cause is the same in all three: the reviewer doesn't have the information they need to make a real decision.

What the PR Process Assumes

The traditional PR workflow assumes several things:

1. The Author Is Still Available

If a reviewer has a question, they can ask the author. This is foundational. It enables rapid clarification, discussion of trade-offs, and collaborative problem-solving.

With AI code, the author ceased to exist the moment the code was written. There's no one to ask.

2. The Author Can Explain Their Reasoning

A good author can articulate why they made specific decisions:

• "I chose this library because it handles edge case X better than the alternative"
• "This architecture avoids circular dependencies in our dependency graph"
• "The loop is O(n) because I considered O(log n) but the constant factor was worse for our data sizes"

Explanation requires understanding intent, which humans have and AI decision-making doesn't always make visible.

3. Bad Code Gets Sent Back for Revision

If the reviewer finds problems, they request changes. The author revises. Rinse, repeat.

With AI code, who revises? You can re-run the AI with new instructions, but that might generate completely different code. You've lost the context of what the original code was trying to do.

4. The Reviewer Has Enough Context to Judge

Reviewers trust that they can see the code, understand what it does, and determine whether it's good or not.

For simple code (a function that sorts a list), this works. For complex code (a caching strategy, a state machine, a distributed system), understanding what the code does is only half the battle. You also need to understand why this design was chosen over alternatives. Without that context, judging the code is guess-work.

The Rubber-Stamp Problem

When reviewers face AI-generated code, they often take the path of least resistance: approve it.

This happens because:

1. Questioning is pointless: If you can't ask the author why, and you can't understand the intent from the code alone, why spend time questioning? There's no answer coming.
2. The AI probably knows better anyway: The instinct is "the AI was trained on lots of code, and I'm not as smart as the AI, so if it compiled and didn't trigger the linter, it's probably fine." This is false confidence, but it's human nature.
3. Time pressure: You have 50 PRs to review this week. Approving without deep scrutiny is faster than reverse-engineering intent.
4. Plausible deniability: If something breaks in production, you can say "I reviewed it like any other code; it's not my fault the AI made a bad decision." The governance responsibility gets muddied.

The result: code goes to production without anyone actually understanding why it was written the way it was.

This is worse than no review. At least with no review, you admit the code isn't reviewed. With rubber-stamping, you create the illusion of review while providing no actual governance.

The Over-Scrutiny Problem

Some teams go the opposite direction. Facing AI-generated code, they apply extra scrutiny because they don't trust the reasoning.

"I can't talk to the AI, so I need to verify every detail."

The review becomes adversarial. The code needs to prove itself to an extremely high bar. Testing requirements become exhaustive. Documentation requirements become excessive. Any edge case that could fail gets flagged.

This is also a failure mode, just a different one:

1. Extreme bottleneck: Code doesn't move forward until it passes an unreasonably high bar
2. Demoralizes the team: Developers lose faith in AI assistance if every PR takes two weeks to review
3. Wastes reviewer effort: You're spending 4 hours reviewing code that needed 15 minutes of review with proper context
4. False security: Over-scrutinizing a few PRs doesn't scale to a team generating dozens of AI-written PRs per week

Both rubber-stamping and over-scrutiny are responses to the same underlying problem: lack of context about decision-making.

What Reviewers Actually Need

Effective code review requires specific information at each stage. This information comes from committed checkpoints and detailed reasoning traces:

For Initial Assessment: "Is This the Right Solution?"

Before diving into implementation details, reviewers need to know:

• What problem is this solving?
• What were the alternatives?
• Why was this alternative chosen?
• What constraints drove the decision?

With this information, a reviewer can make a quick judgment: "This is the right approach given those constraints" or "Wait, we changed the constraint last month; this needs to be revisited."

Without it, reviewers either assume it's right (rubber-stamp) or question everything (over-scrutinize).

For Implementation Review: "Is This Well-Written?"

Once the approach is justified, reviewers can focus on quality:

• Does the code implement the approach correctly?
• Are there bugs, performance issues, or maintainability problems?
• Does it follow team standards?

This is where traditional code review excels. But it's premature if you haven't validated that the approach itself is sound.

For Governance Review: "Does This Respect Our Constraints?"

Finally, reviewers need to verify:

• Are there architectural constraints this code violates?
• Does it introduce risky dependencies?
• Does it bypass security or compliance requirements?

Again, this is much faster if you know what constraints the AI actually considered.

The Workflow Breakdown

Let's trace what breaks in the normal PR workflow when the author is an AI:

The question-and-answer loop is broken. The feedback mechanism is broken. The collaborative refinement process is broken.

Real Example: The Caching Decision

Let's trace a real scenario through a traditional PR workflow versus the broken workflow:

Scenario: User Profile Caching Implementation

TRADITIONAL (HUMAN-AUTHORED):

Reviewer: "Why use Redis for this? Why not Memcached?"

Author: "Redis because we need to share cache state across three app servers behind a load balancer. Memcached doesn't support that pattern reliably."

Reviewer: "Got it. Why 5-minute TTL?"

Author: "Two reasons. First, user profiles change infrequently—maybe once a day per user on average. Second, we have a compliance audit that requires 10-minute max staleness. 5 minutes gives us a safety margin."

Reviewer: "Understood. But I'm concerned about cache invalidation if a user edits their profile. What happens?"

Author: "Good catch. The profile update endpoint clears the cache entry for that user. So if there's an edit, the next read will miss cache and pull fresh data."

Reviewer: "Perfect. Approved."

AI-AUTHORED (CURRENT WORKFLOW):

Reviewer: [reads diff] "Redis cache, 5-minute TTL, soft deletes..." [has no idea why]

Reviewer: "Could use Memcached for this, cheaper. Why Redis?"

AI Author: [doesn't exist; no response]

Reviewer: "OK, I'll trust it's fine. TTL seems low though."

AI Author: [still doesn't exist]

Reviewer: "Is there cache invalidation on profile edit?"

AI Author: [silence]

Reviewer: [flips a coin] "Whatever, it looks reasonable. Approving."

[Code goes to production. A compliance audit notices the 5-minute TTL but the SLA was supposed to be 10 minutes maximum. But the AI's choice was actually conservative—it was 5 minutes to give a safety margin. The reviewer can't know this.]

The human-authored code moved fast because context was shared. The AI-authored code is either slow (over-scrutiny trying to figure out intent) or risky (rubber-stamping without understanding).

The Information Gap

The problem is fundamentally an information gap. The PR review process works when everyone has the same information:

• The author knows why they made decisions
• The reviewer can ask about decisions
• Both are working from the same understanding

With AI code, this information is one-way: the AI knows why it made decisions, but the information is lost when the AI session ends. The reviewer gets the code but not the reasoning.

Closing this gap is the core problem that governance systems need to solve.

How Different Review Processes Handle This

No Visibility, No Enforcement

Default approach. Code gets rubber-stamped or over-scrutinized. Rework happens in production. No real governance.

Visibility Only (Reasoning Traces Available)

Reviewers can see the AI's reasoning trace: prompt, constraints discovered, alternatives considered. They understand why decisions were made. Review becomes effective: "Is this approach still sound given what we know now?"

Much faster than reverse-engineering. Safer than rubber-stamping.

Visibility + Pre-Commit Enforcement

High-risk code is caught before it reaches review. Hard constraints (don't touch auth code, don't introduce unapproved dependencies) are enforced automatically. Review focuses on the code that actually needs human judgment.

Reviewers can be selective rather than blanket-skeptical.

Visibility + Pre-Commit Enforcement + Post-Commit Audit

Every decision is logged and auditable. When something breaks, you can trace back: Did the reasoning stand up? Did the constraint change? Was the review thorough? Did we miss something?

Governance becomes data-driven rather than guesswork.

Redesigning PR Review for AI Code

What should the PR review workflow look like when AI is the author?

The key differences:

1. Hard constraints are enforced before review (fewer surprises)
2. Reasoning is visible in the PR (reviewers understand context)
3. Review is about validating approach + intent, not guessing at purpose
4. Audit trail is comprehensive (you can learn from decisions)

Implications for Team Structure

This changes what code review requires:

With AI code and no visibility:

• Requires senior engineers (only they have enough context to reverse-engineer intent)
• Requires slow review (extensive questioning and verification needed)
• Requires high skepticism (can't trust the code without understanding intent)

With AI code and visibility:

• Junior engineers can do meaningful review (context is provided)
• Review can be faster (questions are answered by the decision record)
• Skepticism can be balanced (you understand what was considered)

This has HR implications: AI governance tools can accelerate junior engineers, because they provide the context that normally requires years of experience to accumulate.

Frequently Asked Questions

Should we have special review rules for AI-generated code?

Not special, but adapted. Traditional code review is optimized for human-to-human collaboration. AI code review needs to account for the lack of author context. That doesn't mean lower standards; it means different information requirements.

How do we prevent reviewers from rubber-stamping?

Make it harder to rubber-stamp without due diligence. If the decision record clearly states the constraints, a reviewer who approves without considering them is making a documented choice. That's traceable and auditable, which itself is a form of accountability.

What if the AI's reasoning is wrong?

Then the reviewer should see it in the reasoning trace. If the AI reasoned "This function is O(n) so it's efficient," and the reviewer knows that O(n) is actually slow for your dataset size, they can request a revision. The visibility makes it easy to spot logical errors in the reasoning, not just implementation errors.

Can junior engineers review AI code?

Yes, with visibility. If the decision record explains "We chose approach A because of constraint B," a junior engineer can read that and evaluate whether constraint B is still valid. Without visibility, reviewing AI code requires the judgment that only comes from experience.

How do we handle AI code that's generated by a different team?

Same as human code: visibility + clear governance policies. If Team A generates code that Team B uses, Team B needs to understand the reasoning and constraints. This is actually more important for AI code, because Team B can't just ask Team A's engineer.

What if reviewers disagree with the AI's decision?

That's a feature, not a bug. If a reviewer thinks the approach is wrong, they can see the AI's reasoning and explain why. "You chose approach A because you thought constraint B required it. We just changed constraint B; here's why approach C is better now." This is exactly the kind of informed decision-making that governance enables.

Does visibility slow down review?

Initially, it might add a few minutes per PR (reading the decision record). But overall, it makes review much faster because you're not wasting time reverse-engineering intent or over-scrutinizing out of confusion.

Primary Sources

• Framework for AI governance with review and approval requirements. NIST AI RMF
• Secure software development framework with review and approval practices. NIST SSDF
• Supply chain security levels with code review and artifact approval requirements. SLSA Framework
• SOC 2 governance criteria for review and approval control design. SOC 2 AICPA
• OWASP security risks to detect through code review and approval processes. OWASP Top 10 LLM
• OpenSSF scorecard for evaluating code review and approval security practices. OpenSSF Scorecard