How Much Human Review Does AI-Generated Code Need?

Nobody on my team opens huge AI-written pull requests any more, me included. We split tickets more carefully than we used to, so each one asks for a smaller implementation. Smaller is relative, though. The larger of what's left still takes me a day or two to get through once I've tested it.

How I review those has varied. Some I skimmed. Some I pulled down and ran. A few went back to the author to be split again. And some I left entirely to an AI review.

The one I remember was a feature that wasn't complicated but had fallen behind schedule. I asked for an AI review, it found nothing, and I read the pull request description and the review items without opening the code at all. There wasn't time, and I told myself this wasn't the end of the development, so there would be plenty of chances to catch anything later. Bugs did turn up afterwards, mostly edge cases. I think they came in with later work rather than with that pull request.

For the others, I can't tell you now which changes they were or what made me decide, because I never wrote it down.

So I don't have a fixed amount of review per change. It gets decided anyway, one pull request at a time, and what I'd change first is that nothing recorded why.

Why the depth keeps getting decided in a hurry

AI sped up writing a change. Checking it still runs on the hours of whoever knows the code well enough to approve it, and a long thread on r/ExperiencedDevs was mostly people describing that reviewer as the one absorbing the difference.

The round trips speed up as well. An author working with an agent can answer your comments within minutes, so a small pull request can come back three or four times in a day, and one commenter on r/cscareerquestionsEU described each answer regenerating half the files so that the re-review started from nearly nothing. Under that kind of churn the shallow look is always the tempting one, which is exactly when nobody writes down that it was shallow. If a pull request has come back four times today, it's also fair to ask whether you're being used as the test suite.

Decide depth by what a mistake would cost

Diff size is a poor guide. The question I'd use instead has two halves: if this change is wrong, what does it cost, and can it be undone?

A label that's wrong on a settings page costs someone a moment of confusion, gets reported, and is fixed in one line. Nothing is lost in between. A change to who is allowed to do what, or one that deletes or rewrites stored data, can do damage that no revert takes back.

The public example I keep coming back to is Cloudflare's OAuth library for Workers. Its own history file says it was largely written with Claude and that engineers with security experience reviewed the output. In May 2025 the project published an advisory: the authorization endpoint never checked that a redirect URI was on the client's allowed list. The same check did exist at the token step, which is the less important of the two. In the advisory's note, the author says they knew about the requirement and somehow never made sure the check was in the code.

How thorough that review was is disputed. A security writer, Neil Madden, looked at the commit history a month later and doubted there had been much. Either way, it's the kind of mistake I'd expect to slip through careful reading. The code that existed was fine. The check it needed had never been written, and reading the existing lines more slowly wouldn't have shown that.

If I wrote my own rule down today, it would start roughly like this:

• Permissions and auth, deletes, migrations, writes to stored data, anything involving money. Read every line and run it. Then list the checks the change must contain and find each one in the code, because reading won't show you a missing one.
• Feature logic with tests you've actually looked at. Read for intent, and check the tests cover the case the change is about.
• Copy, labels, config that's quick to revert. An AI pass, then a look at what it flagged. If it flagged nothing, still open the diff for a minute; otherwise this has quietly become the next line.
• No human look at all. I've done this. The reason I gave myself was the schedule, and the feeling that there'd be time to catch problems later. Neither says anything about the change itself.

I'd defend the first three. The fourth I've used without a rule.

Depth doesn't have to be one per pull request, either. Take a made-up but ordinary case: an agent-written pull request replaces an old email-only notification flag with a new setting, adds a migration that fills the new setting for existing accounts from the old flag, and changes the label on the settings page. The migration belongs to the first group. The label belongs to the third. A reasonable review reads the migration and its test line by line, runs it against a copy of real data, and gives the label an AI pass and a look. The checklist for that migration is short: anyone who had turned email notifications off must still have them off afterwards, and new accounts need a default. Both have to be found in the code, not assumed.

Write the depth down

A plain approval looks the same whether you read every line or glanced at the title. The author can't tell which it was, and three weeks later you can't either. One line in the approval comment fixes that. For the case above: "read and ran the migration and its test against a data copy; found where the old opt-out carries over and the default; AI pass plus a look at the label."

If the backfill goes wrong a month later, whoever looks into it knows what was read. It also gives you something to check yourself against. I suspect my own line moved with how full the queue looked that day more than with the change itself, and a written note is the only thing that would show it.

The obvious risk is that the note turns into a ritual, or into something people point at when a change breaks. I don't know yet who on a team should read these, or how often.

What an empty AI review tells you

Before AI, my reviews ran on the code and what I knew about the domain. Now I usually start by asking a model what the change is for, what it touches and which parts look risky. Then I read the code myself, and the decision stays with me.

That opening question is the broad kind, and I'd expect broad passes to be weak at the Cloudflare sort of problem for the same reason people are. One developer on r/codereview described an authorization change that went through a general AI pass with no findings and broke in production. Asked afterwards whether the access boundary was enforced, the same model found the problem straight away. That's one account, not a study. The checklist in the first group is where the narrow questions come from: where is the redirect checked, where is the delete scoped, where does an existing opt-out carry across.

Leaving a change entirely to an AI review is the broad pass with nobody asking those. I've told myself the difference from a team that lets AI reviewers approve AI-written changes is that a person chose it for that change. That holds only if the person's reason is about the change. On that late feature, mine was about the deadline.

Where PR Flow fits

PR Flow puts an estimated risk and complexity reading on each pull request as it lands, using the AI provider you set up. It's a model's broad reading of the change, so a low reading deserves the same caution as an empty review. With AI on, the reading is one of the things that can move a pull request up your list, alongside work that came back to you and how long it has waited on you since. The reason on the row says which of those put it there.

When new commits arrive, the reading is marked outdated and names the commit it was made against. It keeps counting in the order until you refresh it, so if a late commit adds a migration or a permission change, refresh the reading so the order catches up. How deep to go still comes from what the commit changed.

Discuss is where I'd ask the narrow questions, about one file or the whole pull request. PR Flow's own agents don't approve, merge or change code. The local AI code review page covers where prompts go for each provider.

None of that tells you a change is safe, or which changes are fine to leave without a human read. PR Flow doesn't know which checks a change ought to contain, and it doesn't keep the depth note; that's yours to write. If generated code keeps outgrowing the people who can read it, that's a planning decision about how much work goes in and who reviews it.

I know the schedule decided it at least once. A month of depth notes, set against how full the queue was on those days, would tell me how often.