AI Productivity Review Tax
Summary
AI saves time generating content but creates new mandatory work reviewing it — a “review tax” that nearly everyone pays. In a survey of 1,750 tech workers, 92.4% reported at least one significant downside to AI tools, with generic outputs (56.2%) and hallucinations (51.9%) as the top two complaints. The third-most-cited issue — time spent managing AI outputs (37.7%) — is the direct consequence of the first two.
This creates a productivity paradox: AI compresses generation time dramatically (3-10x across roles) but introduces a new work category that didn’t exist before. The net effect is still positive for most people, but the people getting the most value are those who’ve built workflows that account for the review tax rather than wishing it away. They use AI for first drafts, not final drafts. They verify before they ship. They’ve accepted that “good enough to edit beats perfect from scratch.”
Notably, the review tax varies by role. Engineers report the highest quality concerns (51% better but 21% worse — the highest “worse” of any role), likely because code has a higher bar for correctness: a “somewhat better” first draft of a PRD is useful; a “somewhat better” but buggy function is not.
How to Apply
- Budget for review: When estimating time savings from AI, don’t subtract only generation time — add back review/editing time. The net is still positive, but over-promising leads to disillusionment.
- Design workflows around it: Separate generation from review as distinct steps. AI generates; human reviews. Don’t blur the boundary.
- Match the tax to the stakes: Low-stakes outputs (brainstorm lists, first-draft emails) need minimal review. High-stakes outputs (code, public communications, financial models) need thorough review. Calibrate your effort accordingly.
- Watch for role-specific severity: Engineering teams will feel the review tax more acutely because correctness requirements are binary. Factor this into adoption planning.
Sources
From: AI Tools Are Overdelivering: Large-Scale AI Productivity Survey
Key quote: “The productivity paradox: AI saves time generating content but creates new work reviewing it. […] The people getting the most value aren’t waiting for these problems to be solved. Instead, they’ve built workflows that account for them. They use AI for first drafts, not final drafts. They verify before they ship. They’ve accepted that ‘good enough to edit’ beats ‘perfect from scratch.’” Attribution: Lenny Rachitsky, Noam Segal What this source adds: The first large-scale quantification of the review tax: 92.4% report downsides, with specific percentages for generic outputs, hallucinations, and time managing outputs. Role-by-role quality ratings provide the basis for understanding where the tax is heaviest (engineering: 21% say quality is worse). Links: Original | Archive
Related
- LLMs Generate, Not Retrieve — Root cause: the review tax exists because LLMs produce statistically plausible text, not verified text
- Follow the Drudgery — The review tax is itself a form of drudgery — and an opportunity for AI-assisted review tools
- AI Production-Thinking Spectrum — The review tax is heaviest for production use cases; thinking use cases (sounding board, brainstorming) have lower correctness requirements