AI Tools Are Overdelivering: Results from Our Large-Scale AI Productivity Survey

By: Lenny Rachitsky, Noam Segal Source: Lenny’s Newsletter Type: newsletter

Summary

One of the largest independent surveys on AI’s impact on tech workers (1,750 respondents across PMs, engineers, designers, and founders). The headline — 55% say AI exceeded expectations, 50%+ save at least half a day per week — is striking but the real value is in the role-by-role breakdown. Founders report highest satisfaction because they use AI to think (decision support, ideation, strategy), while PMs and designers use it to produce (PRDs, prototypes, copy). The biggest demand gap for PMs: user research (+27.2pp between current and desired use). Engineers have accepted AI as a coding partner and now want it for the boring-but-necessary work: docs, tests, code review. Agent adoption is still nascent — 14% active users vs. 49% interested. The clear mental model: follow the drudgery to find where AI creates the most value.

Key Ideas Extracted

Follow the Drudgery: The roles most transformed by AI won’t be where AI is “smartest” — they’ll be where there’s the most tedious busywork. Engineers want AI for docs/tests/review, not the hard stuff.
AI Production-Thinking Spectrum: Founders use AI to think (strategy, ideation, decision support) and report highest satisfaction. PMs use it to produce (PRDs, prototypes, comms) and haven’t cracked upstream/strategic use yet. Maturity = moving from production to thinking.
AI Productivity Review Tax: AI saves time generating but creates new work reviewing — 92.4% report downsides, mainly generic outputs (56.2%) and hallucinations (51.9%). “Good enough to edit beats perfect from scratch.”
AI Disrupts Strategic PM Skills Most (enriched): Survey data showing the demand gap — PMs want AI for research (+27.2pp) and prototyping (+24.6pp) but haven’t cracked it yet. Validates the disruption potential with hard numbers on where adoption lags.

Notes

Survey methodology: 1,750 respondents, well-distributed across company sizes (~40% small, 33% midsize, 28% enterprise). Seasoned audience — 53% have 6-15 years experience, 33% have 16+.
Tool landscape snapshot (late 2025): ChatGPT dominates for non-engineers. Engineers split between Cursor (33.2%), ChatGPT (30.8%), Claude Code (29.0%). Switching costs low.
The “Sean Ellis PMF question” applied to AI tools: ChatGPT has 50.2% loyalty but that’s lower than usage, suggesting vulnerability. Specialized tools (Cursor, Claude Code, Granola) show disproportionate loyalty.
n8n dominates agent platforms (219 mentions vs. Zapier’s 85), despite Zapier’s broader name recognition. Agent workflows remain 75%+ human-directed even among users.
The 3x-10x time compression pattern is consistent across roles: PRDs (days → <1 hour), competitive research (weeks → days), prototyping (month → day/hour).
Designers report lowest satisfaction — only 45% positive ROI, 31% below expectations (triple founders’ rate). Possible explanations: tools not yet good enough for precision visual work, or designers have higher standards for output quality.

Raw Content

Full article text captured from Lenny’s Newsletter. See source URL for original with charts.

tl;dr: AI is overdelivering.

55% of respondents say AI has exceeded their expectations, and almost 70% say it’s improved the quality of their work.
More than half of respondents said AI is saving them at least half a day per week on their most important tasks.
Founders are getting the most out of AI. Half (49%) report that AI saves them over 6 hours per week. Close to half (45%) also feel that the quality of their work is “much better” thanks to AI.
Designers are seeing the fewest benefits. Only 45% report a positive ROI (compared with 78% of founders), and 31% report that AI has fallen below expectations, triple the rate among founders.
Engineers have accepted AI as a coding partner and now want it to handle the more boring (but necessary) work of building products: documentation, code review, and writing tests.
n8n is currently dominating the agent landscape, though actual adoption of agentic platforms in 2025 has been slow.
A whopping 92.4% of respondents report at least one significant downside to using AI tools.

PM AI Use Cases (Current):

Write PRDs: 21.5%
Create mockups/prototypes: 19.8%
Improve communication: 18.5%
User research: 4.7%
Roadmap ideas: 1.1%

PM AI Use Cases (Demand Gap — want to use next):

User research: +27.2pp
Creating mockups/prototypes: +24.6pp (most-wanted future use case at 44.4%)

Designer AI Use Cases:

User research synthesis: 22.3%
Content and copy: 17.4%
Design concepts ideation: 16.5%
Visual design: 3.3% (#8)

Founder AI Use Cases:

Productivity/decision support: 32.9%
Product ideation: 19.6%
Vision/strategy: 19.1%

Founder AI Demand Gap:

Product ideation: +29.0pp
Growth strategy/GTM planning: +24.7pp
Market analysis: +24.0pp

Engineer AI Use Cases:

Writing code: 51% (dominant)
Documentation: 7.7%
Testing: 6.2%
Code review: 4.3%

Engineer AI Demand Gap:

Documentation: +25.8pp
Code review: +24.5pp
Writing tests: +23.5pp

Time Saved Per Week:

63% of PMs: 4+ hours
83% of founders: 4+ hours
49% of founders: 6+ hours

Quality Impact by Role:

PMs: 70%+ report improvements
Founders: 70%+ report improvements
Engineers: 51% better, 21% worse (highest “worse”)
Designers: 60% better, 13% worse

Tool Popularity:

PMs: ChatGPT (57.7%), Claude (#2), Perplexity, Cursor, Lovable
Designers: ChatGPT (49.6%), Claude (#2)
Founders: ChatGPT (72.1%), Claude (#2)
Engineers: Cursor (33.2%), ChatGPT (30.8%), Claude Code (29.0%)

PMF / “Very Disappointed to Lose” Signals:

ChatGPT: 50.2% overall (lower than usage = vulnerability)
Claude ecosystem: 27.5% overall
Cursor: 20.7% among engineers
Claude Code: 17.1% among engineers
Granola: ~2.5x ratio of loyalty to usage among PMs

Agent Adoption:

Active users: ~14%
Interested/planning: 49%
n8n: 219 mentions
Zapier: 85 mentions
Manus: 35 mentions
47% of agent users: 75% non-agentic workflows
Blockers: organizational (8.2% company restrictions), not technical

Downsides (92.4% report at least one):

Generic outputs: 56.2%
Hallucinations: 51.9%
Time managing outputs: 37.7%
Concern about eroding critical thinking: close fourth

Key Quotes:

“The people winning with AI are treating it as a genuine collaborator, one that requires context, iteration, and realistic expectations but that rewards investment with compounding returns.”
“People don’t want AI to do the interesting parts of their job. They want it to do the parts they hate.”
“The roles most transformed won’t be the ones where AI is ‘smartest.’ They’ll be the ones with the most tedious busywork. Follow the drudgery, and you’ll find where AI creates the most value.”
“The people extracting the most value […] have found their AI jobs-to-be-done. Trying to use AI for everything means mastering nothing.”
“The shift from ‘AI helps me write’ to ‘AI helps me decide what to write’ is where the next wave of productivity will come from.”
PRDs: “PRDs that would take a few days now take less than an hour.”
Prototyping: “What used to take me a month to build and validate is now at most a day. Sometimes max one hour.”