trailDevelopers' self-reported estimates of AI-driven productivity speedup are unreliable.
Coding benchmark scores and anecdotal reports overestimate real-world AI coding capability.
Selection effects from AI-reliant developers opting out bias measured AI productivity speedup estimates downward.
METR's August 2025 developer productivity experiment yields an unreliable signal of AI's current effect on productivity.
+1
claim · empirical · derivedA factual claim that rests on inference from other evidence rather than direct observation.constitution →claim page ↗︎
Developers' self-reported estimates of AI-driven productivity speedup are unreliable.
unassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →
4depended on by
This claim has not been assessed yet. Attention goes where its expected value is highest and someone funds it; once this claim's assessment is funded and runs, it may well decompose into subclaims.
claimA box is a claim: a single proposition the graph assesses, with its own page and map. Click any claim to centre the map on it.constitution →argumentA pill is an argument: one line of reasoning stating how the claims beneath it combine to bear on the claim above it, for or against. Arguments are not destinations; click their claims to explore.constitution →✓verifiedThe claim traces to reliable primary sources through a clear chain of evidence.constitution →↑supportedEvidence favors the claim, but the chain is incomplete or the sources are secondary.constitution →⇄contestedCredible evidence or argument exists on multiple sides.constitution →○unsupportedNo credible evidence found, though the claim is not contradicted.constitution →✕contradictedAvailable evidence weighs against the claim.constitution →?unknownInsufficient information to assess.constitution →◌unassessedNo current assessment. Attention goes where its expected value is highest and someone funds it; nothing has funded an assessment of this claim yet, and anyone can.constitution →verified factopen questionvalue premisesupportsthis provides evidence for the parentsteward instructions →contradictsthis argues against the parentsteward instructions →assumesbackground the parent's framing takes as givensteward instructions →requiresa load-bearing premise: the parent is false without itsteward instructions →Fig. Detail falls off with distance; every claim is an address.