Welcome to DU!
The truly grassroots left-of-center political community where regular people, not algorithms, drive the discussions and set the standards.
Join the community:
Create a free account
Support DU (and get rid of ads!):
Become a Star Member
Latest Breaking News
Editorials & Other Articles
General Discussion
The DU Lounge
All Forums
Issue Forums
Culture Forums
Alliance Forums
Region Forums
Support Forums
Help & Search
General Discussion
Showing Original Post only (View all)Last year a study showed experienced software developers were slowed down by AI. This year they balked at being tested. [View all]
The same company, METR, wanted to test them.
My thread about that study, from last July:
AI slows down some experienced software developers, study finds
https://www.democraticunderground.com/10143494159
Link to last year's study: https://arxiv.org/abs/2507.09089
Before starting tasks, developers forecast that allowing AI will reduce completion time by 24%. After completing the study, developers estimate that allowing AI reduced completion time by 20%. Surprisingly, we find that allowing AI actually increases completion time by 19%--AI tooling slowed developers down.
They ran into problems conducting a study this year:
https://metr.org/blog/2026-02-24-uplift-update/
Unfortunately, given participant feedback and surveys, we believe that the data from our new experiment gives us an unreliable signal of the current productivity effect of AI tools. The primary reason is that we have observed a significant increase in developers choosing not to participate in the study because they do not wish to work without AI, which likely biases downwards our estimate of AI-assisted speedup. We additionally believe there have been selection effects due to a lower pay rate (we reduced the pay from $150/hr to $50/hr), and that our measurements of time-spent on each task are unreliable for the fraction of developers who use multiple AI agents concurrently.
-snip-
Our raw results show some evidence for speedup. Our early 2025 study found the use of AI causes tasks to take 19% longer, with a confidence interval between +2% and +39%. For the subset of the original developers who participated in the later study, we now estimate a speedup of -18% with a confidence interval between -38% and +9%. Among newly-recruited developers the estimated speedup is -4%, with a confidence interval between -15% and +9%.
-snip-
Recruitment and retention of developers has become more difficult. An increased share of developers say they would not want to do 50% of their work without AI, even though our study pays them $50/hour to work on tasks of their own choosing. Our study is thus systematically missing developers who have the most optimistic expectations about AIs value.
-snip-
Some developers were less likely to complete tasks that they submitted if they were assigned to the AI-disallowed condition. One developer did not complete any of the tasks that were assigned to the AI-disallowed condition.
-snip-
-snip-
Our raw results show some evidence for speedup. Our early 2025 study found the use of AI causes tasks to take 19% longer, with a confidence interval between +2% and +39%. For the subset of the original developers who participated in the later study, we now estimate a speedup of -18% with a confidence interval between -38% and +9%. Among newly-recruited developers the estimated speedup is -4%, with a confidence interval between -15% and +9%.
-snip-
Recruitment and retention of developers has become more difficult. An increased share of developers say they would not want to do 50% of their work without AI, even though our study pays them $50/hour to work on tasks of their own choosing. Our study is thus systematically missing developers who have the most optimistic expectations about AIs value.
-snip-
Some developers were less likely to complete tasks that they submitted if they were assigned to the AI-disallowed condition. One developer did not complete any of the tasks that were assigned to the AI-disallowed condition.
-snip-
As for what's going on here... I'd guess a lot of developers who like using AI for coding don't want to know if it isn't making them much more productive. Last year's findings were probably an unpleasant shock for them. They might not want to find out just how slow they'd be doing that coding without AI, either - whether other devs being tested could write software faster without AI than they could.
Much easier for them to continue using AI and assuming they're more productive.
And a very bright undergrad student (computer science and AI) at a college in India suggested the developers were addicted to AI.
https://www.linkedin.com/pulse/has-ai-already-crossed-addiction-threshold-developers-aditya-tomar-trvnc
Has AI Already Crossed The Addiction Threshold For Developers?
Aditya Tomar
Published Feb 26, 2026
-snip-
The effect that is obvious? The refusal. The psychological dependence. The I really like using AI! confession from someone paid to help science.
So lets get uncomfortable: Is AI the first widely adopted technology that creates more skilled helplessness than actual skill?
Weve seen this before with calculators and mental math, GPS and spatial reasoning, social media and attention spans. But never at the core of high-leverage intellectual work like software engineering. Never with tools that market themselves as augmenting while quietly eroding the very faculties they claim to enhance.
-snip-
Developers arent choosing AI because the data convinced them. Theyre choosing it because not choosing it now feels like self-harm.
Aditya Tomar
Published Feb 26, 2026
-snip-
The effect that is obvious? The refusal. The psychological dependence. The I really like using AI! confession from someone paid to help science.
So lets get uncomfortable: Is AI the first widely adopted technology that creates more skilled helplessness than actual skill?
Weve seen this before with calculators and mental math, GPS and spatial reasoning, social media and attention spans. But never at the core of high-leverage intellectual work like software engineering. Never with tools that market themselves as augmenting while quietly eroding the very faculties they claim to enhance.
-snip-
Developers arent choosing AI because the data convinced them. Theyre choosing it because not choosing it now feels like self-harm.
And what that student wrote last week about METR's study seems in agreement with this Business Insider article published today:
https://www.businessinsider.com/claude-outages-anthropic-ai-software-engineers-developers-coding-dependance-2026-3
Claude outages lay bare software developers' growing reliance on AI: 'I guess I'll write code like a caveman'
By Hugh Langley and Pranav Dixit
-snip-
Gauresh Pandit, a senior software engineer at Meta, told Business Insider that tools like Claude have quickly become embedded in engineers' day-to-day work. He said that when Claude went down, he turned his attention to non-coding tasks because he believed it might be slower to tackle the coding manually.
"It might not be that the muscle is lost but it feels like it's just simple to use an LLM even for the simplest things now, because it acts like a single button action to get things done," he said, referring to large language models.
"Claude outages hit way harder when you realize you've outsourced half your brain to it," one Redditor posted. Another joked: "I guess I'll write code like a caveman."
-snip-
By Hugh Langley and Pranav Dixit
-snip-
Gauresh Pandit, a senior software engineer at Meta, told Business Insider that tools like Claude have quickly become embedded in engineers' day-to-day work. He said that when Claude went down, he turned his attention to non-coding tasks because he believed it might be slower to tackle the coding manually.
"It might not be that the muscle is lost but it feels like it's just simple to use an LLM even for the simplest things now, because it acts like a single button action to get things done," he said, referring to large language models.
"Claude outages hit way harder when you realize you've outsourced half your brain to it," one Redditor posted. Another joked: "I guess I'll write code like a caveman."
-snip-
4 replies
= new reply since forum marked as read
Highlight:
NoneDon't highlight anything
5 newestHighlight 5 most recent replies
Last year a study showed experienced software developers were slowed down by AI. This year they balked at being tested. [View all]
highplainsdem
Wednesday
OP
I'm not a developer, but I'll say in my job it has honestly probably slowed me down.
meadowlander
Wednesday
#2
Thanks for the reply! Ever since genAI tools became widely available, I've seen this contrast between
highplainsdem
Thursday
#3