According to ZDNet, a new study from Scale AI and the Center for AI Safety reveals that today’s most advanced AI agents can only automate less than 3% of tasks required by freelance workers. The research tested six industry-leading models including Google’s Gemini 2.5 Pro, OpenAI’s GPT-5, and Anthropic’s Sonnet 4.5 across 23 freelance categories like graphic design, CAD, and game development. The study establishes a new benchmark called the Remote Labor Index specifically designed to measure AI’s ability to perform economically valuable freelance work. Manus scored highest with just 2.5% automation rate, followed by Grok 4 and Claude Sonnet 2.5 at 2.1%. The findings directly challenge claims from tech leaders like Anthropic CEO Dario Amodei who predicted AI could replace half of white-collar jobs within five years.
Sponsored content — provided for informational and promotional purposes.
How they tested AI freelancers
Here’s what makes this study interesting: researchers didn’t just ask AI to write emails or generate images. They created a realistic freelance environment where models received project briefs and necessary files, then had to deliver completed work that would actually satisfy a client. The deliverables were manually compared against work done by human freelancers, with researchers asking the key question: would a reasonable client accept this as commissioned work?
They used actual freelance platforms like Upwork to identify real-world tasks and skill requirements. This grounds the benchmark in what people actually get paid to do rather than theoretical capabilities. And they tested across diverse categories – everything from product design to game development. Basically, they tried to simulate what it’s like to hire an AI freelancer for real work.
Why AI still struggles with real work
The results are pretty telling. Even the best model automating less than 3% of tasks shows how far we are from AI replacing human freelancers. But why are these supposedly advanced systems failing so badly?
Look, freelance work requires way more than just technical skills. There’s client communication, understanding nuanced requirements, negotiating changes, managing timelines – all the messy human stuff that AI agents simply can’t handle yet. The study authors note that their benchmark doesn’t even capture many aspects of freelancers’ daily work lives, like communicating with clients. So that 3% figure might actually be generous.
And here’s the thing: most freelance projects require combining multiple skills. A graphic designer doesn’t just create images – they interpret client needs, understand brand guidelines, incorporate feedback, and deliver files in specific formats. Current AI systems are good at individual tasks but terrible at orchestrating the whole process.
What this means for the future
Now, before freelancers celebrate too much, it’s worth noting that AI capabilities are improving rapidly. The full study on arXiv shows these systems are getting better, and tech companies are pouring billions into development. But the gap between current capabilities and the hype is enormous.
Consider the context: nearly 73 million Americans performed freelance work according to a recent survey, representing about 43% of the US workforce. That’s a huge market that AI companies would love to automate. But right now, they’re barely making a dent.
So should freelancers worry? Probably not in the immediate future. The study suggests we’re years away from AI being able to handle the complexity and diversity of real freelance work. But it does show where things are heading – and gives human workers time to adapt and focus on the skills that AI still can’t replicate.
