Careers

AI Agent Clears 41.3% of Curated Freelance Projects, With Big Caveats

A new benchmark and an OpenAI–Upwork study point in different directions, and neither shows that AI can do most freelance work.

11 min read
Man in glasses and a dark jacket sits on a grey couch, typing on a silver laptop resting on an orange side table.
Image by StartupStockPhotos from Pixabay

On the Remote Labor Index, Anthropic's Claude Fable 5.2 produced client-acceptable work on 41.3% of a curated set of real Upwork projects, up from 2.5% a year earlier. The figure covers a narrow, visually skewed sample, and demand data suggest the effect on freelancers is shifting work, not simply erasing it.

On one benchmark built from real, paid Upwork projects, the best AI agent now delivers work a client would accept about two times in five. According to a Nov. 2026 report by The Decoder, Anthropic's Claude Fable 5.2 reached 41.3% on the Remote Labor Index (RLI), a test from Scale Labs and the Center for AI Safety. When the index launched on Oct. 30, 2025, the top agent, Manus, scored 2.5%.

That jump matters to anyone who sells skills online, but the number is easy to misread. It describes a curated set of 240 projects, not the freelance economy, and it comes from a single benchmark with no independent replication found. A second study released the same week, from OpenAI and Upwork, does not measure completion at all; it examines how client demand is shifting.

What the 41.3% actually measures

The RLI's measure is an automation rate: the share of projects where an AI agent's deliverable is judged at least as good as what the human freelancer delivered, the standard a reasonable client would accept. That is how MLQ describes the grading. One aggregator describes it as the share of projects scoring at least 2 out of 3.

The agents are not chatbots answering a single prompt. They work in a virtual desktop with professional tools such as Blender, GIMP and Audacity. Trained experts open each submission in professional software and compare it with the freelancer's delivered work, and the Center for AI Safety's attempts to automate that grading with AI reviewers have so far failed, according to the research brief.

A narrow slice of the market

The index contains 240 projects drawn from 358 verified Upwork professionals, with a combined value of about $144,000. A typical job paid an average of $632.60 and took just under 29 hours. The projects cover 23 of Upwork's 64 domains, and the index excludes work that needs physical labor, long-term evaluation or direct client interaction.

The mix is also unusual. A secondary report on the paper, from Neurohive, lists the largest categories at launch as video and animation (13%), 3D modeling (12%), graphic design (11%), game development (10%), audio (10%), architecture (7%) and product design (6%). Nearly all are visual or multimedia, so the sample does not mirror the writing, translation, marketing and customer-service work that fills much of the freelance market.

Some details remain unsettled. MLQ says leaderboard scores use 230 private projects, with 10 public, while The Decoder and other coverage say 240. Jaroha's own arithmetic finds that 99 of 240 projects works out to 41.25%, which rounds to 41.3% and fits a 240 denominator, but that is an inference that Scale has not confirmed in the material reviewed.

From 2.5% to 41.3% in a year

At launch in Oct. 2025, Scale reported that the best agents automated 2.5% of the projects. Manus led, earning $1,720 in project value. Neurohive reported that Grok 4 and Claude Sonnet 4.5 followed at 2.1%.

The latest leaderboard, as reported by Cellcog and cross-checked against BenchLM, shows a wide gap at the top. Fable 5.2 sits at 41.3%, and The Decoder says that is more than double the next-best systems, Claude Opus 5.5 and OpenAI's GPT-6 Astra. GPT-5.2 and Gemini 3 Pro both scored under 3%.

RLI automation rate by model, leaderboard as of Nov. 10, 2026 (% of projects)
RLI automation rate by model, leaderboard as of Nov. 10, 2026 (% of projects)
Label Automation rate (%)
Claude Fable 5.2 41.3
Claude Opus 5.5 21.25
GPT-6 Astra 20.83
Claude Fable 5.1 17.92
Gemini 4 Argon 17.08

The path to the top was not smooth. According to AI Weekly, Fable 5 scored about 16% in July 2026, and Fable 5.1 reached 17.9%. The move from 17.9% to 41.3% came in a single model release, not a gradual climb, so the intermediate steps shown in secondary coverage are less reliable than the endpoints.

Where the gains landed

Scale Labs said the biggest improvements were in audio and music, architecture and interior design, and 3D modeling, as relayed by The Decoder and other outlets. Industrial product design and animation remain far from solved. Nearly 50 of Fable 5.2's successful completions were jobs that no agent in the test had solved before.

When the agents fail, MLQ's account of The Decoder's reporting lists amateur-quality output, incomplete or broken files and contradictory designs. That is a single secondary source, but it fits the nature of the test: a deliverable that looks plausible in a screenshot can still collapse when an expert opens it in professional software.

Reasons for caution about the headline

BenchLM says it first recorded the Fable 5.2 RLI rates from Anthropic's system card, and that Scale AI ran the Claude models at default high effort. The figure therefore appears in the vendor's launch material as well as on Scale's board. The research brief could not open the Scale leaderboard page itself, and the leaderboard publishes no confidence intervals, so it is unclear how much of the score is statistical noise.

The sources also do not say why the score jumped, whether from the model itself, the agent setup, tool use or effort level. They do not report what an agent attempt costs compared with a freelancer's fee. Completing a project is not the same as completing it profitably.

Easier jobs, and what human help adds

Upwork ran its own test, called UpBench, which is part of its Human+Agent Productivity Index. According to the company's arXiv preprint, UpBench uses 322 real, economically verified Upwork jobs and tested Gemini 2.5 Pro, OpenAI's GPT-5 and Claude Sonnet 4. The paper has not been peer reviewed, and Upwork wrote it.

These tasks are deliberately the simplest slice of the marketplace. They usually cost under $500 and make up less than 6% of Upwork's gross services volume, and VentureBeat reports that Upwork picked simple, well-defined projects on purpose. That makes the results a different experiment from the RLI's long, multi-hour projects, and the completion rates of the two tests cannot be compared directly.

Upwork's Nov. 13, 2025 press release says collaboration between humans and agents raised completion rates by up to 70% compared with agents working alone. The release does not make clear whether that is a relative gain or a change in percentage points. The examples reported by VentureBeat suggest a relative gain: Claude Sonnet 4 in data science and analytics went from 64% alone to 93% after human feedback, and GPT-5 on engineering and architecture tasks climbed from 30% to 50%. A summary post on X adds that Gemini 2.5 Pro in sales and marketing rose from 17% to 31%.

Human reviewers typically spent about 20 minutes per feedback cycle, per the research brief. Agents alone did best in web development, mobile app development and data science, especially structured computational work. Writing, translation and marketing tasks saw improvements of up to 17 percentage points.

The paper itself flags a limit: the agents were minimal wrappers around foundation models, with no retrieval, web tools or fine-tuning, and the models are older than today's frontier. Upwork also has a commercial interest in showing that people plus AI beat AI alone. In a March 2026 Semafor interview, Upwork's CEO said professionals doing AI-enabled work command 36% premiums in hourly rates, a company claim with no independent verification.

What demand data show about freelancers

The OpenAI–Upwork report, published Nov. 10, 2026, is a demand-side study and should not be paired with the RLI as a second measurement of the same thing. According to MLQ's summary, it matches Upwork job postings against what people do in ChatGPT across 63 work subcategories. In 43 of the 63, tasks common on ChatGPT lost share on Upwork. In the other 20, the pattern ran the other way, and OpenAI said ChatGPT use alone does not explain changes in freelance demand.

The study also finds that AI-related projects carry larger fixed-price budgets and more hours, while posted hourly rates are about the same. The authors read that as clients scoping bigger jobs, not paying more per hour. Writing makes up a far larger share of ChatGPT activity than of freelance requests, while video and animation are a bigger slice of what clients post on Upwork.

Cellcog's summary of the report gives two figures: content writing's share of Upwork postings fell by roughly 31% from the first half of 2025 to the first half of 2026, and AI-related projects had 52% bigger budgets. Both come from one commercial blog and could not be checked against OpenAI's full report, so they should be treated as unverified.

Older evidence points the same way on text work

An independent blog analysis by Bloomberry looked at the 12 most popular Upwork categories from Nov. 1, 2022 to Feb. 14, 2024. Writing jobs declined 33%, translation 19% and customer service 16%, while most of the 12 categories grew. The analysis is not peer reviewed and cannot separate AI substitution from platform or economic changes.

A literature review posted on arXiv concludes that generative AI creates winners and losers, with language-intensive work more exposed to substitution and technical work benefiting from complementarity. One study it cites found that freelancers in AI-exposed writing occupations had 2% fewer jobs per month and 5% lower monthly earnings after ChatGPT's release. The review is a preprint, and the underlying studies were not retrieved for this article.

Why the tests seem to disagree

The RLI says the hardest work for agents is industrial product design and animation, which are visual and multimedia. Demand data say the sharpest declines in postings are in writing, translation and customer service, which are text-based. The two findings answer different questions.

The RLI asks whether an agent can finish a deliverable end to end. Posting data ask whether clients stop requesting a task when a chat assistant makes it easy to do themselves. A task can be hard for an agent to deliver as a polished project and still be easy for a client to handle in a chat window, and the reverse can also hold. One result does not predict the other.

The posting data have their own limits. The OpenAI–Upwork analysis is descriptive: it counts changing patterns in postings, not jobs eliminated by AI, and it does not establish that the pattern holds on other marketplaces or for work inside firms. OpenAI is also both a party to the study and a seller of the AI tool.

Which work looks least exposed today

Two groups of work look least exposed on current evidence, and they are protected for different reasons. The first is work that depends on direct client interaction, ongoing relationships or physical presence. The RLI excluded those, so they are untested, not proven safe.

The second group is complex, multi-file design work, with industrial product design and animation named as the weakest areas for agents. That protection is conditional. The same benchmark shows that audio, architecture and 3D modeling, once distant, began to fall within a single model release, and it carries no guarantee that animation will not follow.

The evidence also cuts against a pure replacement story. More than half of the RLI projects still defeat the best model, and the OpenAI–Upwork pattern is mixed across categories. Even where AI helps, UpBench found that expert human feedback lifted completion rates on simple jobs, which suggests that for now a freelancer who can direct and correct an agent holds a role that the agent does not fill alone.

What to watch next

The year's clearest lesson is about speed, not about any single score. A result of 2.5% in Oct. 2025 looked like proof that agents could not do real work, and by Nov. 2026 that reading was out of date. Any figure published today should be assumed to have a short shelf life.

Three things would sharpen the picture: an independent replication of the RLI, confidence intervals on the leaderboard, and earnings and spending data split by category over the same months as the model releases. Until then, the supportable position is narrow. AI can finish a real share of well-defined, curated projects, a larger share than a year ago, and the effect on freelancers is showing up first as a change in what clients ask for.

Frequently asked questions

What share of freelance projects can AI complete today?

On the Remote Labor Index, the top model, Claude Fable 5.2, produced client-acceptable work on 41.3% of a curated set of 240 real Upwork projects. That is not 41.3% of all freelance work, because the set excludes projects needing physical labor, long-term evaluation or direct client interaction. It is also a single benchmark with no independent replication found.

How fast has AI improved on real freelance projects?

The best RLI score rose from 2.5% at launch on Oct. 30, 2025 to 41.3% in November 2026. Most of that rise came in one step, from 17.9% for Fable 5.1 to 41.3% for Fable 5.2. The endpoints are better documented than the intermediate scores.

Is the OpenAI–Upwork study a measure of what AI can do?

No. It compares Upwork job postings with the tasks people perform in ChatGPT across 63 work subcategories, so it describes shifting demand. In 43 of 63 categories, tasks common on ChatGPT lost share on Upwork, but the study does not count jobs eliminated by AI.

Which freelance tasks does AI still struggle with?

According to Scale Labs, as relayed by The Decoder, industrial product design and animation remain far from solved. Failures typically involve amateur-quality work, incomplete or broken files and contradictory designs. Work needing direct client contact was excluded from the test, so it is untested.

Does human help make AI better at freelance tasks?

Upwork's UpBench study, a company preprint, found that human feedback raised completion rates by up to 70% on simple jobs under $500. Claude Sonnet 4 in data science, for example, went from 64% to 93%. The agents tested were older and minimal, and Upwork has a commercial interest in the result.

Sources (17)

  1. The Remote Labor Index: Measuring the Automation of Work , Scale AI (2025-10-30)
  2. Anthropic's Fable 5.2 completes 41 percent of real freelance jobs, up from 2.5 percent a year ago , The Decoder
  3. Anthropic's Claude Fable 5.2 completes 41% of real freelance jobs on the Remote Labor Index, double the next model , MLQ.ai
  4. Remote Labor Index: Fable 5.2 Hits 41.3%, Claim Checked , Cellcog
  5. Remote Labor Index Leaderboard & Scores — November 2026 , BenchLM (2026-11-10)
  6. Remote Labor Index: Top AI Agents Successfully Complete 2.5% of Freelance Projects , Neurohive
  7. Anthropic's Fable 5.2 Clears 41% of Real Upwork Projects , AI Weekly
  8. UpBench: A Dynamically Evolving Real-World Labor-Market Agentic Benchmark Framework Built for Human-Centric AI , Upwork research team (arXiv preprint) (2025-11-01)
  9. Upwork study shows AI agents excel with human partners but fail independently , VentureBeat
  10. Upwork Human+Agent Productivity Index Reveals Up to 70% Boost in Work Completion from Human and AI Agent Collaboration vs. Agents Working Alone , GlobeNewswire via Manila Times (2025-11-13)
  11. Rohan Paul on X: summary of the Upwork study , X
  12. Upwork's CEO on the AI agents that try to hire human workers , Semafor (2026-03-12)
  13. Work at the Frontier: AI's effect on freelance work , OpenAI (2026-11-10)
  14. OpenAI and Upwork study finds AI projects get bigger budgets and more hours on freelance markets , MLQ.ai
  15. OpenAI and Upwork: How AI Is Changing Freelance Work , Cellcog
  16. The jobs being replaced by AI - an analysis of 5M freelancing jobs , Bloomberry
  17. AI and jobs. A review of theory, estimates, and evidence , arXiv preprint

This article was drafted with the help of AI and reviewed by Jaroha editors before publication.

Share this article

Back to list