Alibaba's Qwen4-Max Coding Claim Is Narrower Than Headlines Suggest
Independent tests put the closed-weight model below the top tier overall, and the Claude Fable 5.1 comparison remains unverified.
Bloomberg reported that Alibaba says Qwen4-Max outscored Anthropic's Claude Fable 5.1 on coding benchmarks. Alibaba's own comparison table, as reproduced by one outlet, uses other rivals and shows a lead on one coding test by under 2 points. Independent evaluators place the model below the leaders overall, though it is cheap per task and competitive on web development.
Alibaba unveiled Qwen4-Max on November 16, 2026, and Bloomberg reported that the company said the model scored near the top of coding benchmarks, ahead of Anthropic's Claude Fable 5.1 and Moonshot's Kimi K3.1 Pro. The evidence available on launch day does not confirm that framing. The comparison table from Alibaba's own announcement, as reproduced by one European outlet, lists other rivals, and the model trails on most agentic-coding rows in it.
The question matters beyond one product launch. Chinese labs are widely seen as the closest competitors to US frontier developers, and benchmark claims are the main way a new model is judged in its first hours. Qwen4-Max is also not open weight, so outsiders cannot download it and inspect it, which changes how much weight a company's own numbers can carry.
What Alibaba actually claimed
According to Bloomberg, Alibaba said Qwen4-Max performed near the top on coding benchmarks and outperformed both Claude Fable 5.1 and Kimi K3.1 Pro. The full Bloomberg text was not available for this article, only its summary lines, so the specific benchmarks and settings behind that sentence are not known.
Several outlets, including Newsbytes and a DEV Community post, repeat the Fable 5.1 comparison. They cite Bloomberg or Alibaba, so they are not independent confirmation. The DEV Community post attributes the coding results to Alibaba's own reporting.
The model's stated design
Alibaba's launch blog describes a 2.4-trillion-parameter backbone, native vision input and a context window of 1 million tokens. News reports relaying Alibaba's press release say the model is a sparse mixture-of-experts design that activates 71 billion parameters per token, which the company puts at 25% fewer than its predecessor, Qwen3.8-Max.
Alibaba also made efficiency claims. Reports of the launch say the average cost per task on Work360, a benchmark Alibaba built itself, fell from $4.43 to $1.67, a 62% reduction, while the score rose from 47.74 to 52.69. Token use on Terminal-Bench 4.0 fell by roughly 48% to 49% compared with Qwen3.8-Max. Alibaba's internal safety review, by its own account, found no risk that the model would give users a significant uplift in capabilities. That assessment is self-reported.
The comparison table in Alibaba's blog
Trending Topics published a reading of Alibaba's blog that reproduces the company's comparison table. It was not checked against the blog image, so it should be treated as a secondary account. The comparators are Anthropic's Opus 5.5, two OpenAI models (GPT-6.1 Sol and GPT-6 Astra) and Moonshot's Kimi K3.1. Fable 5.1 does not appear in the version reported.
| Benchmark | Qwen4-Max | Opus 5.5 | GPT-6.1 Sol | GPT-6 Astra | Kimi K3.1 |
|---|---|---|---|---|---|
| DeepSWE v1.1 (coding) | 75.7 | 74.2 | 75.2 | 74.1 | 75.5 |
| Terminal-Bench 4.0 | 48.5 | 59.6 | 56.1 | 59.1 | 67.2 |
| ProgramBench | 31.5 | 65.0 | 52.5 | 55.0 | 25.0 |
| HLAB-Vals 1.1 (legal) | 21.3 | 9.2 | 6.3 | 7.1 | 11.7 |
Read row by row, the table is mixed. Qwen4-Max leads on DeepSWE and on the legal benchmark, and it trails on Terminal-Bench 4.0 and, except against Kimi, on ProgramBench. Trending Topics concluded that Alibaba's model is strong at legal agent work and long tasks, while Anthropic, OpenAI and Moonshot are ahead in agentic coding. Crypto Briefing also framed the DeepSWE result as ahead of Opus 5.5 and competitive with Kimi K3.1 Pro, without mentioning Fable 5.1.
The single coding row Qwen4-Max wins is narrow. Based on the table, its margin over the other four models ranges from 0.2 points (Kimi K3.1) to 1.6 points (GPT-6 Astra), by simple subtraction. No confidence intervals were reported, so differences that small cannot be separated from noise on the evidence at hand.
Where Fable 5.1 stands
Anthropic released Claude Fable 5.1 on September 1, 2026. According to the evaluator Vals AI, it lists at $10 per million input tokens and $50 per million output tokens, and its weights are private. Vals ranks it first on the Vals Index at 67.87%. The retrieved material contained no Vals Index figure for Qwen4-Max, so that ranking cannot be compared directly.
On the one benchmark where both companies name a number, Anthropic's launch post on X reports Fable 5.1 at 55.8% on Terminal-Bench 4.0, up from 42.0% for the earlier Fable 5. Alibaba's figure for Qwen4-Max on the same benchmark is 48.5. The two scores may come from different harnesses and settings, so the comparison is indicative rather than decisive, but it points the opposite way from the headline claim.
Fable 5.1 is also no longer Anthropic's top model. Anthropic's model documentation lists it as a legacy model, with Fable 5.2 now the current Fable model. A DEV Community post, which is a weak source and unverified against Arena, reports Fable 5.2 at No. 1 on Agent Arena with 28.61% task success, Fable 5.1 at No. 6 with 14.10%, and Qwen4-Max at No. 14 with 7.46%. Those figures need checking against Arena before they are relied on.
If Fable 5.1 appears in a chart in Alibaba's blog that could not be read, the claim could still be supported on some benchmark. Until Alibaba or Bloomberg specifies the benchmark, harness and reasoning-effort setting, the claim stays unverified.
What independent evaluators found within hours
Artificial Analysis: improved, but not at the top
Artificial Analysis scores Qwen4 Max at 43 on its Intelligence Index, above the median of 41 for similarly priced models. The index is version 5.0.1 and combines 9 evaluations, including Terminal-Bench 4.0 and FrontierCode-AA 2.0. Alibaba's launch table does not cite this independent result.
In its thread on X, Artificial Analysis says the model gained 9 points over Qwen3.8 Max and lands 3 points behind Kimi K3.1 Pro at about 11% lower cost per task. It puts the cost at $3.27 per Intelligence Index task, 67% below the $10.02 of Qwen3.8 Max. Trending Topics reports that among the 10 leading models Artificial Analysis compares, a score of 43 is enough only for ninth place, and that Moonshot is the best Chinese developer on that ranking.
For scale, a GPT-6.1 Sol (Max) page on Artificial Analysis shows 55 on the same index revision. No version 5.0.1 score for Fable 5.1 or 5.2 turned up, so a like-for-like Artificial Analysis comparison with Anthropic's models is missing. Earlier Qwen3.8-Max figures of 45 come from an older index version and are not comparable.
Terminal tasks: the weak spot
Artificial Analysis measures Qwen4 Max at 48% on Terminal-Bench 4.0, behind Kimi K3.1 Pro at 67% and ahead of Tencent's Hy4 at 37%. That independently matches Alibaba's own 48.5. A secondary site, Kingy.ai, reports a Vals result of 48.48% (plus or minus 0.88) for Qwen4-Max at its xhigh setting, against 65.15% for Opus 5.5 at max. That is a single secondary source, and it also says Qwen was cheaper per test but slower on the terminal benchmark.
Web development: a real strength, with caveats
Qwen4-Max debuted at No. 6 on Arena's Code Arena WebDev and Image-to-WebDev leaderboards, and both results are preliminary. A Digg summary of Arena posts gives a WebDev score of 1750, up 74 points from Qwen3.8-Max's 1676 and just above Claude Opus 5 (Max) at 1664 and GPT-6 Sol (Max) at 1657. Kingy.ai reports Opus 5.5 (Max) at 1816, well above Qwen4-Max.
Two cautions apply. These Arena figures reached this article through aggregators, not Arena's own pages. And Kingy.ai notes the Qwen frontend row had 1,365 votes versus 3,237 for Opus, so the position may move as votes accumulate. The label on that row also does not establish whether the checkpoint served was the final stable API version.
Cost: cheaper per task, but with a token caveat
Trending Topics, citing Artificial Analysis data, puts Qwen4-Max at $3.27 per task against $10.04 for Opus 5.5. The same article reports that the model used more output tokens than the median in the index run, 127 million versus 78.6 million. Cheaper per task and more verbose per task can both be true, because the price per token is lower. Chinese models are not uniformly cheaper, either: according to a secondary comparison, Kimi K3 costs $15 per million output tokens, more than Claude Sonnet 5 at $10.
How big is the US–China gap?
Estimates range from a few percent to well over half a year, and most of the spread comes from what is being measured. None of the main independent estimates covers Qwen4-Max itself, so they describe the general pattern rather than this model.
Government testing on held-out tasks
The US National Institute of Standards and Technology's Center for AI Standards and Innovation (CAISI) evaluated DeepSeek V4 Pro and found its capabilities lag the frontier by about 8 months. CAISI reported that benchmarks DeepSeek chose itself showed the model roughly on par with leading US systems, while benchmarks absent from DeepSeek's report showed worse results on some reasoning and agent-based tests. It also said DeepSeek nearly matches top US models on math.
That pattern is the closest analogue to the Qwen4-Max question, but it is not proof about Alibaba. CAISI tested a different Chinese model. It is also a US government unit, and a Yahoo Tech article reports expert doubts about its method, including criticism of a price-comparison filter that left only one US model. According to a secondary summary of CAISI's work, DeepSeek's cost per correctly solved task ranged from 53% cheaper to 41% more expensive than the US baseline.
Epoch, Arena and other yardsticks
OfficeChai, relaying Epoch AI, reports that the Epoch Capabilities Index puts the top US model, GPT-6 Astra, at 168.9 against 158.7 for Kimi K3, the best Chinese model. The report says the gap has widened again in the second half of 2026 and that every step up in the global frontier since 2023 has come from a US model. Epoch's own page was not retrieved, and the data pre-date both Kimi K3.1 and Qwen4-Max. Second Talent, a commercial site, says Chinese models trailed by 7 months on average, with a range of 4 to 14 months; that figure should be verified with Epoch directly.
A law-firm summary from Sheppard Mullin says CAISI judged GLM-5.2 in July 2026 to be comparable to a US model released about six months earlier, Opus 4.6. That is a single secondary source. At the other extreme, secondary summaries of the Stanford AI Index 2026 put the gap between Claude Opus 4.6 and China's Dola-Seed-2.0 Preview at only 2.7% on Arena. That measures preference votes on text chat, not coding or agentic work, so it says little about how far apart the models are on hard engineering tasks.
The pattern across these measures is consistent. Preference votes and vendor-chosen benchmarks tend to compress the gap, while held-out, agentic and novel tasks tend to widen it. Qwen4-Max's own results fit that pattern: a narrow lead on one public coding test, a rank of sixth on preliminary web-development voting, and ninth among leading models on a broad index.
What closed weights change, and what they do not
An open-weight model has downloadable parameters that anyone can run and fine-tune. Alibaba did not release Qwen4-Max that way, though it has said smaller versions of the series will be released as open source soon. No date has been given, and the licence for those smaller models is unknown.
Closed weights did not stop outside testing. Artificial Analysis, Vals and Arena all evaluated Qwen4-Max through its API within hours of launch, which is why independent numbers already exist. What closed weights do prevent is inspection of the model itself, checks of training data for benchmark contamination, and exact replication of a vendor's runs. That limitation is analysis rather than a measured finding, but it follows from the access the public has.
The shift matters for the Qwen brand, which became widely known through open releases. The Sheppard Mullin summary says Chinese labs have been drifting toward closed-source releases and reportedly began deliberating in July 2026 on restricting overseas access to leading models. That rests on one law-firm source and the word reportedly. A separate allegation, from an investor blog, that Chinese labs including Qwen ran industrial-scale distillation of US models is unproven, single-sourced, and no Alibaba response was found.
Reading benchmark claims like this one
Several features of the launch show why vendor numbers need discounting. Companies choose which benchmarks to publish, may use custom scaffolds and high reasoning-effort settings such as xhigh, and rarely report error bars. Work360 and a benchmark called Last Secure Code Bench were built by Alibaba and do not resemble public tests.
Pre-launch rumor offers a second lesson. A forecasting site predicted a score of 55 for Qwen4-Max and a screenshot reading 53 circulated for a preview, but Artificial Analysis measured 43. Another site claimed a score of 58 and an earlier release date, which contradicts Artificial Analysis's own page and the November 16 launch. Those claims were excluded here.
Index versions matter too. Artificial Analysis moved from version 4.3.2 to 5.0.1, so scores across versions cannot be compared. Reasoning-effort settings change both scores and costs, which is why a model run at max and one run at xhigh are not on equal footing.
What to watch next
Taken together, the evidence supports a narrower story than the launch headline. Qwen4-Max is a real step up from its predecessor, with a 9-point gain on Artificial Analysis's index and a lower cost per task. It is competitive on web development and appears strong at legal agent work in Alibaba's own table. It is not shown to beat the best US models at coding, and its terminal-task results sit well below Anthropic's and Moonshot's.
The broader picture is months, not years, with the exact figure tied to the yardstick, and with Epoch's data suggesting the distance re-widened in the second half of 2026. The next tests to watch are Arena once votes accumulate, any held-out evaluation of Qwen4-Max, and whether the smaller open-weight Qwen4 models arrive and let outsiders check what the closed flagship cannot.
Frequently asked questions
Did Qwen4-Max beat Claude Fable 5.1 on coding benchmarks?
Bloomberg reports that Alibaba said so, but the claim is unverified. Alibaba's comparison table, as reproduced by Trending Topics, lists Opus 5.5, GPT-6.1 Sol, GPT-6 Astra and Kimi K3.1 instead, and Qwen4-Max trails on most agentic-coding rows. Anthropic's own Terminal-Bench 4.0 figure for Fable 5.1 (55.8%) is above Alibaba's 48.5 for Qwen4-Max, though the harnesses may differ.
Is Qwen4-Max open weight?
No. Alibaba did not release the model's weights, so it cannot be downloaded or self-hosted. The company has said smaller versions of the Qwen4 series will be released as open source soon, without giving a date.
How does Qwen4-Max rank on independent tests?
Artificial Analysis scores it 43 on its Intelligence Index, above the 41 median for similarly priced models but 3 points behind Kimi K3.1 Pro. Trending Topics reports that this is ninth among the 10 leading models Artificial Analysis compares. On Arena's preliminary web-development leaderboard it ranks sixth.
How far behind are Chinese AI models compared with US ones?
It depends on the measure. NIST's CAISI put DeepSeek V4 Pro about 8 months behind the frontier, and a secondary report of Epoch AI data says the best US model leads the best Chinese model on its capabilities index, 168.9 to 158.7. Arena text-chat preference votes showed a gap of only 2.7%, but that is a different kind of measure.
Does a closed-weight model make benchmark results less trustworthy?
Independent testing is still possible, since evaluators ran Qwen4-Max through its API within hours of launch. What outsiders cannot do is inspect the model, check training data for benchmark contamination or replicate a vendor's exact runs. That makes held-out, third-party results more important than the company's own table.
Sources (21)
- Alibaba Unveils New Flagship AI Model in Riposte to US Rivals , Bloomberg (2026-11-16)
- Qwen4-Max: Your New Teammate Has Arrived , Qwen team, Alibaba (qwen.ai) (2026-11-16)
- Alibaba's Giant Qwen4-Max Model Trails Claude, GPT and Moonshot AI , Trending Topics (2026-11-16)
- Qwen4 Max - Intelligence, Performance & Price Analysis , Artificial Analysis (2026-11-16)
- Artificial Analysis on X: Full breakdown of Qwen4 Max's results across the evaluations in the Artificial Analysis Intelligence Index , Artificial Analysis (X) (2026-11-16)
- Qwen4-Max doubles up at No. 6 in Arena's preliminary web-dev rankings , Digg (2026-11-16)
- Alibaba says Qwen4-Max cuts task costs by 60%, boosts cyber safeguards , Business Standard (2026-11-16)
- Alibaba unveils Qwen4-Max, claims it beats rival models on coding , Crypto Briefing (2026-11-16)
- Qwen4-Max Cuts Average Task Cost 62% to $1.67 , Shiftdelete.net (2026-11-16)
- Qwen4-Max vs Claude Opus 5.5: Benchmarks, costs and tests , Kingy.ai
- Qwen4-Max Beat Claude on Coding Benchmarks — Then Alibaba Locked the Weights , DEV Community (2026-11-16)
- Claude on X: We're introducing Claude Fable 5.1 and Claude Mythos 5.1 , Anthropic (X) (2026-09-01)
- Claude Fable 5.1 - Claude Platform Docs , Anthropic
- Claude Fable 5.1 Benchmarks, Cost and Capabilities , Vals AI
- CAISI Evaluation of DeepSeek V4 Pro , NIST Center for AI Standards and Innovation (2026-05-01)
- US Government Says China's Best AI Models Lag Behind. Experts Aren't So Sure , Yahoo Tech (2026-05-01)
- Gap Between US And Open Chinese Models Is Now Among The Largest In Last 2 Years , OfficeChai
- US vs. China AI Models: A Cost, Performance, and Value Comparison [2026] , Second Talent
- Choosing Between U.S. and Chinese AI Models: The Export Control Risks on Both Sides , Sheppard Mullin
- The US-China AI Race: Capability, Cost and Value , Acquinox Capital
- Are Chinese AI Models Really Catching the US Frontier? , MindStudio