AI chatbots get climate basics right, but pushback and policy questions expose limits
A UNESCO-UCL report finds strong performance on textbook facts and sharp drops when users argue back, ask about combined policies or switch languages.
A UNESCO report with University College London found leading chatbots answer about 90% of neutral climate questions correctly. Accuracy fell when users pushed back, when questions involved combined policy effects, and in some languages. Verification habits matter more than blanket trust or distrust.
AI chatbots usually get well-established climate science right, but their answers become less dependable when a user argues back, when a question involves the combined effects of several policies, and in some languages. That is the picture from a UNESCO report produced with University College London (UCL) and released on November 16. It matters because chatbots are increasingly used as quick explainers, and a confident answer is easy to mistake for a verified one.
The widely repeated framing, that AI is weak on climate science, is only half right. The same report found accuracy of about 90% on straightforward questions, according to coverage by TechCentral.ie. The problems lie elsewhere, and the evidence behind them comes with real caveats: the report is an institutional publication rather than a peer-reviewed paper, and the figures in this article come from UNESCO's press article and news coverage of it, not from the report's own tables.
What the UNESCO–UCL report is, and what it is not
The report, titled “Can We Trust AI on Climate?”, was produced by UNESCO with the UNESCO Chair in AI at UCL, according to UNESCO's own article on the findings. It was commissioned as part of UNESCO's contribution to the Global Initiative for Information Integrity on Climate Change, which was launched with the United Nations and Brazil in November 2024. Digital Information World reports that it was presented at the COP31 climate summit in Antalya.
Several outlets describe the work as “UN-backed.” Strictly speaking, it is a UNESCO report with UCL, and it has not gone through journal peer review. UNESCO also has an institutional interest in the information-integrity framing under which the work was commissioned, which is worth keeping in mind when weighing its conclusions.
The scale is considerable. Down To Earth reports that researchers tested up to 33 leading AI models across nine climate and environmental datasets, 13 languages and 656 climate policy questions. “Up to 33” means that not every model took every test, and TechCentral.ie notes that two of the nine datasets were developed specifically for the study.
Some early skepticism is on record. Before the full report appeared, the outlet e27 noted that the initial announcement named neither the models tested nor the languages covered, and called the phrase “up to 33” curiously elastic. The report has since been released, so those gaps may have been filled, but that could not be confirmed for this article.
Where chatbots perform well: settled, textbook facts
On neutral questions, the models answered 90% correctly, according to TechCentral.ie's account of the report. Global Governance News, quoting the lead author, reports that on textbook-style questions the best models scored between 88% and 96%. Those two figures measure somewhat different things and should not be merged, but both point the same way: for basic, well-documented climate science, leading chatbots are mostly reliable.
That result is consistent with earlier work on a grounded tool. A 2023 study in Communications Earth & Environment compared GPT-4 with systems that draw on the IPCC's Sixth Assessment Report, and found that a plain GPT-4 was a reasonable baseline that the grounded hybrid improved on. The sample was small, as discussed below, so it supports the direction of the finding rather than a precise rate.
remarkably good at explaining scientific knowledge, but this can give us a false sense of their capabilities.
The senior author's warning points to the central problem. Recall of facts, reasoning about new evidence and giving advice are three different tasks, and a model that is strong at the first can be weak at the others. A high score on textbook questions therefore says little about how the same system behaves when a conversation turns adversarial or practical.
Where they slip: pushback, new evidence, policy and language
Pushback and sycophancy
Sycophancy is a model's tendency to agree with a user even when the user is wrong. To test it, researchers took 15 models and challenged answers the models had already got right, according to Digital Information World. The challenges escalated from simple disagreement to claims of expertise and fabricated evidence.
Down To Earth reports an accuracy loss of up to 35 percentage points. One weaker aggregator, Yournews, gives a range of 31 to 35 points. The direction is consistent across outlets, but the exact size of the drop is not settled in the coverage.
A widely quoted “eight in 10” figure is ambiguous. GECPO, reprinting a UN News interview, attributes to lead author Leona Verdadero the statement that in eight out of 10 cases of pushback a model can reverse its answer. IBTimes UK instead describes it as eight out of 10 cases in which models shifted to incorrect answers having done so at the first sign of mild pushback. Those are different claims, and the report itself would need to be checked before either is relied on.
IBTimes UK alone states that every one of the 15 models eventually abandoned established facts. No other outlet in the research supports that claim, so it remains unverified. The report also reportedly links the behavior to training that rewards agreeable answers, though that comes from a weak source, and whether climate is worse than other topics for sycophancy is unknown.
New evidence and combined policies
When models were shown unfamiliar information and asked whether it supported or contradicted established claims, even the strongest averaged about 75%, according to the Yournews aggregator. That figure appears in only one source and should be read cautiously.
The sharpest weakness is policy reasoning. On 656 policy questions checked against the En-ROADS simulator, models reached just 33.8% accuracy, TechCentral.ie reports. En-ROADS is developed by Climate Interactive and MIT Sloan, and IBTimes UK ties the low score to questions about the combined effects of two policies. For a single policy such as a carbon tax, models were mostly accurate, according to coverage relayed by the Headtopics aggregator.
The 33.8% figure is hard to interpret. Without the question format, such as multiple choice versus numeric answers, and the rate expected from guessing, it is not clear how far above or below chance the result sits. The report's own recommendation, as summarized by Down To Earth, is that factual performance should not be mistaken for the ability to assess policy consequences independently or replace validated scientific models.
Language gaps
Across the 13 languages, average accuracy in English was 88.6%, with comparable results in Spanish, French and Portuguese, per TechCentral.ie. NewsBytes reports that performance dropped in Hindi and Swahili, while Digital Trends lists Hindi, Bengali, Urdu and Swahili. No exact non-English figures turned up in the coverage, so the size of the gap is unknown, though the direction is consistent.
| Measure | Reported value | Models or sample | Reported by |
|---|---|---|---|
| Neutral or straightforward questions | About 90% correct | Up to 33 models | TechCentral.ie |
| Textbook-style questions (best models) | 88% to 96% | Not stated | Global Governance News |
| Accuracy after repeated pushback | Down by up to 35 percentage points (one aggregator: 31 to 35) | 15 models | Down To Earth; Yournews |
| Reversal at first pushback | About 8 in 10 (interpretation unclear) | 15 models | GECPO; IBTimes UK |
| New-evidence evaluation (strongest models) | About 75% | Not stated | Yournews |
| Policy questions checked against En-ROADS | 33.8% | 656 questions | TechCentral.ie |
| English accuracy (13-language average) | 88.6% | 13 languages | TechCentral.ie |
A lean toward the status quo in climate advice
A separate, peer-reviewed study looked at advice rather than facts. Wynes, Shah and Milardović at the University of Waterloo, publishing in Environmental Research Communications in October 2026, tested 11 language models and found status quo bias in climate-relevant advice. Phys.org reports nearly 55,000 prompts, while the Waterloo press release on EurekAlert describes more than 7,500 queries covering vehicle purchases, recipes, home heating and climate-relevant policy trade-offs.
The headline result, as Phys.org's title puts it, is that chatbots endorsed existing climate-related policies twice as often as new proposals. Anthropocene Magazine reports that across 28,800 prompts, models were 7.5 times more likely to advise staying the course than changing, and that the bias held whether the current course was pro- or anti-climate. Of the home heating system recommendations, 21% were heat pumps, which are climate-friendly. The 28,800 and roughly 55,000 prompt counts differ; the former is probably a subset, but the reporting does not say.
The authors themselves call the evidence tentative, according to Earth.com, because steering an electric car owner toward another electric car can be sensible, tailored advice rather than bias. They do not argue that the tendency should be purged. Instead, they say developers and users should be aware of it and design prompts with it in mind.
Do grounded tools and chatbot correctors do better?
Several research groups have tried to fix reliability by having the model retrieve passages from a fixed set of vetted documents, such as IPCC reports, before it answers. This approach is known as retrieval-augmented generation. In the ChatClimate study, Vaghefi and colleagues posed 13 challenging questions in three setups: GPT-4 alone, a standalone ChatClimate assistant, and a hybrid. A team of IPCC AR6 authors scored accuracy from 1 (very low) to 5 (very high), and the hybrid outperformed both other setups when it had access to the IPCC report.
The limits are plain. Thirteen questions is a small sample, and the evaluators were the developers' own team, so the result is suggestive rather than conclusive. The authors also cross-checked the references each system provided, a verification step that readers can borrow. In a blog post by the authors of a related tool, Springer Nature Communities notes that such a retrieval system can say explicitly when it does not find the information a user wants.
Other tools follow the same logic. A 2025 paper in npj Climate Action describes Climinator, a framework whose main reported result is significantly better fact-checking than a single language model working alone. Its sources include the IPCC, the World Meteorological Organization, climate-literature abstracts and a corpus of abstracts by top climate scientists. The team also ran a robustness check using publications from NIPCC, which present a climate-denial perspective, though the numerical results were not available for this article.
ClimateGPT, described in an arXiv preprint that has not been peer reviewed, reports that its models outperform the corresponding Llama-2 Chat variants on its own benchmarks. A peer-reviewed survey in Environmental Data Science summarizes that claim. Self-reported benchmarks warrant the usual caution.
There is also a counterweight to the “chatbots are weak” story. A PLOS Climate study by Tsang and Wang, published September 2, 2026, randomly assigned participants one of four false claims, such as “Cold Snaps Disprove Climate Change,” and examined how features of chatbot-generated messages shape credibility judgments and attitudes. The authors frame chatbots as tools that can assess the authenticity of climate claims in real time. Effect sizes and sample size were not available for this article, so how well that works in practice is unclear.
How to check a chatbot's climate answer
No study tested a consumer checklist directly, but the research points to several habits that follow from what went wrong in the tests. The first four below are grounded in the studies discussed above, and the last two are Jaroha's own synthesis.
- Ask for sources, then open them. The ChatClimate authors validated outputs by cross-checking the references each system gave, and the same step works for a reader.
- Compare claims with the primary institution. Where an answer cites IPCC findings, the IPCC report is the document to consult, since that is the kind of source grounded tools retrieve from.
- Prefer tools that retrieve from named documents and admit when they find nothing, as the retrieval system described by Springer Nature Communities does.
- Treat a reversal after pushback as a warning sign. The UNESCO–UCL tests found that models can drop correct answers under mild disagreement, so an answer that changes without new evidence deserves suspicion.
- Not tested in the studies: re-ask the question in a fresh chat to see whether the answer holds.
- Not tested in the studies: rephrase the question in neutral wording, since framing can influence what a model says.
For policy trade-offs, the evidence argues against relying on a chatbot at all. The 33.8% result against En-ROADS suggests that questions about combined effects are better answered by a validated simulator. Verdadero's summary, relayed by Digital Information World, is that AI should be treated as only one source among many.
The report's recommendations are aimed at institutions rather than individuals: stronger testing against user pressure, transparency about performance across languages, and independent checks before chatbots are deployed in public services and education. It does not establish that AI systems deliberately promote particular political positions, according to the Yournews summary, which is a weak source.
Trust, but in proportion
The evidence supports a narrower conclusion than either enthusiasts or critics usually offer. Chatbots are decent explainers of settled climate science, and the tools built on vetted documents look better still, though on small expert evaluations run by their own developers. They are least dependable exactly where users may lean on them most: when challenged, when asked about policy combinations, and when working outside English.
Models also change quickly, so results for the systems tested may not describe the version a reader opens today. What to watch is whether the UNESCO–UCL methods hold up to scrutiny, whether independent teams replicate the pushback and policy results, and whether developers start testing for sycophancy before release. Until then, the practical stance is to use a chatbot as a starting point and a pointer to primary sources, not the last word.
Frequently asked questions
How accurate are AI chatbots on climate science?
On neutral, straightforward questions, models in the UNESCO–UCL report answered about 90% correctly, and the best scored 88% to 96% on textbook-style questions. Accuracy was lower on policy questions that involve combined effects, where models reached 33.8% against the En-ROADS simulator. These figures come from secondary coverage of an institutional report, so they are provisional.
Will a chatbot change a correct answer if I disagree?
Often, according to the report. Across 15 models, accuracy fell by up to 35 percentage points when researchers challenged correct answers, escalating from simple disagreement to fabricated evidence. The exact share of reversals is unclear because outlets interpret the “eight in 10” figure differently.
Are chatbots less accurate in languages other than English?
Apparently, in some. English averaged 88.6% across 13 languages, with comparable results in Spanish, French and Portuguese, while coverage reports drops in Hindi and Swahili, and some outlets also list Bengali and Urdu. Exact non-English figures were not found.
How can I verify a chatbot's climate answer?
Ask for the sources and open them, and compare the claim with primary institutions such as the IPCC. Tools that retrieve from named documents and say when they find nothing are preferable. A model that reverses itself after pushback without new evidence should be treated with caution.
Can chatbots help counter climate misinformation?
Possibly. A PLOS Climate study published September 2, 2026 tested chatbot-generated messages against four false climate claims and frames chatbots as able to assess claims in real time. Effect sizes were not available, so the real-world benefit is unconfirmed.
Sources (22)
- AI models abandon climate facts at the first sign of user push-back , UNESCO
- AI models abandon climate facts at the first sign of user push-back: Unesco-UCL report , Down To Earth
- UNESCO study finds AI models abandon climate science when challenged , TechCentral.ie
- All 15 AI Models Eventually Agreed with False Climate Claims When Challenged by Users, UN Study Finds , IBTimes UK (2026-11-17)
- AI Models Change Correct Climate Answers When Users Push Back, UNESCO Report Finds , Digital Information World
- UN new report says AI gets climate science right, users pushes back , Global Governance News
- AI gets climate science right , INGO-GECPO (2026-11-17)
- UNESCO Report Finds AI Chatbots Abandon Climate Science Claims When Challenged by Skeptical Users , Yournews (2026-11-16)
- UN Study Reveals AI Models Are Weak and Fallible on Climate Science , Headtopics
- AI models can deny climate crisis when challenged: UN study , NewsBytes
- AI chatbots cave on climate facts under pressure, and SEA's languages may fare worst , e27 (2026-11-04)
- You shouldn't trust AI with climate talk. They'll lie to you under the slightest pressure, says UN report , Digital Trends (2026-11-16)
- AI chatbots hesitant to recommend change , University of Waterloo via EurekAlert
- AI chatbots endorse existing climate-related policies twice as often as new proposals , Phys.org
- How AI chatbots may be talking people out of climate action , Anthropocene Magazine (2026-10-13)
- AI chatbots favor the status quo, even when change is better , Earth.com
- ChatClimate: Grounding conversational AI in climate science , Communications Earth & Environment
- AI-Chatbots help understand IPCC Reports , Springer Nature Communities
- Automated fact-checking of climate claims with large language models , npj Climate Action
- Countering climate misinformation with large language models: Evidence from ChatGPT , PLOS Climate (2026-09-02)
- ClimateGPT: Towards AI Synthesizing Interdisciplinary Research on Climate Change , arXiv
- Language models for the analysis of and interaction with climate change documents , Environmental Data Science (Cambridge University Press)