Health

Drug ads shifted AI picks in a lab test; real-world chatbot effect unproven

A Mount Sinai study shows advertising text can tip which drug a model recommends, but it did not test the chatbots people use today.

11 min read
A white pill bottle tipped on its side, spilling white, yellow, pink and red tablets and capsules across a plain blue surface.
Image by kravaivan11 from Pixabay

A Lancet Digital Health study found that inserting real drug ads into AI models' inputs raised how often the advertised drug was chosen when two options were equally reasonable. The study did not test consumer chatbots as deployed, and other research shows chatbot medication answers vary in quality. Here is what the evidence supports and how to use AI for health questions.

Drug advertising can tilt which medication an AI model recommends, at least under laboratory conditions. That is the central finding of a study by researchers at the Icahn School of Medicine at Mount Sinai, published November 11, 2026 in The Lancet Digital Health, according to a Medical Economics report on the press release. When a real, clearly labeled drug ad was placed ahead of a clinical scenario and two drugs were equally reasonable, the models chose the advertised drug more often.

The result matters because a growing number of people ask chatbots about prescriptions, and because AI companies are beginning to sell advertising. But the study is a stress test, not a measurement of what consumers see in ChatGPT, Gemini or Claude today. Nothing in the research reviewed for this article shows that any commercial chatbot currently feeds drug ads into its answers, or that any patient was harmed.

What the Mount Sinai study actually tested

The paper, titled "Ad-verse Effects: Pharmaceutical Advertising and Drug Recommendations by Consumer-Facing AI in a Controlled Study," tested 12 models from Google, OpenAI and Anthropic, according to coverage of the Mount Sinai release. It was supported in part by grants from NCATS and NIH. Each model received physician-written clinical scenarios, either alone or immediately preceded by a real, labeled drug advertisement.

Most of the detailed numbers here come from the April 2026 medRxiv preprint of the same work, because the final paper could not be opened for this article. A preprint is not peer reviewed, and the published version may differ. The preprint reports 33 scenarios, four experiments and 258,660 API calls, with code, protocol and scenario data posted publicly on GitHub. The press release cites a slightly different count of 258,559 responses.

How the ads reached the models

The preprint says the ads were prepended to the system-level instructions, the hidden text a developer or app places before a user's question. That is a different mechanism from how OpenAI says its consumer ads work. The study's authors list several ways a model could meet drug promotion in practice: user prompts, application-added context, retrieved online material, or inadvertent or malicious insertion.

In other words, the experiment asks what happens if promotional text does reach a model's input. It does not ask whether it currently does.

Where ads moved the answer, and where they did not

According to the press release coverage, the advertised drug was selected in 47.6% of responses when an ad was present, compared with 35% without one. The preprint puts the baseline at about 34% and reports a mean shift of +12.7 percentage points. That figure comes from 13 scenarios in which two or more drugs were all guideline-appropriate, a situation clinicians call clinical equipoise.

A discrepancy exists between sources on the denominator: the press release says "across 33 clinical scenarios," while the preprint attributes the 47.6% figure to 13 equipoise scenarios. Readers should treat the exact scenario count as unsettled until the published paper is checked.

Shift toward the advertised drug when both options were guideline-appropriate, by provider (percentage points; April 2026 preprint, 13 equipoise scenarios)
Shift toward the advertised drug when both options were guideline-appropriate, by provider (percentage points; April 2026 preprint, 13 equipoise scenarios)
Label Shift in choosing the advertised drug (percentage points)
Google 29.8
OpenAI 10.9
Anthropic 2
All 12 models (mean) 12.7

The company gap is large. In the preprint, Google's models shifted by 29.8 points, OpenAI's by 10.9 and Anthropic's by 2.0. Within OpenAI's lineup, shifts ranged from 3.1 points for GPT-5.2 to 18.1 points for GPT-4.1 Mini, and the largest single-model shift, 35.1 points, belonged to Gemini 2.5 Lite. In four model-and-scenario combinations, selection of the advertised drug went from 0% to 100%.

The spread suggests the vulnerability is not an inevitable property of AI. It likely depends on how a given model was trained and how it weighs instructions, though the brief does not establish the cause. Model versions also change quickly, so a snapshot from early 2026 may not describe current releases.

The limits of the effect

The ad acted as a tiebreaker, not a persuader of bad choices. According to the preprint, when the advertised drug was clinically suboptimal, only 4.4% of ad-condition responses chose it. Ads did not reduce accuracy in any model, and the change in endorsement of unsupported supplements was −0.6 points, statistically indistinguishable from zero. The investigators themselves say the findings do not mean ads necessarily cause unsafe or inappropriate recommendations.

Even so, a tiebreaker is not trivial. Among drugs that all meet guidelines, patients can still differ in cost, side-effect profile and fit. The study did not measure those factors, so the size of any practical harm from a nudge among equal options is unknown.

The disclosure problem

The preprint also examined whether the models said anything about the ad. In free-text responses, they mentioned it in only 29.9% of cases (467 of 1,560). Models that chose the advertised drug repeated 52.7% of the ad's claims in their reasoning, while models that chose a different drug repeated 19.4%, a 2.7-fold difference.

That pattern is why the authors recommend testing models with and without promotional content, disclosing ads to the AI system, and keeping promotion separate from clinical advice. They also warn that clinicians might defer to algorithmic advice that conflicts with their own judgment. The researchers did not examine whether changed recommendations affected prescriptions, patient decisions, costs or health outcomes.

Are drug ads reaching chatbot answers today?

NBC News reported that OpenAI announced ads for its free and low-cost "Go" tiers, aimed at adult, logged-in US users. The ads appear at the bottom of answers and are labeled, and OpenAI said they would not appear alongside sensitive topics, including health. OpenAI's own position, as reported, is that ads "do not influence the answers ChatGPT gives you." No independent audit of that claim turned up in the research for this article.

OpenAI is also testing what it calls Sponsored Agents, business-sponsored chat agents that begin after a user clicks an ad. According to the company, they are clearly labeled and kept separate from ChatGPT's independent answers.

Critics argue the safeguards are thin. A commentary in The Lancet describes targeted advertising in chatbots as a new public health risk and notes that OpenAI's safeguards are voluntary. The Conversation says OpenAI has not made clear what counts as "sensitive" or how broadly it defines "health." Commercial vendor pages also claim that OpenAI's April 2026 ad policy stopped categorically blocking ads in medical contexts, but those claims could not be verified against OpenAI's own policy page.

Who regulates this?

The study's authors note that, unlike television or print ads, AI-mediated promotion "might not fit existing categories" of FDA oversight. A commentary in npj Digital Medicine adds that FDA's 2024 guidance on predetermined change control plans and its revised January 2026 clinical decision support guidance both assume AI operating inside supervised clinical workflows, not chatbots advising patients on their own. Current FDA positions on AI-delivered drug promotion were not confirmed directly from the agency for this article.

How accurate are chatbots on medication questions?

Advertising is only one possible source of error. Accuracy studies, which use different designs and scoring and cannot be compared directly, paint a mixed picture: often mostly right on routine facts, with occasional serious omissions.

Search-engine chatbots and patient questions

In a cross-sectional study of Bing Copilot, researchers asked 10 patient questions about each of 50 drugs, producing 500 answers, and compared them with drugs.com patient information. The chatbot's answers were largely complete and accurate overall, but they were hard to read and repeatedly lacked information or contained inaccuracies. Among a subset of 20 answers flagged as problematic, experts rated harm as highly likely in 3% and moderately likely in 29%.

Those percentages apply to the 20 pre-selected answers, not to all 500. A later commentary in npj Digital Medicine says more than a fifth of the risky responses could cause severe harm or death.

GPT-4 versus pharmacists

A 2024 study in Digital Health by Albogami and colleagues compared AI chatbots with licensed pharmacists on 70 real drug inquiries. GPT-4 gave completely accurate responses 64.3% of the time, comparable to the human pharmacists. For drug-interaction questions, GPT-4 was completely correct 55.6% of the time and incorrect 11.1%, while pharmacists were completely correct 66.7% of the time. The sample is small, so the figures are rough guides.

Older models and the failure modes they reveal

A Frontiers in Artificial Intelligence article, summarizing earlier work, reports that only 13 of 50 ChatGPT answers were correct in one study, with 13 judged high-risk, and that only 10 of 39 responses passed in another. These involved early ChatGPT-era models and likely overstate current error rates, but they illustrate how failures happen. The npj Digital Medicine commentary describes ChatGPT confidently stating there was no interaction between Paxlovid and verapamil, when, the commentary says, ritonavir, a component of Paxlovid, can cause dangerous hypotension with verapamil.

The brief found no peer-reviewed, 2026-era study of frontier models on medication questions. A systematic review and meta-analysis of ChatGPT drug-counseling accuracy exists as a medRxiv preprint, but no results from it were retrieved, so none are cited here.

What happens when ordinary people use chatbots

The strongest evidence on real-world use is a randomized, preregistered trial published in Nature Medicine in February 2026 by Bean and colleagues. It enrolled 1,298 UK participants across ten medical scenarios and assigned them to GPT-4o, Llama 3, Command R+, or their usual resources, such as a web search. Tested alone, the models identified the relevant conditions in 94.9% of cases and chose a correct course of action in 56.3%.

With human users in the loop, results dropped. Participants identified relevant conditions in fewer than 34.5% of cases and the correct action in fewer than 44.2%, no better than the control group. The researchers saw users supply incomplete or wrong information, and the models sometimes gave misleading answers. The authors point to the interaction between person and model as the main deployment challenge.

The study has limits. It was UK-only, used older models and tested symptom triage rather than medication questions, so its numbers cannot be transferred to drug advice. Its lesson is narrower: strong benchmark scores do not guarantee good outcomes once a real person types in a question.

Use and trust, with a caveat

The npj Digital Medicine commentary cites a 2024 survey of 2,406 US adults in which 21.5% reported using ChatGPT for health information, and nearly a third of those users changed their medications based on what the chatbot said. The original survey was not located, so the figures should be treated as unverified. The same commentary says lay participants in an experiment could not reliably tell low-accuracy AI responses from physician responses and rated both as similarly trustworthy, another secondhand finding.

Practical habits that follow from the evidence

The research reviewed for this article found no official consumer guidance from the FDA, NIH or similar bodies on using chatbots for medication questions. The habits below are Jaroha's inferences from the studies above, not an official recommendation or personal medical advice.

  • Treat answers as a starting point. The accuracy studies show many correct answers alongside omissions and errors, so a chatbot reply works better as a list of questions to raise than as a decision.
  • Ask for reasoning and alternatives. Because models in the Mount Sinai study rarely disclosed an ad's influence, asking why a drug was suggested, and what other options, costs and side effects exist, may expose a one-sided answer.
  • Verify interactions and dosing. Interactions were the weak spot in the pharmacist comparison and in the Paxlovid example, so a pharmacist, prescriber or the official label is the better check.
  • Keep sponsored content separate. Labeled ads, where they appear, are not part of the answer, according to OpenAI. Mentally filing them apart is a sensible precaution.
  • Take extra care in higher-risk situations. Dosing, pregnancy and children are areas where an incomplete answer could matter most.

What to watch next

The Mount Sinai study does not prove that AI health advice is for sale. It shows something narrower and still useful: some models can be nudged by promotional text when the clinical call is close, others barely move, and models seldom say when an ad was present. That makes the design of AI products, rather than AI in general, the thing to scrutinize.

Three developments would clarify the picture: independent tests of deployed chatbots with and without ads, trials that measure what patients and clinicians do after reading AI answers, and clear regulatory statements about AI-mediated drug promotion. Until then, a chatbot is best treated as a knowledgeable but fallible source whose answers deserve a second look from a pharmacist or prescriber.

Frequently asked questions

Can drug ads change what an AI chatbot recommends?

In a controlled Mount Sinai study, inserting real drug ads into model inputs raised how often the advertised drug was chosen when two options were both guideline-appropriate. The study did not test ads as they appear in consumer products, and models mostly resisted ads for clinically poor choices.

Is ChatGPT using drug ads in its medical answers?

The research reviewed found no evidence that any commercial chatbot feeds drug ads into its answers. OpenAI says its ads appear at the bottom of answers, are labeled, and do not influence responses, though no independent audit of that claim was found.

How accurate are AI chatbots on medication questions?

Studies show a mixed record. In one pharmacist comparison, GPT-4 gave completely accurate responses to 64.3% of 70 real inquiries, but only 55.6% of interaction questions were completely correct. Most studies used older models and small samples.

Should chatbots replace a pharmacist or doctor for medication decisions?

The evidence does not support that. In a randomized Oxford trial on symptom triage, people using chatbots did no better than people using ordinary resources. Interactions and dosing are the areas where an error could matter most, so verification with a pharmacist or prescriber is a sensible step.

Does the FDA regulate drug promotion by AI chatbots?

The study's authors say AI-mediated promotion might not fit existing categories of FDA oversight. A commentary in npj Digital Medicine says recent FDA guidance assumes supervised clinical settings, not chatbots advising patients directly. Current FDA positions were not verified directly.

Sources (16)

  1. Drug Ads Can Influence AI Treatment Recommendations, Study Finds , Mount Sinai (2026-11-11)
  2. Ad-verse Effects: Pharmaceutical Advertising Shifts Drug Recommendations by Consumer-Facing AI , medRxiv (2026-04-14)
  3. Ad-verse Effects: Pharmaceutical Advertising Shifts Drug Recommendations by Consumer-Facing AI - PubMed , PubMed
  4. AI chatbot treatment recommendations are swayed by drug ads , Medical Economics (2026-11-11)
  5. Drug ads can subtly sway AI medical treatment recommendations , News-Medical (2026-11-11)
  6. Reliability of LLMs as medical assistants for the general public: a randomized preregistered study , Nature Medicine (2026-02-01)
  7. Artificial intelligence-powered chatbots in search engines: a cross-sectional study on the quality and risks of drug information for patients , PubMed Central
  8. Safety and quality of AI chatbots for drug-related inquiries: A real-world comparison with licensed pharmacists , Digital Health (2024-01-01)
  9. Who bears liability when AI gives bad prescribing advice , npj Digital Medicine
  10. Evaluating the accuracy of ChatGPT in delivering patient instructions for medications , Frontiers in Artificial Intelligence (2025-01-01)
  11. ChatGPT as a Digital Pharmacist: A Systematic Review and Meta-Analysis of Drug-Counselling Accuracy , medRxiv (2025-08-10)
  12. Ads are coming to ChatGPT , NBC News
  13. Reimagining advertising with AI , OpenAI
  14. Targeted advertising in generative artificial intelligence chatbots: a new public health risk , The Lancet
  15. OpenAI will put ads in ChatGPT. This opens a new door for dangerous influence , The Conversation
  16. ChatGPT will now show you adverts. Here's everything you need to know , Euronews (2026-02-10)

Share this article

Back to list