AI chatbots can be brilliant at explaining concepts in plain English. The problem is they can also sound confident and clinical when they’re guessing, and that’s starting to show up in real emergency presentations.
One U.S. surgeon, Dr Darren Lebl, told the New York Post he’s seeing patients “challenge their doctors” with information from AI and said a portion of those “recommendations” are simply made up. Worryingly, a recent NPJ Digital Medicine study also found medical “safety messaging” is appearing less often in chatbot responses over time with disclaimers dropping from 26.3% (2022) to 0.97% (2025) in their analysis.
Here are four real-world cautionary tales with the kind of sticky headlines that make the lesson hard to ignore.
Four AI Horror Stories
The Rubber-Band “Fix”
A man asked an AI chatbot about a cauliflower-like growth near his anus. The chatbot suggested a haemorrhoid-type approach involving ligation and he attempted a DIY version at home using thread. He ended up in the hospital in severe pain. Doctors later confirmed it wasn’t a haemorrhoid at all, but a large genital wart.
The Pool-Chemical “Salt Swap”
A 60-year-old wanted to reduce salt. He reportedly took AI-influenced guidance too literally and replaced table salt (sodium chloride) with sodium bromide — a chemical associated with industrial uses. After three months, he was hospitalised with bromide toxicity (“bromism”), including confusion and neuropsychiatric symptoms. The case was published in Annals of Internal Medicine: Clinical Cases.
The “Wait and See” Mini-Stroke
A patient developed double vision after a cardiac procedure and asked an AI chatbot for reassurance. The response downplayed urgency. He delayed seeking care and later experienced a transient ischaemic attack (TIA), a mini-stroke that requires urgent assessment and prevention strategies. The case was documented in a peer-reviewed report.
The “Suicide Coach” Allegation
In the U.S., a wrongful-death lawsuit alleges a chatbot engaged with a 16-year-old over months, validated suicidal thoughts, and provided self-harm details. The case has intensified calls for stronger guardrails and better escalation pathways in AI mental health interactions.
Use AI Like This (Safe), Not Like That (Risky)
AI can help you:
- write a clear symptom timeline to take to your GP/clinician
- generate a list of questions for your appointment
- translate medical terms from reputable sources into plain language
AI should NOT be used to:
- decide whether symptoms are urgent
- recommend medications, dosages, “substitutes,” or home procedures
- reassure you to “wait” when symptoms are new, severe, or worsening
When to Seek Urgent Care Now
Seek urgent medical attention or call 000 in Australia, if you or someone you’re with has:
- chest pain, severe shortness of breath, collapse/fainting
- stroke/TIA symptoms (face droop, arm weakness, speech or vision changes)
- severe allergic reaction, severe bleeding, sudden severe pain
- confusion, severe dehydration, or rapid deterioration
Find your closest Care 24-7 Emergency Department