There’s one rule of the internet that everybody learns the hard way: never Google your symptoms. The results will either diagnose that you have cancer or something worse. It's hardly ever true.
More importantly, don’t take medical advice from an AI summary sitting at the top of your search page. Google’s AI Overviews is as hallucinative as every other large language model out there, and it barely gets the basics right.
It’s no wonder most people shut it off. In a recent investigation, the tool was caught delivering dangerously oversimplified medical advice, specifically about liver function tests. Hopefully, you didn't take it to heart.
AI Overviews thinks liver test ranges are universal
Google’s AI Overviews gave incorrect medical advice about liver function tests to users. The Guardian, who launched an investigation, discovered the error.
The tool summarized that there is one fixed, universal “normal range” for liver blood test results when they searched “what is the normal range for liver blood tests?”.
Specifically, it showed similar numbers for the following factors:
- ALT normal range: shows liver cell injury.
- AST normal range: shows liver or muscle damage.
- ALP normal range: shows bile duct or bone issues.
- Bilirubin normal range: shows how well the liver removes waste.
- Albumin normal range: shows how well the liver makes essential proteins.
- Total protein normal range: shows overall protein production and immune status.
These ranges were taken from a Max Healthcare website, and must've been taken out of context. Regardless, they're scientifically false for the query the AI was supposed to answer.
Liver test ranges change depending on the following conditions:
• Age.
• Sex.
• Ethnicity.
• Medical history.
• Pregnancy.
• Medications.
• Underlying conditions.
The AI made it sound like everyone should fall within the same numbers. It could lead someone with serious liver disease to think they’re healthy and avoid getting medical help.
Experts called it dangerous and oversimplified. If you rephrase the question and type something like “LFT reference range” instead of “normal liver test range”, the AI still gives the same wrong advice.
Google denies claims of removing sketchy AI Overviews
It seems Google removed AI Overviews for the specific medical searches that were proven to be dangerous. They no longer appear when you search for them.
However, in a PR-style response to The Guardian, the company said they will not be commenting on the incident, and that they trust in their AI.
We do not comment on individual removals within Search. In cases where AI Overviews miss some context, we work to make broad improvements. We also act under our policies where appropriate.
Google
The company has also said its medical experts reviewed the disputed AI summaries. To them, they were not inaccurate for some topics like cancer and mental health.
All there is to learn from this is to see your doctor for any health concerns. There is only so much a device in hand can do for you. Replacing trained medical expertise is not one of them.
