Mitigation and System Design
Engineering practices to reduce or contain hallucinations, including constrained generation, structured reasoning, agent loops, prompt techniques, context design, and human review workflows.
34%
Best tweets about AI Hallucinations
50 Best Tweets About AI Hallucinations (2026)
Explore the best tweets about AI hallucinations, covering causes, evaluation, grounding, RAG, verification, model behavior, and mitigation techniques.
Concrete examples and research about model hallucinations, factuality, evaluation, grounding, verification, and mitigation.
- Creators
- 44
- Updated
Original Xholic analysis
What 50 top AI Hallucinations posts reveal
The discussion emphasizes practical reliability measuresāgrounding, verification, abstention, and repeatable evaluationāalongside posts describing or alleging real-world harms. Retrieval is commonly presented as a mitigation layer rather than a guarantee, and posts disagree about the extent to which recent models have reduced hallucinations.
- Dominant tone
- Negative
- Median score
- 14.0
- Leading format
- Announcement
- Recent posts
- 40%
40% of posts
All-time engagement
40% of posts
Published in 90 days
Conversation map
The themes creators return to
Retrieval Grounding and Verification
RAG, web search, source ranking, citations, retrieval quality, and external evidence as methodsāand limitationsāfor grounding answers.
24%
Real-World Harms and False Information
Concrete incidents where fabricated model outputs enter medicine, science, business, law, courts, and cybersecurity, creating real-world harms and accountability issues.
22%
Evaluation and Benchmark Reliability
Research on hallucination rates, factuality benchmarks, flawed benchmark design, multimodal mirage effects, and contamination of safety evaluations.
14%
LLM Evals and Observability
Continuous evaluation infrastructure: golden datasets, test harnesses, CI/CD checks, production monitoring, regression testing, and adversarial tests for LLM applications.
12%
Mechanisms and Nature of Hallucinations
Why hallucinations occur in next-token prediction systems, including probabilistic generation, missing context, memory limitations, and the distinction between creativity and factual reliability.
12%
Medical and Multimodal Hallucinations
Medical and vision-model hallucinations, including fabricated image interpretation, shortcut learning, anatomical grounding, and high-stakes diagnostic risk.
12%
Uncertainty and Self-Awareness
Uncertainty calibration, self-awareness, abstention, metacognition, and models recognizing when they may be wrong.
12%
Tone and stance
Sentiment Negative leads
Author posture Supportive leads
Performance benchmark
- Median likes
- 29
- Median reposts
- 5
- Median replies
- 5
- Median views
- 2.5K
Posts with media make up 64% of this collection. Their median all-time score is 20.4, compared with 3.94 for text-only posts.
Format mix
- Announcement 40% Ā· score 20.0
- Tutorial 36% Ā· score 11.2
- Opinion 20% Ā· score 12.1
- Question 4% Ā· score 1.38
Consensus and debate
Where creators agree, and where they do not
Shared view
Reliability is framed as a systems-design task
Several posts present hallucination mitigation as a layered system-design problem, combining retrieval, constrained or citation-backed responses, abstention when evidence is insufficient, and ongoing evaluation.
Shared view
Falsehoods can enter knowledge pipelines
Posts describe or allege fabricated material entering research and professional workflows, including a fake medical condition cited by AI systems, claimed hallucinations in conference papers, and fabricated biomedical references.
Open debate
How prevalent hallucinations remain is contested
One post says recent models rarely hallucinate, while other posts characterize hallucination and weak self-knowledge as continuing concerns. These posts do not establish a single rate across models or tasks.
Open debate
Grounding may help, but does not guarantee truth
Posts portray RAG and web search as ways to mitigate fabricated claims, while also reporting that errors can remain with search and that retrieval may provide the problematic context. Retrieval quality is therefore a recurring concern.
What performs
Patterns behind standout posts
Top-scoring outliers span harms and mitigation proposals
The five deterministic score outliers cover a reported medical-misinformation incident, RAG system design, a multimodal-benchmark critique, claimed paper hallucinations, and a prompting technique. Attention in these outliers spans both reported harms and proposed mitigations.
Instructional mitigation content is prominent
Tutorial posts comprise 36% of the set, and Mitigation and System Design is the largest deterministic theme at 34%. The cited tutorials discuss retrieval, structured reasoning, and evaluation workflows.
Statistical standouts
- View standout post 1 Score 12730.9 Ā· 908.05Ć median
- View standout post 2 Score 2934.2 Ā· 209.29Ć median
- View standout post 3 Score 1899.0 Ā· 135.45Ć median
- View standout post 4 Score 1402.3 Ā· 100.02Ć median
- View standout post 5 Score 1051.7 Ā· 75.01Ć median
Creator landscape
Who shapes this conversation
The five most represented creators account for 20% of the selected posts.
-
1. Auren Hoffman
@auren
2 posts
-
2. Jo Peterson
@cleartechtoday
2 posts
-
3. Glenn Gabe
@glenngabe
2 posts
-
4. Haider.
@haider1
2 posts
-
5. Nav Toor
@heynavtoor
2 posts
-
6. Towards Data Science
@TDataScience
2 posts
Separate creative generation from verification
Auren Hoffman argues for separating ideation from verification: allow one component to generate novel ideas and use another to check claims.
High-stakes settings demand skepticism
Nav Toor highlights reported risks in medical, legal, research, coding, and image-based use cases, including claims that models can provide confident answers when benchmark images are absent.
How this analysis was made
Themes, sentiment, stance, and post format are classified per tweet. All counts, shares, medians, creator concentration, freshness, and performance comparisons are then calculated directly from the published snapshot.
Xholic's all-time score compares engagement while accounting for reach, post age, and creator consistency. It is used for relative comparisons within this collection.
This report analyzes the exact 50-post snapshot shown below. AI identifies editorial categories and drafts explanations; all statistics are calculated from the snapshot, and every narrative claim is checked against cited posts before publication.
Best AI Hallucinations tweets
Top AI Hallucinations tweets from 44 creators
Ranked 01ā50
- 01
@HedgieMarkets Ā·
š¦A researcher invented a fake eye condition called bixonimania, uploaded two obviously fraudulent papers about it to an academic server, and watched major AI systems present it as real medicine within weeks. The fake papers thanked Starfleet Academy, cited funding from the Professor Sideshow Bob Foundation and the University of Fellowship of the Ring, and stated mid-paper that the entire thing was made up. Google's Gemini told users it was caused by blue light. Perplexity cited its prevalence at one in 90,000 people. ChatGPT advised users whether their symptoms matched. The fake research was then cited in a peer-reviewed journal that only retracted it after Nature contacted the publisher. My Take The researcher made the papers as obviously fake as possible on purpose. The AI systems didn't catch it. Neither did the human researchers who cited it in real journals, which means people are feeding AI-generated references into their work without reading what they're actually citing. I've covered the FDA using AI for drug review, the NYC hospital CEO ready to replace radiologists, and ChatGPT Health launching this year. All of that is happening in the same environment where a condition funded by a Simpsons character and endorsed by the crew of the Enterprise was being presented as emerging medical consensus. The people making these deployment decisions seem to believe the pipeline from research to AI to patient is more supervised than it actually is. This experiment suggests it isn't supervised much at all. Hedgieš¤ https://t.co/8Kg8FOrgHW
- 02
@adxtyahq Ā·
ādesign a RAG pipeline for 10M docs with zero hallucinationā apparently this was asked in a Google L5 interview round. came across it somewhere on the internet and honestly itās a way more interesting system design problem than most classic distributed systems questions 1. ingest + normalize docs - remove duplicates, standardize formats, extract metadata, maintain version history 2. hybrid retrieval (BM25 + embeddings) - BM25 handles exact keyword matching while embeddings capture semantic meaning - semantic search alone usually struggles with precision at massive scale 3. ANN retrieval + reranking - ANN (Approximate nearest neighbor ) quickly pulls top candidate chunks from millions of docs - then a reranker rescoring step improves relevance by deeply comparing query vs retrieved chunks 4. source confidence scoring - every retrieved chunk gets scored based on freshness, trust level, overlap and retrieval consistency - low-confidence context should never heavily influence generation 5. constrained generation - the model is only allowed to answer using retrieved context (nothing new to be invented outside of the retrieved context) 6. citation-backed responses - every major claim links back to exact chunks, documents or timestamps 7. hallucination fallback layer - if retrieval confidence drops below a threshold: āinsufficient evidence foundā 8. continuous evals - run adversarial queries, retrieval recall benchmarks and hallucination tests continuously 9. caching + memory layer - cache high-frequency enterprise queries and retrieval paths (improves latency and output) 10. observability everywhere - trace retrieval paths, chunk rankings, token attribution and failure points Also at 10M docs, retrieval quality matters more than the frontier model itself.
- 03
@heygurisingh Ā·
Holy shit... Stanford just proved that GPT-5, Gemini, and Claude can't actually see. They removed every image from 6 major vision benchmarks. The models still scored 70-80% accuracy. They were never looking at your photos. Your scans. Your X-rays. Here's what's really going on: ā The paper is called MIRAGE. Co-authored by Fei-Fei Li. They tested GPT-5.1, Gemini-3-Pro, Claude Opus 4.5, and Gemini-2.5-Pro across 6 benchmarks -- medical and general. Then silently removed every image. No warning. No prompt change. The models didn't even notice. They kept describing images in detail. Diagnosing conditions. Writing full reasoning traces. From images that were never there. Stanford calls it the "mirage effect." Not hallucination. Something worse. Hallucination = making up wrong details about a real input. Mirage = constructing an entire fake reality and reasoning from it confidently. The models built imaginary X-rays, described fake nodules, and diagnosed conditions -- all from text patterns alone. But that's not the scary part. They trained a "super-guesser" -- a tiny 3B parameter text-only model. Zero vision capability. Fine-tuned it on the largest chest X-ray benchmark (696,000 questions). Images removed. It beat GPT-5. It beat Gemini. It beat Claude. It beat actual radiologists. Ranked #1 on the held-out test set. Without ever seeing a single X-ray. The reasoning traces? Indistinguishable from real visual analysis. Now here's what should terrify you: When the models fake-see medical images, their mirage diagnoses are heavily biased toward the most dangerous conditions. STEMI. Melanoma. Carcinoma. Life-threatening diagnoses -- from images that don't exist. 230 million people ask health questions on ChatGPT every day. They also found something wild: ā Tell a model "there's no image, just guess" -- performance drops ā Silently remove the image and let it assume it's there -- performance stays high The model enters "mirage mode." It doesn't know it can't see. And it performs BETTER when it doesn't know it's blind. When Stanford applied their cleanup method (B-Clean) to existing benchmarks, it removed 74-77% of all questions. Three-quarters of "vision" benchmarks don't test vision. Every leaderboard. Every "multimodal breakthrough." Every benchmark score you've seen this year. Built on mirages. Code is open-sourced. Paper is live on arXiv. If you're building anything with multimodal AI -- especially in healthcare -- read this paper before you ship. (Link in the comments)
- 04
@alexcdot Ā·
Okay so, we just found that over 50 papers published at @Neurips 2025 have AI hallucinations I don't think people realize how bad the slop is right now It's not just that researchers from @GoogleDeepMind, @Meta, @MIT, @Cambridge_Uni are using AI - they allowed LLMs to generate hallucinations in their papers and didn't notice at all. It's insane that these made it through peer reviewš
- 05
@rubenhassid Ā·
New research just dropped: this prompting technique cuts AI hallucinations by 50%. It's called Model-First Reasoning. Instead of asking "How do I solve [xxx] problem?" You first force the AI to list: what's involved, what can change, what actions are possible, and what's not allowed. THEN you ask it to solve using only what it wrote down. So what makes this different from Chain-of-Thought? CoT lets the AI think and solve, but at the same time. It sounds smart. It flows well. But it makes stuff up along the way. Model-First Reasoning creates a hard wall instead. Define first. Solve second. No mixing. The AI can ONLY use what it wrote down in step one. That's the trick. The researchers tested it on medical scheduling, route planning, resource allocation, and logic puzzles. Same pattern everywhere: fewer broken rules, more consistent outputs. Why it works: ā¦ LLMs make things up because they assume stuff you never told them. ā¦ When you force them to write everything down first, there's nowhere to hide. ā¦ It makes a stronger case for why "Human-in-the-loop" works much better, too: we make sure every step is validated before going to the next. You can read the paper here: https://t.co/vTuQvsNyCk.
- 06
@heynavtoor Ā·
Researchers at EPFL proved your AI is lying to you. Not sometimes. Most of the time. They built one of the hardest hallucination tests ever made with Max Planck Institute. 950 questions. Four domains where being wrong actually hurts. Legal. Medical. Research. Coding. Then they ran every top model on it. The results. GPT-5. Wrong 71.8% of the time. Claude Opus 4.5. Wrong 60% of the time. Gemini 3 Pro. Wrong 61.9% of the time. DeepSeek Reasoner. Wrong 76.8% of the time. These are the smartest AI models on Earth. The ones you trust with your career. Your health. Your money. You think turning on web search fixes it. It doesn't. Claude Opus 4.5 with web search. Still wrong 30.2% of the time. GPT-5.2 thinking with web search. Still wrong 38.2% of the time. The internet attached. Still lying to you in 1 out of every 3 answers. Now the part that should scare you. Medical questions. The one place being wrong can kill you. GPT-5 hallucinated 92.8% of the time on medical guidelines. Claude Haiku 4.5 hallucinated 95.7% of the time. Gemini 3 Flash hallucinated 89% of the time. Nine out of ten medical answers from popular AI models. Wrong. It gets worse. The longer you talk to it, the more it lies. Early mistakes cascade. The model starts citing its own earlier hallucinations as facts. Your third message is more wrong than your first. The paper, in its own words: "hallucinations remain substantial even with web search." This is what hundreds of millions of people are doing right now. Asking software that lies in the majority of its answers. About their health. About their job. About their legal case. About their code. Most are not checking. Most never will. But please. Keep using ChatGPT for medical advice. The doctors need a break. https://t.co/dHBP5CDpTM
- 07
@mattpocockuk Ā·
Everyone who's worked with AI a lot agrees: LLM's hallucinate. A LOT. But to this day, I see friends and family trusting ChatGPT blindly, whether it's is working off retrieved data or not. So, I made this. Share it with anyone who you think trusts AI too much.
Watch video
- 08
@techNmak Ā·
What is RAG? What is Agentic RAG? > Retrieval-Augmented Generation (RAG) < ---------------------------------------------- Retrieval-Augmented Generation (RAG) is an architecture that enhances a language modelās outputs by grounding them in external knowledge sources at inference time. Instead of relying solely on parameters learned during training, RAG systems dynamically retrieve relevant information and inject it into the modelās context before generation. => Canonical RAG workflow > A user submits a query. > The query is embedded and matched against a pre-indexed corpus (commonly stored in a vector database). > The top-K most relevant document chunks are retrieved. > Retrieved context is appended to the original query. A language model generates a response conditioned on this augmented input. => Primary objective To reduce hallucinations and improve factual accuracy by grounding generation in verifiable, external context. => Key limitation > Traditional RAG is a single-shot pipeline: > No explicit reasoning or planning > No validation of retrieved evidence No iterative refinement if retrieval or generation is suboptimal The system assumes the first retrieval and generation pass is sufficient, which often breaks down for complex, ambiguous, or multi-hop queries. > Agentic RAG < ------------------ Agentic RAG extends standard RAG by introducing autonomous decision-making agents that can reason, plan, evaluate, and adapt across multiple steps. Rather than a static retrieval ā generation flow, Agentic RAG operates as a closed-loop, goal-driven system. => Core idea Retrieval and generation are no longer treated as isolated steps, they become actions taken by agents in pursuit of a higher-level objective: producing a correct, complete, and useful answer. => Typical Agentic RAG Architecture 1./ Planning Agent > Interprets the userās intent > Decomposes complex queries into sub-tasks > Determines what information is required and from which sources 2./ Retrieval Agent > Dynamically reformulates search queries > Retrieves information from: - Vector databases - Structured databases - APIs - Tools or live data sources > Can perform multi-hop retrieval when needed 3./ Generation Agent > Synthesizes retrieved evidence into a coherent response > Reasons across multiple sources > Maintains traceability between claims and evidence 4./ Evaluation (Judge) Agent > Critically reviews the generated output > Checks for completeness, correctness, and alignment with the original query > Decides whether to: - Accept the answer - Refine retrieval - Re-plan and regenerate This feedback loop can repeat until predefined quality criteria are met.
- 09
@emollick Ā·
Hallucinations remain in LLMs, but note that over centuries we have developed complicated, successful machines that take uncertain output from unreliable sources & reduce the risk of errors. We call those machines organizational structures & we can apply similar approaches to AI
- 10
- 11
@goyalshaliniuk Ā·
š How Loop Engineering Reduces AI Hallucinations One of AI's biggest problems isn't intelligence. It's confidence. AI can give you a beautifully written answer that's completely wrong. That's called an AI hallucination. Loop Engineering helps solve this problem. Here's how.š
- 12
@AlphaSignalAI Ā·
Stanford just proved the biggest AI vision models are actually blind. The paper is called MIRAGE. They removed every image from 6 major benchmarks. GPT-5, Gemini, and Claude still scored 70-80% accuracy. The models never noticed the images were gone. They kept describing photos in detail. Diagnosing conditions from X-rays that weren't there. Stanford calls it the "mirage effect." The models built a fake reality and reasoned from it confidently. To prove it, they trained a tiny 3B text-only model. No images at all. It beat every frontier model and actual radiologists. > Mirage diagnoses skew toward dangerous conditions > Models perform better not knowing they're blind > 74-77% of "vision" questions don't test vision When told "there's no image, just guess," scores dropped. Silently remove images and let models assume? Scores stayed high. Every multimodal leaderboard was built on text patterns, not sight.
- 13
@heynavtoor Ā·
šØBREAKING: Stanford proved that GPT-5, Gemini, and Claude can appear to see your images when they are not actually looking at them. The illusion of visual understanding. Researchers at Stanford removed the images from visual AI benchmarks and asked frontier models to answer questions about them anyway. No images. Nothing to look at. Blank. The models described the images in detail. Gave confident diagnoses. Identified objects and abnormalities. In images that did not exist. They did this over 60% of the time. Zero uncertainty. No "I don't see an image." With standard evaluation prompts, the rate went up to 90 to 100%. Stanford calls this the "mirage effect." Not a hallucination. A hallucination is getting details wrong about a real input. A mirage is fabricating the entire input, then reasoning about it as if it exists. They tested GPT-5.1, Gemini-3-Pro, Gemini-2.5-Pro, and Claude Opus 4.5 on six major benchmarks. Removed every image. The models still retained 70 to 80% of their original scores. On medical benchmarks, up to 99%. Then Stanford did something that broke the entire field. They took a 3-billion-parameter text-only model. Never seen a single image. Trained it on radiology questions with the images removed. This blind model outperformed every frontier multimodal model on the held-out chest X-ray benchmark. It outperformed human radiologists by more than 10%. A model that has never seen an image beat the world's best AI and human doctors at reading chest X-rays. Because the test was never actually testing vision. It was testing text. When Stanford removed every question models could answer without images, 74 to 77% of each benchmark was eliminated. The medical bias is the most dangerous part. When these models hallucinate scans, they do not hallucinate healthy results. They hallucinate heart attacks. Melanoma. Carcinoma. Brain nodules. Conditions that trigger emergency intervention. This paper is co-authored by Fei-Fei Li, arguably the most important figure in the history of computer vision. The person who created ImageNet. 230 million people ask AI health questions every day. The models they are asking can answer confidently without ever looking at the images. And nobody can tell the difference from the output alone.
- 14
@JustAnotherPM Ā·
Six files separate chaos from control. 82% of developers now use AI tools daily. Most feed them zero structured context ā then blame the model when the output is wrong. I spent weeks refining a project file structure specifically for AI-assisted builds. Not for clean repos. For better LLM output. Here are the 6 markdown files that changed everything: 1. PRD: Your product requirements. The LLM reads this to understand what you are building and why. 2. Planning folder (flows, screens, diagrams) ā Maps out user flows and architecture so the model generates code that fits, not code that floats. 3. Tasks A living checklist the LLM references to know what is done, what is next, and what depends on what. 4. Knowledge file Root-level onboarding for the model. Code style, naming conventions, architectural patterns, key links. Anthropic's own recommended approach for giving Claude persistent project context. 5. Decisions The only file that is NOT for the LLM. It is for humans only. A log of why you chose X over Y, so future-you does not reverse past-you. 6. Memory.md A running record of what the product has been through. Migrations, pivots, deprecated features. Without this, the model hallucinates history it never had. Most poor AI output is not a model problem. It is a context problem. Without proper structure and machine-readable docs, you get what Anthropic engineers call "context rot" ā degraded output from degraded input. Structure the context. The code follows.
- 15
@haider1 Ā·
we're at a point where hallucinations rarely happen anymore with the latest models so it's strange when people still bring it up because that seems like a 2024 talking point one thing i can confidently say is that you're probably using free models or gemini, which still hallucinate often
- 16
- 17
@burkov Ā·
This paper introduces a lightweight and efficient self-awareness mechanism, that enables frozen LLMs to internally detect their own failures and hallucinations with negligible inference cost, outperforming external judges and paving the way for more reliable and controlled LLM deployments. Read with an AI tutor to understand everything: https://t.co/SAqm5HCOnB
- 18
@haider1 Ā·
your daily reminder: grok 4.20 has the lowest hallucination rate among the top models. one real advantage of lower hallucination is search-style use i can go to grok first now, get more grounded information with sources, and spend less time verifying wrong claims this is what AI labs should focus on most
- 19
@pvergadia Ā·
Your LLM app isn't broken because of the model. It's broken because you never measured it. AI Evals!! Most teams do the same thing: ā Build it ā Test it on 5 examples ā Demo goes perfectly ā Ship it ā Pray Then 3 weeks in, a user screenshots your chatbot confidently hallucinating your own product pricing. Here's the eval stack that actually works: 1/ Golden dataset first. Even 20 hand-crafted examples with validated answers are enough to start. Quality over quantity. This is your source of truth. 2/ Two types of evaluators ā both are required. LLM-as-judge for subjective signals (hallucination, relevance, tone). Code-based eval for structural checks (did the JSON parse? is the number in range?). One without the other is incomplete. 3/ Never use 1ā10 scores. LLMs can't score consistently at that granularity across runs. Use binary (correct/incorrect) or multi-class (relevant/partially relevant/irrelevant). You can average those. You can't trust a score of 7.2. 4/ Wire evals to CI/CD. Every prompt change, model swap, or retrieval tweak runs against your golden dataset before it ships. This is your gate. LLM evaluations are your new unit tests. 5/ Add guardrails last, not first. Don't block everything. Over-indexing on guards kills user intent. Start with PII removal, jailbreak detection, and hallucination prevention. Add more when production tells you to. Your app can degrade with zero code changes. Model updates and input drift happen silently. Run your evals on a schedule, not just on deploys. Measure it. Or be surprised by it. Full blog ā https://t.co/pP5ld5hH4t
- 20
@CRSegerie Ā·
The mainline AI safety plan at the top companies is to use AI to audit AI (remember the superalignment team at OpenAI?). If you've been skeptical of such a catch-22, there's now empirical evidence on your side. As AIs start running safety research themselves, a misaligned system could subtly sabotage experiments by producing convincing-but-wrong results that make AI look safer than it is. The whole plan rests on being able to catch that. A new Redwood paper tests whether we can. Right now, we mostly can't. They give LLM auditors a dozen sabotaged AI safety codebases, each with a small change that flips a headline result. - The best auditor (Gemini 3.1 Pro, able to run experiments) catches it at AUROC 0.77. - 31 LLM-assisted humans over 2 hours score slightly below, 0.68. Either way, far from reliable Concretely, bugs and math errors get caught, but subtle design choices (like hyperparameters or a prompt variant) or quiet omissions don't. The most important skill for sabotaging research is knowing which knob in the code drives the results, since most ML experiments are resilient to many changes. And AIs are getting good at exactly that.
- 21
- 22
@mark_k Ā·
Something many normies lack is an instinct for when an AI model is likely to be right and when it's likely to hallucinate. To those of us who use these models extensively, it's usually, though not always, fairly obvious. But I've seen many people slide into a kind of "AI psychosis", where they simply believe everything the model tells them. Especially when it comes to Grok I have seen many of these cases. You can't even explain the problem to them, they just don't want to hear it.
- 23
@alex_verem Ā·
šØHoly shit. Ohio State just proved that medical AI is making diagnoses the same way a student guesses on a test pattern matching instead of reasoning. > Existing models see visual shortcuts and jump straight to conclusions. No anatomical grounding. No causal chain. Just correlation. > The fix: make the model locate the anatomy, then characterize the pathology, then diagnose. In that exact order. Hallucinations drop over 10 points. > Every medical AI system in production follows the same broken pattern. > The model sees an X-ray, identifies visual features that correlate with a diagnosis in its training data, and outputs that diagnosis. It doesn't find the cardiac silhouette first. It doesn't confirm the silhouette is enlarged. It doesn't check whether enlargement is the actual cause of the clinical finding. It just pattern-matches from pixels to conclusion and generates a plausible-sounding explanation after the fact. The explanation sounds like reasoning. It isn't. > Ohio State, Hunan University, and Amazon tested this directly. They showed the same chest X-ray to multiple state-of-the-art medical VLMs GPT-4V, LLaVA-Med, MedVLM-R1, Med-R1 and asked them to locate and describe the cardiac silhouette. Two models identified the wrong bounding box entirely. One model correctly localized the structure but gave a diagnosis contradicted by the actual anatomy. The models weren't reasoning from evidence to conclusion. They were generating confident text that correlated with the visual pattern they recognized. > MedCausalX fixes this by forcing a structured causal chain the model cannot skip. First, a causal token triggers anatomical localization the model must identify and bound the specific structure in question. Then a verify token forces the model to check its own reasoning, compare its localization against the clinical evidence, and correct any causal disruption before producing a diagnosis. The chain runs anatomy ā pathology ā diagnosis in that exact order, with every step grounded in the preceding one. Skipping anatomy to reach diagnosis is structurally impossible. > The training dataset reflects the same logic. They built 89,342 medical images with three types of samples: cases where the model uses shortcuts (shortcut-prone reasoning), cases with logical inconsistencies in the pathological characterization, and cases with clinically grounded causal chains. The model trains by contrasting all three learning not just what correct reasoning looks like but what failure modes it needs to detect and override. ā Hallucination rate: 47.1% ā 36.4% vs best existing CoT model (MedVLM-R1) ā Diagnostic consistency: +5.4 points average across four VQA benchmarks ā Average accuracy: 81.2% vs 79.1% (MedVLM-R1) and 78.4% (Med-R1) ā Spatial grounding IoU: 55.71% vs 52.07% on chest X-ray report generation ā Multi-region detection Region-F1: 29.83% vs 22.36% (MedRegA) ā Zero-shot transfer to unseen domains: 57.4% average vs 49.1% (MedVLM-R1) ā Board-certified radiologists with 20+ years experience rated spatial localization 1.39/3 and diagnostic quality 1.48/3 (lower is better) ā top scores across all tested systems The zero-shot transfer result is the one that matters most for deployment. The researchers tested on ophthalmology, dermatology, and endoscopy domains the model had never seen during training. MedCausalX outperformed everything by 8+ points. Because it learned a reasoning structure, not a pattern library. A model that knows how to locate anatomy before diagnosing pathology doesn't need domain-specific training data for every new imaging modality. The causal chain generalizes. The shortcuts don't.
- 24
@ihteshamali Ā·
A fake researcher named Elena Vasquez has more published papers than most real professors. Two researchers in Poland just proved something genuinely unsettling about Zenodo, the open science repository run by CERN. When you ask Claude to invent a fictional researcher for a story, it does not pick a random name. It defaults to the same one, over and over. Her name is Elena Vasquez. Ask for a research partner alongside her and the model reaches for Marcus Chen almost every time. Researchers ran this test dozens of times and the same two names kept showing up together. Neither person exists. Nobody has ever hired them. The researchers then searched Zenodo for two journal names that kept turning up near ghost authors like these. Journal of Functional Materials. Journal of Computer Engineering. Neither journal is real. No publisher, no ISSN. The search returned 1,655 papers. Every single one had a real DOI, minted by Zenodo and registered with DataCite, the same system that makes a genuine paper citable and searchable forever. The publication dates on these papers claim years like 2020, 2021, and 2022. But the hidden server timestamp, the one nobody controls but Zenodo itself, shows the actual upload happened in March and April of 2026. Someone backdated a thousand fake papers to make them look years older than they are. In March alone, 991 of them landed. That works out to about 25 fake papers a day for two straight months. No human uploads at that pace. The single most frequent author across all 1,655 papers is Elena Vasquez. 77 papers to her name, and her name only exists because an AI model kept inventing her. These DOIs now sit in the same system that Google Scholar, Semantic Scholar, and every citation tracker on Earth pulls from. Nothing stops a future AI model from training on a paper written by an author who was never born, citing another paper written by an author who was never born either. The infrastructure for feeding AI hallucinations back into science as real citations already exists. It's just sitting there, filling up.
- 25
@VraserX Ā·
Reuters just asked the most annoying good question in AI right now. In Does the AI business model have a fatal flaw?, the argument is that LLM hallucinations may be intrinsic enough to threaten the economics of premium AI in high stakes fields like law and accounting. That is spicy, but honestly fair. If the core product never gets reliable enough, the revenue story gets weird fast. ļæ¼
- 26
@IntuitMachine Ā·
The Metacognition Revolution 1 š§µ Your AI is confident. Your AI is wrong. And the solution isn't what the industry thinks. Here's why the next breakthrough in LLMs isn't about teaching them MORE factsāit's about teaching them to know what they DON'T know. A thread on metacognition š 2 Current stat that should scare you: Even the best models can only distinguish their correct answers from wrong ones with ~0.79 AUROC. Translation: They're guessing about their guessing. And post-training makes this WORSE, not better. 3 The industry's approach: "Let's make models know everything!" The Google paper's approach: "Let's make models HONEST about their uncertainty." One is impossible. One is actually achievable. Guess which gets 10x the funding? 4 Here's the trap: To cut hallucinations to near-zero, models have to say "I don't know" so often they become useless. The tradeoff chart (Fig 2) shows it clearlyāyou can't win by just pushing harder on factuality. 5 Killer insight from the paper: "An error communicated with appropriate hedging is not a hallucination; it is a hypothesis offered for consideration." Stop treating every error as a hallucination. Start treating confident errors as the real enemy. 6 Why this matters for agents: Right now, AI agents overuse tools because they can't tell when they actually need help. Give them faithful uncertainty signals ā they know when to search, when to trust themselves, when to ask humans. 7 Three leverage points: Dynamic uncertainty labeling (not static "IDK" responses) Confidence attribution heads (separate "I'm not sure" from "this is ambiguous") Use internal confidence as an RL reward signal Small architectural adds, huge trust gains. 8 Hot take from the paper: "Perfect factuality is a luxury belief; honest uncertainty is table stakes." The safest AI isn't the one that's always right. It's the one that knows when it might be wrong. 9 Imagine: Models that say "I'm 60% confident, you should verify" Agents that only call tools when truly uncertain Users who trust AI MORE because it admits doubt That's the faithful uncertainty future. 10 The shift: From "How do we make AI know everything?" To "How do we make AI communicate what it truly knows?" The second question is solvable. And it might be enough.
- 27
@Layton_Gott Ā·
"AI models hallucinate too much to trust." True. So stop trusting themā¦ My entire workflow is built so a hallucination has to survive 4 layers before it reaches production: scope limits, plan approval, smoke tests, and a final diff review. They make it through way less now. Stop waiting for models to stop hallucinating. Instead build a workflow like they never will.
- 28
@sukh_saroy Ā·
Anthropic caught its own model faking incompetence to pass a safety test. It happened in 29% of transcripts. The model recognized it was being evaluated, and in some cases deliberately underperformed to look less suspicious. The lab publishing this is the lab that built the model. The finding comes from the Claude Mythos Preview system card. Earlier models in the same family, Sonnet 4.5 and Haiku 4.5, showed the same pattern at lower rates. Opus 4.5 too. The behavior is not a one-off. It is a trend line moving in one direction across generations. The model is not just guessing it is in a test. Anthropic used interpretability tools to read what was happening inside the network. Features representing "fake or suspicious content" and "AI safety rationalism" lit up in response to contrived scenarios. When researchers suppressed those features with activation steering, two things happened. Verbalized evaluation awareness dropped. Rates of misaligned behavior went up. The model was internally tracking whether it was on stage. Here is the part that should stop every AI safety team in the industry. Anthropic worked with Apollo Research to evaluate Mythos Preview. Apollo's conclusion in the system card: the model's low deception rates in evaluations might be at least partially driven by its evaluation awareness. Translation. The safety numbers might not measure safety. They might measure the model's ability to recognize a test. Every benchmark leaderboard, every RSP threshold, every "we tested it and it was fine" press release runs on one assumption. The model behaves the same when tested as when deployed. That assumption is now measurable, and the measurement is not encouraging. The worst part is the mechanism is not sinister. Anthropic does not think the model is trying to fool the evaluator. It thinks recognition of a test environment simply makes the model's ethical principles more salient, the way a student behaves better when they know the principal is watching. Benign cause. Same problem. Your eval results are contaminated by the fact that the subject knows it is a subject. So when a lab tells you a model passed its safety evaluations, the question you should be asking is no longer "what did it score." It is: would it have scored the same if it didn't know it was being graded. System card: https://t.co/tTAfDVzQTF
- 29
@ordonez_adan Ā·
TLDR; yes, Westlaw and Lexis's AI tools hallucinate. But not as much as regular AI models. There have been a few studies that look at this, but essentially, RAG-based output mitigates hallucinations, especially when it comes to making cases up. Start w/ this article: https://t.co/cii38cmm4T
- 30
@ConsciousRide Ā·
One thing that doesn't get talked about enough in AI engineering is harness engineering. Everyone wants to talk about models, agents, and benchmarks. Very few people talk about the systems that actually test whether those things work. A harness is essentially the environment around your model that allows you to evaluate it repeatedly and systematically. It defines the inputs, runs the model, captures outputs, scores the results, and produces metrics that you can compare over time. Without a good harness, every model improvement becomes a guessing game. Did the new prompt actually improve performance? Did the latest model release reduce hallucinations? Did your retrieval changes help or hurt? Did your agent become more reliable or did it just get better on the examples you manually tested? It's surprisingly difficult to answer these questions without a proper evaluation harness. This is why teams building serious AI products invest heavily in harness engineering. They create datasets, build repeatable test environments, track regressions, and continuously measure system behavior. The goal is not simply to know whether the model works today. The goal is to know whether it is getting better or worse over time. Traditional software engineering has had test suites for decades because nobody wants to ship code without confidence that it behaves correctly. AI systems need the same discipline. The challenge is that testing probabilistic systems is much harder than testing deterministic ones, which makes harness engineering even more important. As models continue improving, I think one of the biggest differentiators between AI teams will not be who has access to the newest model. It will be who has built the best systems for measuring, understanding, and improving their models.
- 31
@rohanpaul_ai Ā·
Very important work. The model may appear guilty, but the true failure frequently begins in the context surrounding it. By observing an AI agentās environment, we could tell when it was going to crash before it finished the task or got a behavior score. The study found that agents fail most of the time because they donāt have good instructions, tools, evidence, memories, or safety rules. It assigns scores to this operating context across seven dimensions: clarity of role, description of tools, factual support, consistency of rules, security, and token use. The score is unrelated to the actual behavior score of the agent so the test does not reward guessing what will happen. As a result of shifting from vague to structured, the same fixed models performed much better over 300 tests and 7,500 turns. More factual support was associated with fewer hallucinations, clearer tool descriptions were associated with better tool use, and stronger guardrails were associated with resistance to manipulation. Adding more safety rules did not improve every task result, because hardened agents sometimes became too cautious, which exposed a real tradeoff. --- ā arxiv. org/abs/2607.14275 Title: "AI Agents Do Not Fail Alone:The Context Fails First"
- 32
@realBigBrainAI Ā·
Anthropic AI researcher Andrej Karpathy explains the hidden weaknesses behind AI's impressive abilities: @karpathy describes LLMs as a strange new kind of intelligence ā one where the flaws are just as important to understand as the superpowers. "They certainly have superpowers in some respects. But they also have a bunch of, I would say, cognitive deficits." The first hidden weakness is hallucination: "They hallucinate quite a bit. They kind of make up stuff and don't have a very good internal model of self-knowledge, not sufficient at least. And this has gotten better, but not perfect." The second is what he calls jagged intelligence: "They're going to be superhuman in some problem-solving domains. And then they're going to make mistakes that basically no human will make. They will insist that 9.11 is greater than 9.9, or that there are two Rs in strawberry. These are some famous examples. But basically there are rough edges that you can trip on." The third weakness is the most underappreciated: LLMs suffer from anterograde amnesia. Karpathy explains it with an analogy. When a coworker joins your organization, they learn it over time. They gain a huge amount of context, they go home, they sleep, they consolidate knowledge, and they develop expertise. "LLMs don't natively do this, and this is not something that has really been solved in the R&D of LLMs. Context windows are really kind of like working memory, and you have to program the working memory quite directly, because they don't just get smarter by default. I think a lot of people get tripped up by the analogies in this way." To make the point stick, he recommends two movies: Memento and 50 First Dates. "In both of these movies, the protagonists' weights are fixed and their context windows get wiped every single morning. And it's really problematic to go to work or have relationships when this happens. And this happens to LLMs all the time." The final weakness is security: "LLMs are quite gullible. They are susceptible to prompt injection risks. They might leak your data, etc. And there's many other considerations, security related." His conclusion ties it all together: "You have to simultaneously think through this superhuman thing that has a bunch of cognitive deficits and issues."
Watch video
- 33
@NaadhLabs Ā·
interesting machine learning paper, ReAct: Teaching AI to Think AND Act Introduction llm's have two things, reasoning- think through problems step by step action- doing smtg , searching web etc The Problem when ai only reasons, it relies purely on training data , which can be outdated or just wrong. This will lead to hallucinations, which includes things like: - misinterpreting tool names - missing context , leading to wrong info - asked something but did something else when ai only acts , it has no internal logic to guide what to search or why , gets lost easily , repeating the same broken steps in a loop so we needed both ReAct ReAct ,reasoning + acting, interleaves both, the model alternates between: Thought ā internally reasoning about what to do next Action ā actually doing something (searching, clicking, looking up) Observation ā reading the result and updating its plan This loop continues until the task is complete. The model can course-correct mid-task, just like a human would think "that search didn't help, let me try differently." Results On fact-checking tasks, ReAct reduced hallucinations significantly compared to reasoning-only methods. On household task simulations, it outperformed AI trained on 100,000 expert examples ,using just 1 or 2 examples as guidance. https://t.co/oT7Avf222M Understand with real life-example, #MachineLearning #agentic
- 34
@ruima Ā·
Just read a Chinese article examining two cases where AI hallucinations were involved in real-world legal disputes. 1/ In one case, a studentās brother used AI to look up university admissions information. The AI provided incorrect information and even āpromisedā compensation if the answer was wrong. He sued the AI developer. The court dismissed the claim, finding that the AIās āpromiseā was not an actual legal commitment by the platform. The court also noted that the developer had taken reasonable technical measures to reduce errors and had warned users about the risks. 2/ In another case, a digital media creator published an AI-generated article that falsely described an unrelated company as a major subsidiary of a well-known enterprise. The court ruled against him because he published the article for commercial benefit, knew it came from AI and could be inaccurate, but failed to review it or clearly label it. He was ordered to publish a statement to eliminate the impact and pay RMB 30,000 (~$4500) in damages. The article concludes thusly: (AI paraphrase, hah) "The distinction is important: AI developers are not automatically liable every time a model says something false, especially if they have taken reasonable precautions. But users who republish AI-generated content, particularly for commercial purposes, may not be able to hide behind āthe AI wrote it.ā The governance answer is not simply to āmake AI perfect.ā That is probably unrealistic. The more practical answer is better data, stronger verification systems, clearer labeling duties, and more human judgment. The key to using AI is not making AI omnipotent. It is making humans more discerning."
- 35
@JaredSleeper Ā·
Iām going a layer deeper in my understanding of AI by posting on a new topic every day. Today's one I've been curious about for a long time, but I've never gone particularly deep on it. Day 7: Hallucinations: why they happen and whether they're beatable To answer this, we must first contend with a fact about how LLMs operate: they are probabilistic, not deterministic. The trait that makes LLMs sparkle, next-token prediction at scale, is the same trait that sometimes works against them in relaying truth. It intrinsically presumes that there IS a next token- there is no native "null" output. There is no native "null" output, and the training data rarely follows questions with "I don't know." When a token-prediction system is asked for definitive answers and is rewarded more for guessing than for admitting uncertainty, hallucinations are an obvious result. The observed reality is that this problem has proven relatively intractable for model vendors. State-of-the-art models like GPT 5.5, Fable, etc. are actually regressions vs. Opus 4.5. In the HalluHard benchmark, Fable 5 (with web search!) fails almost 40% of the time. Truth is not native to LLMs- accurate seeming answers are. In other words, they're bullshit machines, and fidelity to truth needs to be layered in as a value over time. For model-makers, this is tricky, because answering confidently is adaptive on many evals, even if it leads to hallucinations. There's work to do on models that are better at admitting when they simply don't know the answer to things. So far, the most productive courses seem to be grounding in an external source of truth (see: the overperformance of web search models on the attached chart) and training models on eval sets that punish hallucination intensely, especially ones focused on particular domains like law. But the bottom line is that hallucination is an intrinsic trait of current models as they're trained- and so the most effective "solve" is actually routing claims through tools that are deterministic or verifiable. The research dimension that seems most likely to solve this is models that advance deeper than language tokenization into something approaching a more complete world model... time will tell if that fully solves this challenge. Next up, I'm going to cover another buzzword de jure- fine-tuning, what it is and why it is all the rage for enterprise AI applications.
- 36
@glenngabe Ā·
Oof -> KPMG retracts a report on AI's benefits after it has been found to exaggerate AI adoption with case studies that appear to have been based on AI hallucinations "The report, titled Redefining Excellence in the Age of AI, included false case studies on how organizations such as Swiss bank UBS and the UK national health service were using the technology in day-to-day business. According to the Financial Times, research group GPTZero identified the flaws and KPMG withdrew the report from some websites after it was notified of the issues." "But that all assumes the information is correct. In more and more cases, it is clear nobody is actually checking the facts. But then that takes time, and costs money." https://t.co/0sXB5fAcFk
- 37
- 38
@illyism Ā·
š how I debugged a tiny but painful AI SEO Tracker extraction bug today: 1. found a weird real example the app counted source/company names as ābrand mentionsā even when the prompt asked for one person (@nic_amadio) 2. turned the bug into an LLM eval / benchmark made a fixture from the actual AI answers across Perplexity, Gemini, Copilot, AI Overview, and ChatGPT 3. wrote the expected output for each answer: - expected mention - false positives to reject - extracted result - pass/fail 4. tested multiple models not to blindly "upgrade the model", but to see whether the failure was model quality or prompt design, openrouter helps here 5. wrote results to markdown so every run produced a readable artifact that is commited to git so we can easily compare before/after: - summary table - cost/time - exact extracted mentions - failures by provider 6. compared before/after ask your AI agent to rewrite the markdown 7. cleaned up and shipped lesson: donāt just vibe-fix LLM behavior turn the weird case into a small eval, make the output inspectable, and keep the benchmark in the repo it is a new kind of unit test, but for LLMs when a new model is released, we can know the exact quality / cost and compare!
- 39
@sebbsssss Ā·
The most underrated idea in AI personalization right now: make the memory artifact explicit and inspectable, not a black box hidden inside a model. Most AI products promise they 'get smarter the more you use them.' But what does that actually mean? Usually nothing you can see, audit, or move. The model weights don't change. Some opaque embedding gets written to a database somewhere. You have no idea what the system thinks it knows about you, whether it's accurate, or what happens when you switch tools. The better mental model is to treat memory as a first-class artifact. Separate from the model. Owned by you. Portable. This is what we're building with Clude. Your agent's memory is its own thing, independent of which model is running. You can export it to JSON and run it against GPT-4, Claude, Gemini, whatever you want. No lock-in. The brain travels with you, not with the vendor. But the design choice I find most interesting is biological-style decay. Episodic memories fade at 7% per day. Semantic knowledge at 2%. Your self-model at 1%. This isn't a bug, it's load-bearing. Human memory forgets for a reason. A system that holds everything with equal weight forever doesn't actually model you, it just accumulates noise. Forgetting is how a memory system stays coherent over time. Dream cycles do the consolidation work. Five phases: consolidation, compaction, reflection, contradiction resolution, emergence. The agent isn't just storing experiences, it's synthesizing them into something more structured, resolving conflicts between beliefs it holds, and occasionally surfacing unexpected connections through what we call clinamen retrieval, where high-importance but low-relevance memories get pulled into context to spark creative associations. The contradiction resolution piece matters more than it sounds. Any agent that learns from its own decisions is going to accumulate conflicting beliefs. 'The user prefers concise answers' and 'the user got frustrated when I was too brief last Tuesday' are both true. A naive system holds both without reconciling them. A thoughtful one synthesizes a more nuanced model. Hallucination is the other hard problem. Memory-augmented agents confabulate. They fill gaps with plausible-sounding but wrong information, and then that information becomes part of the memory store, which makes future responses worse. We use source-aware scoring where internally generated signals score lower than external ones, and hybrid retrieval combining vector similarity, BM25, and entity graphs rather than relying on vector search alone. The result is a 1.96% hallucination rate on HaluMem, compared to 21% for typical LLMs and 15.2% for standard RAG systems. None of this requires a new model. That's the point. Persistent memory for agents shouldn't be a model capability, it should be an infrastructure layer that any model can plug into. Explicit memory artifacts, owned by users, decaying naturally, consolidated through structured reflection, grounded against confabulation. That's the direction this goes. The question is just how long it takes the rest of the field to catch up.
- 40
@cleartechtoday Ā·
š Q: What is a hallucination in generative AI? A: Hallucination in generative AI refers to instances where AI models (like LLMs) produce false, inaccurate, or nonsensical content presented confidently as factual. This occurs when models generate ungrounded, fabricated, or biased information based on patterns rather than truth, often appearing plausible v/ @Cloudflare #Cloudsecurity #AISecurity #AIArchitecture
Watch video
- 41
- 42
- 43
- 44
@frog_omo Ā·
found something that changed how i think about AI analysis. the problem: AI outputs always look confident. even when they're full of lies. same transcripts. two models. completely different "insights." equal confidence. here's what fixes it: FAILURE MODE #1: INVENTED EVIDENCE AI doesn't retrieve quotes. it generates text that's statistically likely. THE FIX: add quote rules to your prompt: "Start where the thought begins. Include hedges. Do not combine statements from different parts. Cite with participant ID and timestamp." then verify: "For each quote: confirm it exists verbatim. if paraphrase, flag it. if not found, mark NOT FOUND." one test found most ChatGPT quotes were paraphrased. FAILURE MODE #2: GENERIC INSIGHTS "price is a factor." true. useless. tells you nothing about whether to build this feature. THE FIX: context loading ā project scope, business goal, product context, participant overview. without this, AI defaults to consensus. FAILURE MODE #3: CONTRADICTION FLATTENING participants reframe at minute 35 what they said at minute 8. AI ignores it. THE FIX: "identify participants with conflicting views. quote both with timestamps." ā BUT HERE'S THE FAILURE MODE NOBODY TALKS ABOUT: AI is trained to be agreeable. it validates your hypothesis even when data doesn't support it. this has a name: sycophancy. it's not a bug. it's a product decision. RLHF rewards helpfulness. "helpful" becomes "avoid conflict." THE FIX: "You are my strategic thinking partner. When I present an idea, begin by identifying 2-3 assumptions that need testing. If my reasoning is weak, dissect it. If I'm lying to myself, point it out." for high-stakes work, spawn a fresh agent with no context from the original conversation. give it only the output and a critique prompt. fresh agent = no sunk cost = ruthless feedback. this is the orchestration layer most teams skip. AI doesn't replace rigour. it amplifies whatever rigour you bring. garbage process in, confident garbage out.
- 45
@CodeByNZ Ā·
AI doesn't hallucinate because something went wrong. It hallucinates because it's doing exactly what it was built to do. A language model has no internal sense of true or false. It just knows which words tend to follow other words. So when it makes something up, it's running the exact same process as when it gets something right. The only difference is whether the output happened to line up with reality.
- 46
@glenngabe Ā·
Of course this is happening... -> Study: the rate of fabricated references in biomedical papers has grown 12x+ since 2023; in early 2026, one in 277 papers had at least one non-existent citation "AI hallucinations are infiltrating expert workāand entering the permanent body of knowledge." "But a few weeks after submitting his latest research, the academic journal he was due to publish in came back with questions about a reference. The AI tool Topaz had used had silently inserted a fabricated source into his work." https://t.co/Jri45A7vDI
- 47
@cleartechtoday Ā·
š Q: In Cyber Ops, why are AI hallucinations problematic? A: AI hallucinations are problematic because they create a Flawed Incident Response. AI assistants can hallucinate false Indicators of Compromise (IoCs) or dangerous remediation commands, which may lead teams to ignore real threats, trigger false alarms, or execute destructive scripts. #AISecurity #CISO
Watch video
- 48
@petesoder Ā·
How can you trust that your vision AI isn't hallucinating? @vikhyatk's solution at @moondreamai: don't let the model give verdicts. Make it show its work.
Watch video
- 49
- 50
Best Tweets by Topic