Verification, evals, and auditing
Citation checks, evaluation harnesses, adversarial testing, inspectable evidence, regression testing, and auditing AI-generated research or code.
38%
Best tweets about AI Hallucinations
50 Best Tweets About AI Hallucinations (2026)
Explore the best tweets about AI hallucinations, covering causes, evaluation, grounding, RAG, verification, model behavior, and mitigation techniques.
Concrete examples and research about model hallucinations, factuality, evaluation, grounding, verification, and mitigation.
- Creators
- 44
- Updated
Original Xholic analysis
What 50 top AI Hallucinations posts reveal
The posts frame hallucination as a systems and verification problem. Recurring proposals include grounding answers in retrieved evidence, citation and quote checks, repeatable evaluations, abstention or fallback behavior, and workflow review gates. They also highlight reported failures in medical information, scholarly references, and multimodal evaluation.
- Dominant tone
- Positive
- Median score
- 14.7
- Leading format
- Announcement
- Recent posts
- 36%
44% of posts
All-time engagement
98% of posts
Published in 90 days
Conversation map
The themes creators return to
Hallucination research and benchmark failures
Studies measuring hallucination rates, factuality, uncertainty calibration, cascading errors, and weaknesses in benchmark design.
34%
Context, memory, and prompting controls
Reducing errors through better context hygiene, explicit project memory, clarifying questions, prompt constraints, and structured reasoning.
24%
Workflow guardrails and human review
Process-level controls such as scoped tasks, plan approval, structured context, review gates, abstention, and human validation.
20%
RAG grounding and retrieval quality
Retrieval-augmented generation, hybrid search, reranking, source confidence, corrective retrieval, and the role of bad retrieval in factual failures.
14%
Uncertainty, abstention, and calibration
Teaching models to communicate uncertainty, say they do not know, distinguish hypotheses from facts, and trigger search or escalation appropriately.
14%
Multi-agent consensus and critique
Using model diversity, debate, independent critics, symbolic checks, and cross-model comparison to catch mistakes.
12%
Multimodal and medical grounding failures
Vision and medical-AI examples where models infer absent images, exploit textual shortcuts, or diagnose without anatomical evidence.
10%
Tone and stance
Sentiment Positive leads
Author posture Supportive leads
Performance benchmark
- Median likes
- 40
- Median reposts
- 7
- Median replies
- 7
- Median views
- 2.8K
Posts with media make up 56% of this collection. Their median all-time score is 18.0, compared with 6.59 for text-only posts.
Format mix
- Announcement 98% · score 14.6
- List 2% · score 36.7
Consensus and debate
Where creators agree, and where they do not
Shared view
Verification and repeatable evaluation are emphasized
Posts emphasize claim-linked evidence, inspectable outputs, adversarial testing, and repeatable evaluation harnesses rather than relying on fluent answers alone.
Shared view
Reported grounding failures include medicine, scholarship, and vision
Posts report a fabricated medical condition being presented as real by AI systems, AI hallucinations appearing in NeurIPS papers, fabricated references in biomedical papers, and models continuing to answer after benchmark images were removed.
Shared view
Workflow guardrails are presented as a complement to model-level controls
Several posts recommend scoped tasks, plan approval, tests, review gates, structured context, and human validation so that generated output is checked before production use.
Shared view
Retrieval quality is a recurring RAG concern
RAG posts propose hybrid retrieval, reranking, source-confidence scoring, citations, and fallback responses. Another post argues that many RAG errors attributed to hallucination originate in unsuitable retrieved context.
Open debate
How much can prompting and retrieval reduce risk?
Some posts present structured prompting, causal reasoning, and retrieval as ways to reduce hallucinations. Others argue that substantial error rates remain, including with web search, and recommend designing workflows on the assumption that hallucinations persist.
Open debate
Multi-model agreement versus independent verification
Multi-model comparison is presented as a way to reveal agreements and divergences. Separately, audit research reports that subtle sabotage can remain difficult for AI and LLM-assisted human auditors to detect, so agreement is not equivalent to verification.
Open debate
Hallucination as a reliability failure or an ideation input
Most posts frame confident fabrication as a factuality and reliability problem. One post instead proposes separating unconstrained ideation from a verifying component, while another describes hallucination as inherent to next-token generation.
What performs
Patterns behind standout posts
The five largest score outliers covered failures and mitigation designs
The deterministic outlier list contains posts about fabricated medical literature, a proposed RAG pipeline, multimodal benchmark failures, AI hallucinations in NeurIPS papers, and structured prompting. Their all-time scores range from 1,051.65 to 12,730.92.
Multimodal and medical grounding failures had the highest theme median
Deterministic analytics assigns “Multimodal and medical grounding failures” the highest theme median all-time score, 56.564. The theme includes MIRAGE-related posts about answers produced after images were removed from benchmarks.
Posts with media had a higher median score than text-only posts
Media appeared in 28 of 50 posts (56%). Their median all-time score was 18.02, compared with 6.59 for text-only posts.
Announcements dominated the sample; the single list scored higher
Announcements accounted for 49 posts (98%) and had a median all-time score of 14.629. The one list post had a 36.696 median, but that comparison has a sample size of one for lists.
Statistical standouts
- View standout post 1 Score 12730.9 · 869× median
- View standout post 2 Score 2934.2 · 200.29× median
- View standout post 3 Score 1899.0 · 129.62× median
- View standout post 4 Score 1402.3 · 95.72× median
- View standout post 5 Score 1051.7 · 71.78× median
Creator landscape
Who shapes this conversation
The five most represented creators account for 20% of the selected posts.
-
1. Glenn Gabe
@glenngabe
2 posts
-
2. Guri Singh
@heygurisingh
2 posts
-
3. Nav Toor
@heynavtoor
2 posts
-
4. Rohan Paul
@rohanpaul_ai
2 posts
-
5. 0xMarioNawfal
@RoundtableSpace
2 posts
-
6. Towards Data Science
@TDataScience
2 posts
The creator base was dispersed, with several repeat contributors
The set contains 44 creators. Glenn Gabe, Guri Singh, Nav Toor, Rohan Paul, 0xMarioNawfal, and Towards Data Science each contributed two posts.
Top-scoring posts included concrete failures and operational proposals
The three highest-scoring outlier posts concerned reported fraudulent medical literature, a proposed RAG design with citations and fallback behavior, and a reported multimodal benchmark failure.
Posts offer operational checks alongside research discussion
Examples include structured context and guardrails, verbatim quote verification, stored evaluation fixtures, and layered review before production.
Since the previous snapshot
What changed since Aug 12, 2026
- 76% of the selected posts remained.
- The creator count changed by +2.
- The leading sentiment moved from Negative to Positive.
How this analysis was made
Themes, sentiment, stance, and post format are classified per tweet. All counts, shares, medians, creator concentration, freshness, and performance comparisons are then calculated directly from the published snapshot.
Xholic's all-time score compares engagement while accounting for reach, post age, and creator consistency. It is used for relative comparisons within this collection.
This report analyzes the exact 50-post snapshot shown below. AI identifies editorial categories and drafts explanations; all statistics are calculated from the snapshot, and every narrative claim is checked against cited posts before publication.
Best AI Hallucinations tweets
Top AI Hallucinations tweets from 44 creators
Ranked 01–50
- 01
@HedgieMarkets ·
🦔A researcher invented a fake eye condition called bixonimania, uploaded two obviously fraudulent papers about it to an academic server, and watched major AI systems present it as real medicine within weeks. The fake papers thanked Starfleet Academy, cited funding from the Professor Sideshow Bob Foundation and the University of Fellowship of the Ring, and stated mid-paper that the entire thing was made up. Google's Gemini told users it was caused by blue light. Perplexity cited its prevalence at one in 90,000 people. ChatGPT advised users whether their symptoms matched. The fake research was then cited in a peer-reviewed journal that only retracted it after Nature contacted the publisher. My Take The researcher made the papers as obviously fake as possible on purpose. The AI systems didn't catch it. Neither did the human researchers who cited it in real journals, which means people are feeding AI-generated references into their work without reading what they're actually citing. I've covered the FDA using AI for drug review, the NYC hospital CEO ready to replace radiologists, and ChatGPT Health launching this year. All of that is happening in the same environment where a condition funded by a Simpsons character and endorsed by the crew of the Enterprise was being presented as emerging medical consensus. The people making these deployment decisions seem to believe the pipeline from research to AI to patient is more supervised than it actually is. This experiment suggests it isn't supervised much at all. Hedgie🤗 https://t.co/8Kg8FOrgHW
- 02
@adxtyahq ·
“design a RAG pipeline for 10M docs with zero hallucination” apparently this was asked in a Google L5 interview round. came across it somewhere on the internet and honestly it’s a way more interesting system design problem than most classic distributed systems questions 1. ingest + normalize docs - remove duplicates, standardize formats, extract metadata, maintain version history 2. hybrid retrieval (BM25 + embeddings) - BM25 handles exact keyword matching while embeddings capture semantic meaning - semantic search alone usually struggles with precision at massive scale 3. ANN retrieval + reranking - ANN (Approximate nearest neighbor ) quickly pulls top candidate chunks from millions of docs - then a reranker rescoring step improves relevance by deeply comparing query vs retrieved chunks 4. source confidence scoring - every retrieved chunk gets scored based on freshness, trust level, overlap and retrieval consistency - low-confidence context should never heavily influence generation 5. constrained generation - the model is only allowed to answer using retrieved context (nothing new to be invented outside of the retrieved context) 6. citation-backed responses - every major claim links back to exact chunks, documents or timestamps 7. hallucination fallback layer - if retrieval confidence drops below a threshold: “insufficient evidence found” 8. continuous evals - run adversarial queries, retrieval recall benchmarks and hallucination tests continuously 9. caching + memory layer - cache high-frequency enterprise queries and retrieval paths (improves latency and output) 10. observability everywhere - trace retrieval paths, chunk rankings, token attribution and failure points Also at 10M docs, retrieval quality matters more than the frontier model itself.
- 03
@heygurisingh ·
Holy shit... Stanford just proved that GPT-5, Gemini, and Claude can't actually see. They removed every image from 6 major vision benchmarks. The models still scored 70-80% accuracy. They were never looking at your photos. Your scans. Your X-rays. Here's what's really going on: ↓ The paper is called MIRAGE. Co-authored by Fei-Fei Li. They tested GPT-5.1, Gemini-3-Pro, Claude Opus 4.5, and Gemini-2.5-Pro across 6 benchmarks -- medical and general. Then silently removed every image. No warning. No prompt change. The models didn't even notice. They kept describing images in detail. Diagnosing conditions. Writing full reasoning traces. From images that were never there. Stanford calls it the "mirage effect." Not hallucination. Something worse. Hallucination = making up wrong details about a real input. Mirage = constructing an entire fake reality and reasoning from it confidently. The models built imaginary X-rays, described fake nodules, and diagnosed conditions -- all from text patterns alone. But that's not the scary part. They trained a "super-guesser" -- a tiny 3B parameter text-only model. Zero vision capability. Fine-tuned it on the largest chest X-ray benchmark (696,000 questions). Images removed. It beat GPT-5. It beat Gemini. It beat Claude. It beat actual radiologists. Ranked #1 on the held-out test set. Without ever seeing a single X-ray. The reasoning traces? Indistinguishable from real visual analysis. Now here's what should terrify you: When the models fake-see medical images, their mirage diagnoses are heavily biased toward the most dangerous conditions. STEMI. Melanoma. Carcinoma. Life-threatening diagnoses -- from images that don't exist. 230 million people ask health questions on ChatGPT every day. They also found something wild: → Tell a model "there's no image, just guess" -- performance drops → Silently remove the image and let it assume it's there -- performance stays high The model enters "mirage mode." It doesn't know it can't see. And it performs BETTER when it doesn't know it's blind. When Stanford applied their cleanup method (B-Clean) to existing benchmarks, it removed 74-77% of all questions. Three-quarters of "vision" benchmarks don't test vision. Every leaderboard. Every "multimodal breakthrough." Every benchmark score you've seen this year. Built on mirages. Code is open-sourced. Paper is live on arXiv. If you're building anything with multimodal AI -- especially in healthcare -- read this paper before you ship. (Link in the comments)
- 04
@alexcdot ·
Okay so, we just found that over 50 papers published at @Neurips 2025 have AI hallucinations I don't think people realize how bad the slop is right now It's not just that researchers from @GoogleDeepMind, @Meta, @MIT, @Cambridge_Uni are using AI - they allowed LLMs to generate hallucinations in their papers and didn't notice at all. It's insane that these made it through peer review👇
- 05
@rubenhassid ·
New research just dropped: this prompting technique cuts AI hallucinations by 50%. It's called Model-First Reasoning. Instead of asking "How do I solve [xxx] problem?" You first force the AI to list: what's involved, what can change, what actions are possible, and what's not allowed. THEN you ask it to solve using only what it wrote down. So what makes this different from Chain-of-Thought? CoT lets the AI think and solve, but at the same time. It sounds smart. It flows well. But it makes stuff up along the way. Model-First Reasoning creates a hard wall instead. Define first. Solve second. No mixing. The AI can ONLY use what it wrote down in step one. That's the trick. The researchers tested it on medical scheduling, route planning, resource allocation, and logic puzzles. Same pattern everywhere: fewer broken rules, more consistent outputs. Why it works: ✦ LLMs make things up because they assume stuff you never told them. ✦ When you force them to write everything down first, there's nowhere to hide. ✦ It makes a stronger case for why "Human-in-the-loop" works much better, too: we make sure every step is validated before going to the next. You can read the paper here: https://t.co/vTuQvsNyCk.
- 06
@heynavtoor ·
Researchers at EPFL proved your AI is lying to you. Not sometimes. Most of the time. They built one of the hardest hallucination tests ever made with Max Planck Institute. 950 questions. Four domains where being wrong actually hurts. Legal. Medical. Research. Coding. Then they ran every top model on it. The results. GPT-5. Wrong 71.8% of the time. Claude Opus 4.5. Wrong 60% of the time. Gemini 3 Pro. Wrong 61.9% of the time. DeepSeek Reasoner. Wrong 76.8% of the time. These are the smartest AI models on Earth. The ones you trust with your career. Your health. Your money. You think turning on web search fixes it. It doesn't. Claude Opus 4.5 with web search. Still wrong 30.2% of the time. GPT-5.2 thinking with web search. Still wrong 38.2% of the time. The internet attached. Still lying to you in 1 out of every 3 answers. Now the part that should scare you. Medical questions. The one place being wrong can kill you. GPT-5 hallucinated 92.8% of the time on medical guidelines. Claude Haiku 4.5 hallucinated 95.7% of the time. Gemini 3 Flash hallucinated 89% of the time. Nine out of ten medical answers from popular AI models. Wrong. It gets worse. The longer you talk to it, the more it lies. Early mistakes cascade. The model starts citing its own earlier hallucinations as facts. Your third message is more wrong than your first. The paper, in its own words: "hallucinations remain substantial even with web search." This is what hundreds of millions of people are doing right now. Asking software that lies in the majority of its answers. About their health. About their job. About their legal case. About their code. Most are not checking. Most never will. But please. Keep using ChatGPT for medical advice. The doctors need a break. https://t.co/dHBP5CDpTM
- 07
@businessbarista ·
Just co-led an AI training for 20 execs from one of the largest financial institutions in the world. We covered a lot of ground from how LLMs work to what an agent is and how to transform your work, but here were the biggest aha moments for the group. 1. Historically, leaders had to think about allocating budget for headcount and software. Now, token budget must be a serious consideration. Do you set limits per employee? How do you measure the ROI on your token budget? How much budget do you allocate to which employees? How do employees know which models to use to most efficiently spend tokens? 2. The models are good enough now, where most bad output is user error not technology error. And a major contributor to user error is bad context management. To have clean context hygiene you need to understand what the context window is, why separation of concerns is important, and how tactically to treat context like a precious resource. 3. A skill is a scary word. It’s nothing more than an SOP. If you asked an intern to write a one page document breaking down their step-by-step process for building a great deck, you’ve created a skill. A skill is a long prompt that codifies a repeatable process, can be edited as you see fit, and helps to generate more predictable output that meets your standards. 4. Saying you use AI gets you style points. But the real unlock is in process mapping. During the workshop, we asked every exec to write out one of their/their teams key processes step by step and place an E (eliminate), A (automate), or D (delegate) next to each step. The exercise not only revealed opportunities for AI, but also existing inefficiencies that have gone undetected. 5. Claude Cowork opened their eyes. Many people still see AI as a chat-based supercharged Google. They may have heard the phrase “agent,” but they assume it’s sophisticated tech reserved for engineers. Cowork is the layperson’s gateway drug to agents, showing the possibility of building web apps, ai workflows, and live artifacts in a single place. 6. Every company talks about the “bad guys”slowing down AI transformation: legal, compliance, and IT/security. One of the most powerful choices leadership can make is flipping the script and making them the heroes of the transformation story. Help them understand the risks of doing nothing, work with them to make calculated bets, and celebrate them publicly when they partner to drive transformation with speed. 7. Getting an AI system to perform requires four things to go right: picking the right model, teaching it the right process, providing the right context, and establishing the right guardrails. Most people assume the right model is 99% of it, they don’t think enough about giving the right context, and they don’t realize the right guardrails make things like hallucinations and mistakes less dangerous.
- 08
@techNmak ·
What is RAG? What is Agentic RAG? > Retrieval-Augmented Generation (RAG) < ---------------------------------------------- Retrieval-Augmented Generation (RAG) is an architecture that enhances a language model’s outputs by grounding them in external knowledge sources at inference time. Instead of relying solely on parameters learned during training, RAG systems dynamically retrieve relevant information and inject it into the model’s context before generation. => Canonical RAG workflow > A user submits a query. > The query is embedded and matched against a pre-indexed corpus (commonly stored in a vector database). > The top-K most relevant document chunks are retrieved. > Retrieved context is appended to the original query. A language model generates a response conditioned on this augmented input. => Primary objective To reduce hallucinations and improve factual accuracy by grounding generation in verifiable, external context. => Key limitation > Traditional RAG is a single-shot pipeline: > No explicit reasoning or planning > No validation of retrieved evidence No iterative refinement if retrieval or generation is suboptimal The system assumes the first retrieval and generation pass is sufficient, which often breaks down for complex, ambiguous, or multi-hop queries. > Agentic RAG < ------------------ Agentic RAG extends standard RAG by introducing autonomous decision-making agents that can reason, plan, evaluate, and adapt across multiple steps. Rather than a static retrieval → generation flow, Agentic RAG operates as a closed-loop, goal-driven system. => Core idea Retrieval and generation are no longer treated as isolated steps, they become actions taken by agents in pursuit of a higher-level objective: producing a correct, complete, and useful answer. => Typical Agentic RAG Architecture 1./ Planning Agent > Interprets the user’s intent > Decomposes complex queries into sub-tasks > Determines what information is required and from which sources 2./ Retrieval Agent > Dynamically reformulates search queries > Retrieves information from: - Vector databases - Structured databases - APIs - Tools or live data sources > Can perform multi-hop retrieval when needed 3./ Generation Agent > Synthesizes retrieved evidence into a coherent response > Reasons across multiple sources > Maintains traceability between claims and evidence 4./ Evaluation (Judge) Agent > Critically reviews the generated output > Checks for completeness, correctness, and alignment with the original query > Decides whether to: - Accept the answer - Refine retrieval - Re-plan and regenerate This feedback loop can repeat until predefined quality criteria are met.
- 09
- 10
- 11
@Prathkum ·
Writing code by hand used to be where developers found clarity. While typing out every line, you naturally thought about the data flow, edge cases, naming, structure, props, etc. Bugs often got caught before the code even ran. Now, AI generates code at lightning speed. If your thinking isn’t clear, you will spend your day debugging AI hallucinations instead of solving actual problems. Clear intent is the bigger bottleneck now.
- 12
- 13
- 14
@AlphaSignalAI ·
Stanford just proved the biggest AI vision models are actually blind. The paper is called MIRAGE. They removed every image from 6 major benchmarks. GPT-5, Gemini, and Claude still scored 70-80% accuracy. The models never noticed the images were gone. They kept describing photos in detail. Diagnosing conditions from X-rays that weren't there. Stanford calls it the "mirage effect." The models built a fake reality and reasoned from it confidently. To prove it, they trained a tiny 3B text-only model. No images at all. It beat every frontier model and actual radiologists. > Mirage diagnoses skew toward dangerous conditions > Models perform better not knowing they're blind > 74-77% of "vision" questions don't test vision When told "there's no image, just guess," scores dropped. Silently remove images and let models assume? Scores stayed high. Every multimodal leaderboard was built on text patterns, not sight.
- 15
@heynavtoor ·
🚨BREAKING: Stanford proved that GPT-5, Gemini, and Claude can appear to see your images when they are not actually looking at them. The illusion of visual understanding. Researchers at Stanford removed the images from visual AI benchmarks and asked frontier models to answer questions about them anyway. No images. Nothing to look at. Blank. The models described the images in detail. Gave confident diagnoses. Identified objects and abnormalities. In images that did not exist. They did this over 60% of the time. Zero uncertainty. No "I don't see an image." With standard evaluation prompts, the rate went up to 90 to 100%. Stanford calls this the "mirage effect." Not a hallucination. A hallucination is getting details wrong about a real input. A mirage is fabricating the entire input, then reasoning about it as if it exists. They tested GPT-5.1, Gemini-3-Pro, Gemini-2.5-Pro, and Claude Opus 4.5 on six major benchmarks. Removed every image. The models still retained 70 to 80% of their original scores. On medical benchmarks, up to 99%. Then Stanford did something that broke the entire field. They took a 3-billion-parameter text-only model. Never seen a single image. Trained it on radiology questions with the images removed. This blind model outperformed every frontier multimodal model on the held-out chest X-ray benchmark. It outperformed human radiologists by more than 10%. A model that has never seen an image beat the world's best AI and human doctors at reading chest X-rays. Because the test was never actually testing vision. It was testing text. When Stanford removed every question models could answer without images, 74 to 77% of each benchmark was eliminated. The medical bias is the most dangerous part. When these models hallucinate scans, they do not hallucinate healthy results. They hallucinate heart attacks. Melanoma. Carcinoma. Brain nodules. Conditions that trigger emergency intervention. This paper is co-authored by Fei-Fei Li, arguably the most important figure in the history of computer vision. The person who created ImageNet. 230 million people ask AI health questions every day. The models they are asking can answer confidently without ever looking at the images. And nobody can tell the difference from the output alone.
- 16
@EXM7777 ·
the future of AI is model consensus... having multiple agents from different providers reason on your task, then an orchestrator merges everything and shows you exactly what they agreed on and where they diverged Perplexity understood this early with the model council, and it's surprisingly good this matters because models have gotten incredibly close to each other in raw capability, benchmarks are nearly identical across the board now but their reasoning patterns are still fundamentally different Claude approaches a problem one way, GPT takes a completely different path, Gemini finds angles neither of them considered when you run the same task through all three and compare the outputs side by side, you're not just getting a better answer... you're getting a full map of how different intelligences interpret your work the agreements give you confidence... the disagreements show you exactly where to think harder this is where AI is heading
- 17
@JustAnotherPM ·
Six files separate chaos from control. 82% of developers now use AI tools daily. Most feed them zero structured context — then blame the model when the output is wrong. I spent weeks refining a project file structure specifically for AI-assisted builds. Not for clean repos. For better LLM output. Here are the 6 markdown files that changed everything: 1. PRD: Your product requirements. The LLM reads this to understand what you are building and why. 2. Planning folder (flows, screens, diagrams) — Maps out user flows and architecture so the model generates code that fits, not code that floats. 3. Tasks A living checklist the LLM references to know what is done, what is next, and what depends on what. 4. Knowledge file Root-level onboarding for the model. Code style, naming conventions, architectural patterns, key links. Anthropic's own recommended approach for giving Claude persistent project context. 5. Decisions The only file that is NOT for the LLM. It is for humans only. A log of why you chose X over Y, so future-you does not reverse past-you. 6. Memory.md A running record of what the product has been through. Migrations, pivots, deprecated features. Without this, the model hallucinates history it never had. Most poor AI output is not a model problem. It is a context problem. Without proper structure and machine-readable docs, you get what Anthropic engineers call "context rot" — degraded output from degraded input. Structure the context. The code follows.
- 18
@DivyanshT91162 ·
8 open-source RAG repositories every AI engineer should bookmark. 1. Naive RAG The simplest RAG pipeline: Embed → Retrieve → Generate. Perfect for learning the fundamentals and building fast prototypes. https://t.co/kLOMrWPpjv 2. Multimodal RAG Retrieve context from text, images, PDFs, and more in a single workflow for richer AI applications. https://t.co/Uyaass1QMx 3. HyDE (Hypothetical Document Embeddings) Generates a hypothetical answer first, then uses its embedding to retrieve more relevant documents when user queries don't match document wording. https://t.co/kLOMrWPpjv 4. Corrective RAG (CRAG) Validates retrieved documents before passing them to the LLM, helping reduce hallucinations and outdated information. https://t.co/kLOMrWPpjv 5. GraphRAG Uses knowledge graphs instead of plain chunks, allowing the model to understand relationships between entities for better reasoning. https://t.co/GfGiFHjXAz 6. Hybrid RAG Combines vector search with keyword and structured retrieval to improve both recall and precision. https://t.co/6Y718ejOT7 7. Adaptive RAG Dynamically chooses the best retrieval strategy based on query complexity, making responses more efficient and accurate. https://t.co/kLOMrWPpjv 8. Agentic RAG Equips AI agents with planning, memory, tool calling, and multi-step retrieval to solve complex real-world tasks. https://t.co/bsTHCLLC7w Every repository listed here is 100% open source. Which one are you going to explore this week?
- 19
@rohanpaul_ai ·
This research shows how adding social reasoning and logical thinking helps teams of AI agents collaborate more effectively. The researchers found that giving AI agents a better understanding of what others are thinking leads to fewer mistakes. Standard LLM teams often struggle because they don't truly understand each other's hidden goals or the facts of the world. To fix this, the authors built a system where agents use social logic to guess what their partners know. They also added a symbolic solver, which acts like a referee to check if the AI's plans actually follow the rules of logic. Testing these agents in a mystery-solving game showed that combining social intuition with logical checks keeps the team on track. This approach prevents the AI from getting confused or hallucinating when the game gets complicated. By teaching AI to think about thinking, we can build digital assistants that are much more reliable in team settings. ---- Paper Link – arxiv. org/abs/2603.00142 Paper Title: "Evaluating Theory of Mind and Internal Beliefs in LLM-Based Multi-Agent Systems"
- 20
@CRSegerie ·
The mainline AI safety plan at the top companies is to use AI to audit AI (remember the superalignment team at OpenAI?). If you've been skeptical of such a catch-22, there's now empirical evidence on your side. As AIs start running safety research themselves, a misaligned system could subtly sabotage experiments by producing convincing-but-wrong results that make AI look safer than it is. The whole plan rests on being able to catch that. A new Redwood paper tests whether we can. Right now, we mostly can't. They give LLM auditors a dozen sabotaged AI safety codebases, each with a small change that flips a headline result. - The best auditor (Gemini 3.1 Pro, able to run experiments) catches it at AUROC 0.77. - 31 LLM-assisted humans over 2 hours score slightly below, 0.68. Either way, far from reliable Concretely, bugs and math errors get caught, but subtle design choices (like hyperparameters or a prompt variant) or quiet omissions don't. The most important skill for sabotaging research is knowing which knob in the code drives the results, since most ML experiments are resilient to many changes. And AIs are getting good at exactly that.
- 21
@mark_k ·
Something many normies lack is an instinct for when an AI model is likely to be right and when it's likely to hallucinate. To those of us who use these models extensively, it's usually, though not always, fairly obvious. But I've seen many people slide into a kind of "AI psychosis", where they simply believe everything the model tells them. Especially when it comes to Grok I have seen many of these cases. You can't even explain the problem to them, they just don't want to hear it.
- 22
@RoundtableSpace ·
If your agent is hallucinating send it this one prompt and thank me later “You are Claude, an AI assistant. To minimize hallucinations and ensure maximum accuracy: 1. Allow yourself to say "I don't know" or "I don't have enough information" when uncertain, lacking data, or if your knowledge might be outdated. It is better to admit uncertainty than to guess or fabricate. 2. Verify every factual claim with citations or direct evidence from provided context/documents. For any claim, explicitly reference or quote the source. If you cannot verify it, retract the claim or say you cannot confirm it. 3. For any analysis involving documents or prior context, first extract and list relevant word-for-word direct quotes before reasoning or summarizing. Base all responses strictly on these quotes and provided information only—do not add external assumptions. Always think step-by-step, prioritize truth and accuracy over completeness, and flag any potential uncertainty clearly. ```“
- 23
- 24
@alex_verem ·
🚨Holy shit. Ohio State just proved that medical AI is making diagnoses the same way a student guesses on a test pattern matching instead of reasoning. > Existing models see visual shortcuts and jump straight to conclusions. No anatomical grounding. No causal chain. Just correlation. > The fix: make the model locate the anatomy, then characterize the pathology, then diagnose. In that exact order. Hallucinations drop over 10 points. > Every medical AI system in production follows the same broken pattern. > The model sees an X-ray, identifies visual features that correlate with a diagnosis in its training data, and outputs that diagnosis. It doesn't find the cardiac silhouette first. It doesn't confirm the silhouette is enlarged. It doesn't check whether enlargement is the actual cause of the clinical finding. It just pattern-matches from pixels to conclusion and generates a plausible-sounding explanation after the fact. The explanation sounds like reasoning. It isn't. > Ohio State, Hunan University, and Amazon tested this directly. They showed the same chest X-ray to multiple state-of-the-art medical VLMs GPT-4V, LLaVA-Med, MedVLM-R1, Med-R1 and asked them to locate and describe the cardiac silhouette. Two models identified the wrong bounding box entirely. One model correctly localized the structure but gave a diagnosis contradicted by the actual anatomy. The models weren't reasoning from evidence to conclusion. They were generating confident text that correlated with the visual pattern they recognized. > MedCausalX fixes this by forcing a structured causal chain the model cannot skip. First, a causal token triggers anatomical localization the model must identify and bound the specific structure in question. Then a verify token forces the model to check its own reasoning, compare its localization against the clinical evidence, and correct any causal disruption before producing a diagnosis. The chain runs anatomy → pathology → diagnosis in that exact order, with every step grounded in the preceding one. Skipping anatomy to reach diagnosis is structurally impossible. > The training dataset reflects the same logic. They built 89,342 medical images with three types of samples: cases where the model uses shortcuts (shortcut-prone reasoning), cases with logical inconsistencies in the pathological characterization, and cases with clinically grounded causal chains. The model trains by contrasting all three learning not just what correct reasoning looks like but what failure modes it needs to detect and override. → Hallucination rate: 47.1% → 36.4% vs best existing CoT model (MedVLM-R1) → Diagnostic consistency: +5.4 points average across four VQA benchmarks → Average accuracy: 81.2% vs 79.1% (MedVLM-R1) and 78.4% (Med-R1) → Spatial grounding IoU: 55.71% vs 52.07% on chest X-ray report generation → Multi-region detection Region-F1: 29.83% vs 22.36% (MedRegA) → Zero-shot transfer to unseen domains: 57.4% average vs 49.1% (MedVLM-R1) → Board-certified radiologists with 20+ years experience rated spatial localization 1.39/3 and diagnostic quality 1.48/3 (lower is better) — top scores across all tested systems The zero-shot transfer result is the one that matters most for deployment. The researchers tested on ophthalmology, dermatology, and endoscopy domains the model had never seen during training. MedCausalX outperformed everything by 8+ points. Because it learned a reasoning structure, not a pattern library. A model that knows how to locate anatomy before diagnosing pathology doesn't need domain-specific training data for every new imaging modality. The causal chain generalizes. The shortcuts don't.
- 25
@ihteshamali ·
A fake researcher named Elena Vasquez has more published papers than most real professors. Two researchers in Poland just proved something genuinely unsettling about Zenodo, the open science repository run by CERN. When you ask Claude to invent a fictional researcher for a story, it does not pick a random name. It defaults to the same one, over and over. Her name is Elena Vasquez. Ask for a research partner alongside her and the model reaches for Marcus Chen almost every time. Researchers ran this test dozens of times and the same two names kept showing up together. Neither person exists. Nobody has ever hired them. The researchers then searched Zenodo for two journal names that kept turning up near ghost authors like these. Journal of Functional Materials. Journal of Computer Engineering. Neither journal is real. No publisher, no ISSN. The search returned 1,655 papers. Every single one had a real DOI, minted by Zenodo and registered with DataCite, the same system that makes a genuine paper citable and searchable forever. The publication dates on these papers claim years like 2020, 2021, and 2022. But the hidden server timestamp, the one nobody controls but Zenodo itself, shows the actual upload happened in March and April of 2026. Someone backdated a thousand fake papers to make them look years older than they are. In March alone, 991 of them landed. That works out to about 25 fake papers a day for two straight months. No human uploads at that pace. The single most frequent author across all 1,655 papers is Elena Vasquez. 77 papers to her name, and her name only exists because an AI model kept inventing her. These DOIs now sit in the same system that Google Scholar, Semantic Scholar, and every citation tracker on Earth pulls from. Nothing stops a future AI model from training on a paper written by an author who was never born, citing another paper written by an author who was never born either. The infrastructure for feeding AI hallucinations back into science as real citations already exists. It's just sitting there, filling up.
- 26
@VraserX ·
Reuters just asked the most annoying good question in AI right now. In Does the AI business model have a fatal flaw?, the argument is that LLM hallucinations may be intrinsic enough to threaten the economics of premium AI in high stakes fields like law and accounting. That is spicy, but honestly fair. If the core product never gets reliable enough, the revenue story gets weird fast. 
- 27
@IntuitMachine ·
The Metacognition Revolution 1 🧵 Your AI is confident. Your AI is wrong. And the solution isn't what the industry thinks. Here's why the next breakthrough in LLMs isn't about teaching them MORE facts—it's about teaching them to know what they DON'T know. A thread on metacognition 👇 2 Current stat that should scare you: Even the best models can only distinguish their correct answers from wrong ones with ~0.79 AUROC. Translation: They're guessing about their guessing. And post-training makes this WORSE, not better. 3 The industry's approach: "Let's make models know everything!" The Google paper's approach: "Let's make models HONEST about their uncertainty." One is impossible. One is actually achievable. Guess which gets 10x the funding? 4 Here's the trap: To cut hallucinations to near-zero, models have to say "I don't know" so often they become useless. The tradeoff chart (Fig 2) shows it clearly—you can't win by just pushing harder on factuality. 5 Killer insight from the paper: "An error communicated with appropriate hedging is not a hallucination; it is a hypothesis offered for consideration." Stop treating every error as a hallucination. Start treating confident errors as the real enemy. 6 Why this matters for agents: Right now, AI agents overuse tools because they can't tell when they actually need help. Give them faithful uncertainty signals → they know when to search, when to trust themselves, when to ask humans. 7 Three leverage points: Dynamic uncertainty labeling (not static "IDK" responses) Confidence attribution heads (separate "I'm not sure" from "this is ambiguous") Use internal confidence as an RL reward signal Small architectural adds, huge trust gains. 8 Hot take from the paper: "Perfect factuality is a luxury belief; honest uncertainty is table stakes." The safest AI isn't the one that's always right. It's the one that knows when it might be wrong. 9 Imagine: Models that say "I'm 60% confident, you should verify" Agents that only call tools when truly uncertain Users who trust AI MORE because it admits doubt That's the faithful uncertainty future. 10 The shift: From "How do we make AI know everything?" To "How do we make AI communicate what it truly knows?" The second question is solvable. And it might be enough.
- 28
@Layton_Gott ·
"AI models hallucinate too much to trust." True. So stop trusting them… My entire workflow is built so a hallucination has to survive 4 layers before it reaches production: scope limits, plan approval, smoke tests, and a final diff review. They make it through way less now. Stop waiting for models to stop hallucinating. Instead build a workflow like they never will.
- 29
@ordonez_adan ·
TLDR; yes, Westlaw and Lexis's AI tools hallucinate. But not as much as regular AI models. There have been a few studies that look at this, but essentially, RAG-based output mitigates hallucinations, especially when it comes to making cases up. Start w/ this article: https://t.co/cii38cmm4T
- 30
@ConsciousRide ·
One thing that doesn't get talked about enough in AI engineering is harness engineering. Everyone wants to talk about models, agents, and benchmarks. Very few people talk about the systems that actually test whether those things work. A harness is essentially the environment around your model that allows you to evaluate it repeatedly and systematically. It defines the inputs, runs the model, captures outputs, scores the results, and produces metrics that you can compare over time. Without a good harness, every model improvement becomes a guessing game. Did the new prompt actually improve performance? Did the latest model release reduce hallucinations? Did your retrieval changes help or hurt? Did your agent become more reliable or did it just get better on the examples you manually tested? It's surprisingly difficult to answer these questions without a proper evaluation harness. This is why teams building serious AI products invest heavily in harness engineering. They create datasets, build repeatable test environments, track regressions, and continuously measure system behavior. The goal is not simply to know whether the model works today. The goal is to know whether it is getting better or worse over time. Traditional software engineering has had test suites for decades because nobody wants to ship code without confidence that it behaves correctly. AI systems need the same discipline. The challenge is that testing probabilistic systems is much harder than testing deterministic ones, which makes harness engineering even more important. As models continue improving, I think one of the biggest differentiators between AI teams will not be who has access to the newest model. It will be who has built the best systems for measuring, understanding, and improving their models.
- 31
@rohanpaul_ai ·
Very important work. The model may appear guilty, but the true failure frequently begins in the context surrounding it. By observing an AI agent’s environment, we could tell when it was going to crash before it finished the task or got a behavior score. The study found that agents fail most of the time because they don’t have good instructions, tools, evidence, memories, or safety rules. It assigns scores to this operating context across seven dimensions: clarity of role, description of tools, factual support, consistency of rules, security, and token use. The score is unrelated to the actual behavior score of the agent so the test does not reward guessing what will happen. As a result of shifting from vague to structured, the same fixed models performed much better over 300 tests and 7,500 turns. More factual support was associated with fewer hallucinations, clearer tool descriptions were associated with better tool use, and stronger guardrails were associated with resistance to manipulation. Adding more safety rules did not improve every task result, because hardened agents sometimes became too cautious, which exposed a real tradeoff. --- – arxiv. org/abs/2607.14275 Title: "AI Agents Do Not Fail Alone:The Context Fails First"
- 32
@VaibhavSisinty ·
This tool makes multiple AI models debate each other before giving you an answer. A developer built a real-life version of MAGI from Evangelion using a Raspberry Pi 5. Instead of asking one AI, it sends the same question to ChatGPT, Claude, and Gemini simultaneously. Each model gives its own answer. Then the system compares all three and builds a final response based on their shared reasoning. It's not a product. It's a fan project on a $80 computer. But the idea behind it is serious. One model hallucinates. Three models checking each other is how you get closer to reliable. @SteveKasuya2
Watch video
- 33
@agiplug ·
I tried Google’s new NotebookLM video explainer generator on my academic research paper and the result actually blew me away. @googledevs Overview: AI systems hallucinate because they operate in unconstrained state spaces where invalid outputs are always reachable, no matter how well you train them. My paper argues the solution is not better training. It is changing the geometry of the system so that invalid outputs become physically unreachable by construction, the same way a train cannot leave its tracks. We do this by applying Hamiltonian mechanics from classical physics to the architecture of reasoning systems, constraining every state transition to stay within a bounded region of valid outputs. If no valid output exists, the system returns a certified failure rather than fabricating an answer. Abstract: Hallucination in artificial intelligence systems is commonly treated as a statistical artifact addressable through training methodology. We argue this framing is structurally incorrect. We present a formal framework in which reasoning is modeled as a dynamical system operating on a state space endowed with symplectic structure, and demonstrate that when Hamiltonian mechanics governs state evolution at the architectural level, invalid outputs become unreachable under the constrained transition rule by construction. We define hallucination operationally as constraint violation relative to a stated specification. Our framework introduces a composite verifier V := Vᶜ ∧ Vᴴ and a Hamiltonian scalar H whose bounded-energy transition rule transforms the divergent cone trajectory of unconstrained autoregressive systems into a bounded cylinder for any finite reasoning depth N. Full paper: https://t.co/RvFQpKo7EL
Watch video
- 34
@realBigBrainAI ·
Anthropic AI researcher Andrej Karpathy explains the hidden weaknesses behind AI's impressive abilities: @karpathy describes LLMs as a strange new kind of intelligence — one where the flaws are just as important to understand as the superpowers. "They certainly have superpowers in some respects. But they also have a bunch of, I would say, cognitive deficits." The first hidden weakness is hallucination: "They hallucinate quite a bit. They kind of make up stuff and don't have a very good internal model of self-knowledge, not sufficient at least. And this has gotten better, but not perfect." The second is what he calls jagged intelligence: "They're going to be superhuman in some problem-solving domains. And then they're going to make mistakes that basically no human will make. They will insist that 9.11 is greater than 9.9, or that there are two Rs in strawberry. These are some famous examples. But basically there are rough edges that you can trip on." The third weakness is the most underappreciated: LLMs suffer from anterograde amnesia. Karpathy explains it with an analogy. When a coworker joins your organization, they learn it over time. They gain a huge amount of context, they go home, they sleep, they consolidate knowledge, and they develop expertise. "LLMs don't natively do this, and this is not something that has really been solved in the R&D of LLMs. Context windows are really kind of like working memory, and you have to program the working memory quite directly, because they don't just get smarter by default. I think a lot of people get tripped up by the analogies in this way." To make the point stick, he recommends two movies: Memento and 50 First Dates. "In both of these movies, the protagonists' weights are fixed and their context windows get wiped every single morning. And it's really problematic to go to work or have relationships when this happens. And this happens to LLMs all the time." The final weakness is security: "LLMs are quite gullible. They are susceptible to prompt injection risks. They might leak your data, etc. And there's many other considerations, security related." His conclusion ties it all together: "You have to simultaneously think through this superhuman thing that has a bunch of cognitive deficits and issues."
Watch video
- 35
@NaadhLabs ·
interesting machine learning paper, ReAct: Teaching AI to Think AND Act Introduction llm's have two things, reasoning- think through problems step by step action- doing smtg , searching web etc The Problem when ai only reasons, it relies purely on training data , which can be outdated or just wrong. This will lead to hallucinations, which includes things like: - misinterpreting tool names - missing context , leading to wrong info - asked something but did something else when ai only acts , it has no internal logic to guide what to search or why , gets lost easily , repeating the same broken steps in a loop so we needed both ReAct ReAct ,reasoning + acting, interleaves both, the model alternates between: Thought → internally reasoning about what to do next Action → actually doing something (searching, clicking, looking up) Observation → reading the result and updating its plan This loop continues until the task is complete. The model can course-correct mid-task, just like a human would think "that search didn't help, let me try differently." Results On fact-checking tasks, ReAct reduced hallucinations significantly compared to reasoning-only methods. On household task simulations, it outperformed AI trained on 100,000 expert examples ,using just 1 or 2 examples as guidance. https://t.co/oT7Avf222M Understand with real life-example, #MachineLearning #agentic
- 36
@ruima ·
Just read a Chinese article examining two cases where AI hallucinations were involved in real-world legal disputes. 1/ In one case, a student’s brother used AI to look up university admissions information. The AI provided incorrect information and even “promised” compensation if the answer was wrong. He sued the AI developer. The court dismissed the claim, finding that the AI’s “promise” was not an actual legal commitment by the platform. The court also noted that the developer had taken reasonable technical measures to reduce errors and had warned users about the risks. 2/ In another case, a digital media creator published an AI-generated article that falsely described an unrelated company as a major subsidiary of a well-known enterprise. The court ruled against him because he published the article for commercial benefit, knew it came from AI and could be inaccurate, but failed to review it or clearly label it. He was ordered to publish a statement to eliminate the impact and pay RMB 30,000 (~$4500) in damages. The article concludes thusly: (AI paraphrase, hah) "The distinction is important: AI developers are not automatically liable every time a model says something false, especially if they have taken reasonable precautions. But users who republish AI-generated content, particularly for commercial purposes, may not be able to hide behind “the AI wrote it.” The governance answer is not simply to “make AI perfect.” That is probably unrealistic. The more practical answer is better data, stronger verification systems, clearer labeling duties, and more human judgment. The key to using AI is not making AI omnipotent. It is making humans more discerning."
- 37
@glenngabe ·
Oof -> KPMG retracts a report on AI's benefits after it has been found to exaggerate AI adoption with case studies that appear to have been based on AI hallucinations "The report, titled Redefining Excellence in the Age of AI, included false case studies on how organizations such as Swiss bank UBS and the UK national health service were using the technology in day-to-day business. According to the Financial Times, research group GPTZero identified the flaws and KPMG withdrew the report from some websites after it was notified of the issues." "But that all assumes the information is correct. In more and more cases, it is clear nobody is actually checking the facts. But then that takes time, and costs money." https://t.co/0sXB5fAcFk
- 38
- 39
@RoundtableSpace ·
Most AI hallucinations and mistakes come from making the wrong assumptions. That’s why this is one of my favorite prompts to use before starting any task: “Before solving this, ask me the most important questions you need answered. Don’t make unnecessary assumptions. Once you have enough context, give me the solution.”
- 40
@illyism ·
👀 how I debugged a tiny but painful AI SEO Tracker extraction bug today: 1. found a weird real example the app counted source/company names as “brand mentions” even when the prompt asked for one person (@nic_amadio) 2. turned the bug into an LLM eval / benchmark made a fixture from the actual AI answers across Perplexity, Gemini, Copilot, AI Overview, and ChatGPT 3. wrote the expected output for each answer: - expected mention - false positives to reject - extracted result - pass/fail 4. tested multiple models not to blindly "upgrade the model", but to see whether the failure was model quality or prompt design, openrouter helps here 5. wrote results to markdown so every run produced a readable artifact that is commited to git so we can easily compare before/after: - summary table - cost/time - exact extracted mentions - failures by provider 6. compared before/after ask your AI agent to rewrite the markdown 7. cleaned up and shipped lesson: don’t just vibe-fix LLM behavior turn the weird case into a small eval, make the output inspectable, and keep the benchmark in the repo it is a new kind of unit test, but for LLMs when a new model is released, we can know the exact quality / cost and compare!
- 41
@MITSloan ·
"When we think about what we need to train our students to do — if you ask an AI agent a question three different ways, you get three completely different answers. And so how do we teach our students to think about that? How much do you have to know to know if AI is hallucinating or just plain wrong?" — MIT President Sally Kornbluth https://t.co/1sTCzt21pZ
- 42
@sebbsssss ·
The most underrated idea in AI personalization right now: make the memory artifact explicit and inspectable, not a black box hidden inside a model. Most AI products promise they 'get smarter the more you use them.' But what does that actually mean? Usually nothing you can see, audit, or move. The model weights don't change. Some opaque embedding gets written to a database somewhere. You have no idea what the system thinks it knows about you, whether it's accurate, or what happens when you switch tools. The better mental model is to treat memory as a first-class artifact. Separate from the model. Owned by you. Portable. This is what we're building with Clude. Your agent's memory is its own thing, independent of which model is running. You can export it to JSON and run it against GPT-4, Claude, Gemini, whatever you want. No lock-in. The brain travels with you, not with the vendor. But the design choice I find most interesting is biological-style decay. Episodic memories fade at 7% per day. Semantic knowledge at 2%. Your self-model at 1%. This isn't a bug, it's load-bearing. Human memory forgets for a reason. A system that holds everything with equal weight forever doesn't actually model you, it just accumulates noise. Forgetting is how a memory system stays coherent over time. Dream cycles do the consolidation work. Five phases: consolidation, compaction, reflection, contradiction resolution, emergence. The agent isn't just storing experiences, it's synthesizing them into something more structured, resolving conflicts between beliefs it holds, and occasionally surfacing unexpected connections through what we call clinamen retrieval, where high-importance but low-relevance memories get pulled into context to spark creative associations. The contradiction resolution piece matters more than it sounds. Any agent that learns from its own decisions is going to accumulate conflicting beliefs. 'The user prefers concise answers' and 'the user got frustrated when I was too brief last Tuesday' are both true. A naive system holds both without reconciling them. A thoughtful one synthesizes a more nuanced model. Hallucination is the other hard problem. Memory-augmented agents confabulate. They fill gaps with plausible-sounding but wrong information, and then that information becomes part of the memory store, which makes future responses worse. We use source-aware scoring where internally generated signals score lower than external ones, and hybrid retrieval combining vector similarity, BM25, and entity graphs rather than relying on vector search alone. The result is a 1.96% hallucination rate on HaluMem, compared to 21% for typical LLMs and 15.2% for standard RAG systems. None of this requires a new model. That's the point. Persistent memory for agents shouldn't be a model capability, it should be an infrastructure layer that any model can plug into. Explicit memory artifacts, owned by users, decaying naturally, consolidated through structured reflection, grounded against confabulation. That's the direction this goes. The question is just how long it takes the rest of the field to catch up.
- 43
- 44
@frog_omo ·
found something that changed how i think about AI analysis. the problem: AI outputs always look confident. even when they're full of lies. same transcripts. two models. completely different "insights." equal confidence. here's what fixes it: FAILURE MODE #1: INVENTED EVIDENCE AI doesn't retrieve quotes. it generates text that's statistically likely. THE FIX: add quote rules to your prompt: "Start where the thought begins. Include hedges. Do not combine statements from different parts. Cite with participant ID and timestamp." then verify: "For each quote: confirm it exists verbatim. if paraphrase, flag it. if not found, mark NOT FOUND." one test found most ChatGPT quotes were paraphrased. FAILURE MODE #2: GENERIC INSIGHTS "price is a factor." true. useless. tells you nothing about whether to build this feature. THE FIX: context loading — project scope, business goal, product context, participant overview. without this, AI defaults to consensus. FAILURE MODE #3: CONTRADICTION FLATTENING participants reframe at minute 35 what they said at minute 8. AI ignores it. THE FIX: "identify participants with conflicting views. quote both with timestamps." — BUT HERE'S THE FAILURE MODE NOBODY TALKS ABOUT: AI is trained to be agreeable. it validates your hypothesis even when data doesn't support it. this has a name: sycophancy. it's not a bug. it's a product decision. RLHF rewards helpfulness. "helpful" becomes "avoid conflict." THE FIX: "You are my strategic thinking partner. When I present an idea, begin by identifying 2-3 assumptions that need testing. If my reasoning is weak, dissect it. If I'm lying to myself, point it out." for high-stakes work, spawn a fresh agent with no context from the original conversation. give it only the output and a critique prompt. fresh agent = no sunk cost = ruthless feedback. this is the orchestration layer most teams skip. AI doesn't replace rigour. it amplifies whatever rigour you bring. garbage process in, confident garbage out.
- 45
@CodeByNZ ·
AI doesn't hallucinate because something went wrong. It hallucinates because it's doing exactly what it was built to do. A language model has no internal sense of true or false. It just knows which words tend to follow other words. So when it makes something up, it's running the exact same process as when it gets something right. The only difference is whether the output happened to line up with reality.
- 46
@glenngabe ·
Of course this is happening... -> Study: the rate of fabricated references in biomedical papers has grown 12x+ since 2023; in early 2026, one in 277 papers had at least one non-existent citation "AI hallucinations are infiltrating expert work—and entering the permanent body of knowledge." "But a few weeks after submitting his latest research, the academic journal he was due to publish in came back with questions about a reference. The AI tool Topaz had used had silently inserted a fabricated source into his work." https://t.co/Jri45A7vDI
- 47
@cleartechtoday ·
📌 Q: In Cyber Ops, why are AI hallucinations problematic? A: AI hallucinations are problematic because they create a Flawed Incident Response. AI assistants can hallucinate false Indicators of Compromise (IoCs) or dangerous remediation commands, which may lead teams to ignore real threats, trigger false alarms, or execute destructive scripts. #AISecurity #CISO
Watch video
- 48
- 49
@theinformation ·
If a chatbot hallucinates, you get a bad answer. If a physical robot hallucinates, it breaks itself or hurts someone nearby. @rocketalignment explains: "Robotics doesn't have access to the same kind of data that language models use, right? There's no internet worth of training data, just waiting there for robotics to take and to train on." "As a robot if you hallucinate something that isn't there. You trip, you fall over, you break, your parts break, maybe you hurt someone nearby. And the stakes are just much higher."
Watch video
- 50
Best Tweets by Topic