Retrieval methods and reranking
Dense vectors, lexical/BM25 search, hybrid retrieval, reranking, filtering, ColBERT, and matching retrieval approaches to query types.
46%
Best tweets about RAG
50 Best Tweets About RAG (2026)
Discover the best tweets about retrieval-augmented generation, including RAG architecture, chunking, retrieval, evaluation, grounding, and production systems.
Technical RAG pipelines, retrieval quality, chunking, reranking, grounding, evaluation, failure modes, cost, and production experience.
- Creators
- 42
- Updated
Original Xholic analysis
What 50 top RAG posts reveal
The supplied posts portray RAG as an evidence pipeline spanning ingestion, document structure, retrieval, generation, evaluation, and observability. They challenge a vector-only default with lexical, structure-first, and multimodal alternatives, while repeatedly emphasizing workload-specific testing, customer-data evaluation, citations, no-answer behavior, and traceability.
- Dominant tone
- Positive
- Median score
- 14.5
- Leading format
- Announcement
- Recent posts
- 38%
44% of posts
All-time engagement
44% of posts
Published in 90 days
Conversation map
The themes creators return to
Grounding and answer reliability
Grounded generation, citations, faithfulness, no-answer behavior, uncertainty calibration, contradictions, and source-level claim verification.
32%
RAG evaluation and benchmarks},{
Retrieval and end-to-end evaluation, realistic enterprise benchmarks, corpus-scale degradation, customer-data testing, and failure diagnosis metrics.
26%
Chunking and document structure
Chunk sizing, overlap, semantic and structure-aware segmentation, late chunking, and the drawbacks of fragmenting document context.
18%
Ingestion and index management
Document ingestion, parsing, cleaning, deduplication, metadata, access controls, and index freshness as foundations for retrieval quality.
16%
Graph and evolving knowledge systems
Knowledge graphs, temporal memory, entity relationships, provenance, multi-hop traversal, and accumulated wiki-like knowledge systems.
12%
Vectorless and structured retrieval
Alternatives to conventional vector-and-chunk RAG, including hierarchical document navigation, tree indexes, grep, file-based search, and structured indexes.
12%
Multimodal RAG
Visual, PDF, table, chart, image, video, and OCR-free retrieval pipelines for non-textual enterprise content.
8%
Tone and stance
Sentiment Positive leads
Author posture Supportive leads
Performance benchmark
- Median likes
- 25
- Median reposts
- 4
- Median replies
- 6
- Median views
- 2.6K
Posts with media make up 70% of this collection. Their median all-time score is 17.1, compared with 10.5 for text-only posts.
Format mix
- Announcement 44% · score 13.6
- Opinion 20% · score 7.22
- Tutorial 20% · score 19.2
- List 8% · score 18.6
Consensus and debate
Where creators agree, and where they do not
Shared view
Quality starts before generation
Production-oriented posts treat ingestion, metadata, chunking, retrieval, reranking, answer citations, and stage-level visibility as connected parts of a debuggable evidence pipeline rather than a prompt-only system.
Shared view
Evaluate the deployed system
Posts recommend evaluating at realistic corpus scale and on customer data. They also describe tracing retrieved context, relevance, token counts, and generation stages to diagnose failures.
Shared view
Grounding needs explicit checks
Grounding guidance includes requiring citations, detecting no-answer cases, handling conflicting sources, and evaluating context relevance, faithfulness, answerability, and context support.
Open debate
Default vector retrieval is contested
Posts promoting PageIndex argue for structure-guided navigation instead of vector-and-chunk retrieval. Separately, the EnterpriseRAG-Bench post reports that, in its tested corpus-scale experiment, BM25 declined less sharply than vector search as the corpus grew. Together, they argue for testing retrieval choices against the target workload rather than assuming a universal default.
Open debate
When RAG is necessary remains debated
Some posts describe structured files, indexes, and incrementally maintained wikis as sufficient for relatively small personal knowledge bases. Another characterizes conventional RAG as a cheap, predictable pattern when an answer resides in documents.
Open debate
Retrieval gains may not yield answer gains
These posts caution that retrieval metrics alone do not establish answer reliability. They identify missing or contradictory evidence, models ignoring useful context, and the need to measure faithfulness and answerability alongside retrieval quality.
What performs
Patterns behind standout posts
Alternative retrieval architectures led the largest outliers
The three highest-scoring outlier tweets featured an LLM-maintained wiki workflow, visual screenshot retrieval, and vectorless document navigation. Deterministic analytics reports their all-time scores at 1,378.3×, 138.52×, and 35.49× the overall median, respectively.
Multimodal posts had a higher median score
Multimodal RAG accounts for 8% of tweets and has a median all-time score of 84.62, versus 14.53 overall. The cited posts concern screenshot, video, and rendered-document retrieval.
Reliability discussion outscored evaluation discussion
Grounding and answer reliability has a median all-time score of 24.6, while RAG evaluation and benchmarks has a median of 5.718. The cited posts in both areas focus on locating and diagnosing failures in deployed pipelines.
Statistical standouts
- View standout post 1 Score 20026.7 · 1378.3× median
- View standout post 2 Score 2012.7 · 138.52× median
- View standout post 3 Score 515.6 · 35.49× median
- View standout post 4 Score 452.1 · 31.12× median
- View standout post 5 Score 449.7 · 30.95× median
Creator landscape
Who shapes this conversation
The five most represented creators account for 20% of the selected posts.
-
1. Abhishek Singh
@0xlelouch_
2 posts
-
2. Akshay 🚀
@akshay_pachaar
2 posts
-
3. Aurimas Griciūnas
@Aurimas_Gr
2 posts
-
4. Paul Iusztin
@pauliusztin_
2 posts
-
5. smrati tiwari
@smratitiwa86867
2 posts
-
6. Towards Data Science
@TDataScience
2 posts
Karpathy foregrounds accumulated knowledge
Karpathy describes a personal knowledge-base workflow in which an LLM incrementally compiles raw sources into a linked Markdown wiki. At the cited scale of roughly 100 articles and 400K words, he reports that indexes and summaries let an agent answer complex questions without using a conventional RAG stack; outputs and health checks can feed back into the wiki.
Aurimas Griciūnas emphasizes operations
Aurimas Griciūnas discusses separating rarely changing cached data from retrieved data, with cautions around staleness and RBAC. He also outlines spans that capture retrieved-context relevance, timing, and token counts for cost analysis.
Abhishek Singh catalogs production pitfalls
Abhishek Singh’s two production-mistake lists cover stable document IDs, deduplication, metadata and ACLs, evaluation sets, reranking, citations, and observability.
How this analysis was made
Themes, sentiment, stance, and post format are classified per tweet. All counts, shares, medians, creator concentration, freshness, and performance comparisons are then calculated directly from the published snapshot.
Xholic's all-time score compares engagement while accounting for reach, post age, and creator consistency. It is used for relative comparisons within this collection.
This report analyzes the exact 50-post snapshot shown below. AI identifies editorial categories and drafts explanations; all statistics are calculated from the snapshot, and every narrative claim is checked against cited posts before publication.
Best RAG tweets
Top RAG tweets from 42 creators
Ranked 01–50
- 01
@karpathy ·
LLM Knowledge Bases Something I'm finding very useful recently: using LLMs to build personal knowledge bases for various topics of research interest. In this way, a large fraction of my recent token throughput is going less into manipulating code, and more into manipulating knowledge (stored as markdown and images). The latest LLMs are quite good at it. So: Data ingest: I index source documents (articles, papers, repos, datasets, images, etc.) into a raw/ directory, then I use an LLM to incrementally "compile" a wiki, which is just a collection of .md files in a directory structure. The wiki includes summaries of all the data in raw/, backlinks, and then it categorizes data into concepts, writes articles for them, and links them all. To convert web articles into .md files I like to use the Obsidian Web Clipper extension, and then I also use a hotkey to download all the related images to local so that my LLM can easily reference them. IDE: I use Obsidian as the IDE "frontend" where I can view the raw data, the the compiled wiki, and the derived visualizations. Important to note that the LLM writes and maintains all of the data of the wiki, I rarely touch it directly. I've played with a few Obsidian plugins to render and view data in other ways (e.g. Marp for slides). Q&A: Where things get interesting is that once your wiki is big enough (e.g. mine on some recent research is ~100 articles and ~400K words), you can ask your LLM agent all kinds of complex questions against the wiki, and it will go off, research the answers, etc. I thought I had to reach for fancy RAG, but the LLM has been pretty good about auto-maintaining index files and brief summaries of all the documents and it reads all the important related data fairly easily at this ~small scale. Output: Instead of getting answers in text/terminal, I like to have it render markdown files for me, or slide shows (Marp format), or matplotlib images, all of which I then view again in Obsidian. You can imagine many other visual output formats depending on the query. Often, I end up "filing" the outputs back into the wiki to enhance it for further queries. So my own explorations and queries always "add up" in the knowledge base. Linting: I've run some LLM "health checks" over the wiki to e.g. find inconsistent data, impute missing data (with web searchers), find interesting connections for new article candidates, etc., to incrementally clean up the wiki and enhance its overall data integrity. The LLMs are quite good at suggesting further questions to ask and look into. Extra tools: I find myself developing additional tools to process the data, e.g. I vibe coded a small and naive search engine over the wiki, which I both use directly (in a web ui), but more often I want to hand it off to an LLM via CLI as a tool for larger queries. Further explorations: As the repo grows, the natural desire is to also think about synthetic data generation + finetuning to have your LLM "know" the data in its weights instead of just context windows. TLDR: raw data from a given number of sources is collected, then compiled by an LLM into a .md wiki, then operated on by various CLIs by the LLM to do Q&A and to incrementally enhance the wiki, and all of it viewable in Obsidian. You rarely ever write or edit the wiki manually, it's the domain of the LLM. I think there is room here for an incredible new product instead of a hacky collection of scripts.
- 02
@DAIEvolutionHub ·
WEB SCRAPING JUST GOT A SERIOUS UPGRADE. PixelRAG doesn't read HTML. It reads the page exactly like you do. 100% open-source. Instead of parsing websites into plain text, it captures screenshots and lets a vision model retrieve answers directly from the pixels. Why that's a big deal: • HTML parsers silently lose information. • Tables, charts, formulas, and layouts often disappear. • Even changing the parser can swing RAG accuracy by ~10%. PixelRAG skips that entire bottleneck. It indexes what users actually see. The team built a visual index of 30M+ Wikipedia screenshots, and it outperformed the strongest text-based RAG baseline by 18.1% on text-only QA. Even cooler: It includes a Claude Code plugin that gives Claude visual browsing. Instead of scraping the DOM, Claude can screenshot any webpage, PDF, arXiv paper, or even your local app—and answer based on the rendered page. The pipeline is surprisingly clean: → Render pages into image tiles → Embed with Qwen3-VL-Embedding (LoRA-tuned on screenshots) → Store in a FAISS index → Search visually The best part? Upgrade to a better vision model later, and you don't need to rebuild the index. Because the index stores pixels, not parsed text. Fully open-source under Apache 2.0. GitHub: https://t.co/B7whbNg60s
Watch video
- 03
@AlphaSignalAI ·
Someone removed the vector database from RAG and accuracy jumped to 98.7%. Most RAG systems chunk your documents, embed them as vectors, then retrieve by similarity. The core assumption: similar text means relevant text. That assumption fails on professional documents. Ask "what were the debt trends in Q3?" and vector search returns chunks that look like your question. The real answer sits in an appendix, split across three sections, with zero semantic overlap. Traditional RAG never finds it. PageIndex is an open-source repo that removes the vector database entirely. Inspired by AlphaGo, it builds a tree index from your document. Then it reasons through it like a human expert would. Instead of pattern matching, the model navigates sections logically. "Debt trends live in financial summaries or Appendix G. Let's look there." What changes: 1. No chunking that breaks cross-section context 2. Retrieval traceable to exact pages 3. Multi-step reasoning over document structure It scored 98.7% on FinanceBench. Perplexity scored 45%. GPT-4o hit 31%.
- 04
@techNmak ·
Chunking is the original sin of RAG. You take a beautifully structured document. Slice it into arbitrary 512-token pieces. Destroy all context. Then wonder why retrieval is bad. PageIndex doesn't chunk. Documents stay organized in natural sections. Hierarchy preserved. Context intact. Instead of similarity search over chunks, it uses reasoning over structure. → Build a tree index (like a smart table of contents) → Navigate with LLM reasoning → Find relevant sections through tree search 98.7% accuracy on FinanceBench. No vectors. No chunks. No destroyed context. 18.2K stars. Worth a look. GitHub Repo in comments.
- 05
- 06
@alexxubyte ·
RAGs vs Agents Ask an LLM about your company's data and it will guess. The two patterns that fix this are RAG and agents, and they solve different problems. RAGs: RAGs combine LLMs with retrieval to ground answers in 4 steps. Step 1: The user query is embedded and sent to a retrieval step. Step 2: Retrieval pulls the most relevant chunks from a knowledge base (PDFs, wikis, etc.) Step 3: Those chunks are pasted into the prompt as context. Step 4: The LLM writes the answer, grounded in the retrieved text. One retrieval. One generation. Cheap, predictable, and easy to debug. Agents: Agents wrap LLMs in a reasoning loop with tools to take action. Step 1: The user query goes into the agent runtime. A reasoning loop wrapped around an LLM. Step 2: The LLM reads the goal and picks a tool (Read, Write, Edit, Bash, etc.) Step 3: The runtime executes the tool and feeds the result back to the LLM. Step 4: The LLM reasons again, picks the next tool, and loops until the task is done. More flexible. More tokens. Harder to debug because errors drift across steps. The rule of thumb: Use RAG when the answer lives in your documents. Use an agent when the answer requires action on other systems. Over to you: When do you prefer RAG over agent?
- 07
@techNmak ·
Our RAG system is 90% accurate. Sounds great until you realize: that 10% is destroying user trust. Here's what's happening: 9 out of 10 queries: Perfect answers. Users love it. 1 out of 10 queries: Complete hallucination. Users lose confidence. The trust problem with LLMs: > Users don't know which answers to trust. > One hallucination makes them question everything, even the correct answers. It's like a doctor who's right 90% of the time. Would you trust them? Why 90% isn't good enough: In traditional software: > 90% uptime is terrible > 99.9% is standard > 99.99% is expected In LLM applications: > Many teams are at 80-90% accuracy > Think that's acceptable > Don't realize it's killing adoption What's causing the 10%: After debugging, we found: 40% = Retrieval returned irrelevant documents 30% = LLM ignored good documents and hallucinated 20% = Documents were relevant but contradictory 10% = Prompt was ambiguous You can only see this with proper observability. We implemented Opik (open-source LLM observability): > Traces every retrieval > Scores document relevance > Flags hallucinations automatically > Shows when LLM ignores context > Catches bad outputs before users see them. Built by Comet. Works with LangGraph, LangChain, etc. Self-hostable or cloud. Check the next tweet for GitHub Repo:
- 08
@akshay_pachaar ·
A tricky LLM interview question: Your RAG system scores 90% retrieval accuracy on 5k company docs. But scaling to 500k docs drops the accuracy to just 50%, with the same embedding model and retriever. Why did this happen? The simplest answer is that more documents mean more competition for the top-k retrieval slots. That is true, but it doesn't explain why accuracy drops this dramatically. The answer comes down to how enterprise docs are distributed in the embedding space. Today, a single product decision in a company generates meeting transcripts, Slack threads, Confluence docs, Jira tickets, and email threads. They are related to the same event, so they all land in a similar region of the embedding space. As the company operates over months, this pattern repeats for every project/customer/roadmap, and the embedding space fills up with clusters of closely related documents. But all related docs don't contain the same facts. → Slack thread covers the decision made → Jira has the implementation deadline → Confluence has the technical spec → Email thread has the customer request When a query is about a specific fact (like a deadline), the answer lives in one of those docs. At a 5K corpus size, there might be 3-5 docs touching that topic, and the correct one easily lands in the top-k results. But at a 500K corpus size, there could be 40-60 total docs, and the one containing the actual answer can easily get pushed out of the top-k by other topically relevant docs, degrading retrieval. A recent research paper from Onyx documented this. The researchers used their newly open-sourced EnterpriseRAG-Bench dataset. It has 500k+ synthetic enterprise documents spread across Slack, Gmail, Jira, GitHub, Confluence, Google Drive, HubSpot, Fireflies, and Linear, with realistic noise like misfiled documents, near-duplicates, and conflicting versions. They ran the same retrievers at five corpus sizes from 5K to 500K. → Vector search accuracy dropped from 90.7% at 5K documents to 50.6% at 500K docs. → BM25 degraded more gracefully, from 85.8% to 68.4%. → At every scale, higher neighborhood density in the embedding space monotonically correlated with lower recall. The practical implication here is that retrieval accuracy on a 5k test set tells you almost nothing about production-scale performance. Always test at a realistic volume to measure the neighborhood density in your embedding space to estimate how much headroom the retriever actually has. The entire EnterpriseRAG-Bench dataset (500K docs with questions, and the whole evaluation harness) is open-source. Run your retriever against it at 5K, then at 500K, and see where your own accuracy curve breaks. I have shared the GitHub repo in the replies.
- 09
@Aurimas_Gr ·
Fusion of 𝗥𝗔𝗚 (Retrieval Augmented Generation) and 𝗖𝗔𝗚 (Cache Augmented Generation). You must understand fundamentals behind this architecture to save costs and reduce your system latency efficiently. So how can you benefit from it as AI Engineer? Let’s see what it looks like and what additional considerations should be taken into account. Here are example steps to implement CAG + RAG architecture: 𝘋𝘢𝘵𝘢 𝘗𝘳𝘦𝘱𝘳𝘰𝘤𝘦𝘴𝘴𝘪𝘯𝘨: 𝟭. We use only rarely changing data sources for Cache Augmented Generation. On top of the requirement of data changing rarely we should also think about which of the sources are often hit by relevant queries. Once we have this information, only then we pre-compute all of this selected data into a KV Cache of the LLM. Cache it in memory. This only needs to be done once, the following steps can be run multiple times without recomputing the initial cache. 𝟮. For RAG, if necessary, precompute and store vector embeddings in a compatible database to be searched later in step 4. Sometimes simpler data types are enough for RAG, a regular database might suffice. 𝘘𝘶𝘦𝘳𝘺 𝘗𝘢𝘵𝘩: We can now utilise the preprocessed data. 𝟯. Compose a prompt including user query and the system prompt with instructions on how cached context and retrieved external context should be used by the LLM. 𝟰. Embed a user query to be used for semantic search via vector DBs and query the context store to retrieve relevant data. If semantic search is not required, query other sources, like real time databases or web. 𝟱. Enrich the final prompt with external context retrieved in step 4. 𝟲. Return the final answer to the user. 𝘚𝘰𝘮𝘦 𝘊𝘰𝘯𝘴𝘪𝘥𝘦𝘳𝘢𝘵𝘪𝘰𝘯𝘴: ➡️ Context window is not infinite and even while some models boast enormous context window sizes, the needle in the haystack problem has not yet been solved so use available context wisely and cache only the data you really need. ✅ For some business cases, specific datasets are extremely valuable to be passed to the model as cache. Think about an assistant that has to always comply with a lengthy set of internal rules stored in multiple documents. ✅ While CAG has been popularised for Open Source just recently, it is already viable for some time via Prompt Caching features in OpenAI and Anthropic APIs. It is really easy to start prototyping there. ✅ You should always separate hot and cold data sources, only use cold (data that changes rarely) in your cache, otherwise the data will go stale and the application will go out of sync. ❌ Be very careful about what you cache as the data will be available for all users to query. ❌ It is very hard to ensure RBAC for cached data unless you have a separate model with its own cache per role. Have you used the combination already?
- 10
@svpino ·
We should build a church for people who open-source their code so everyone can learn from it. Here is the complete source code of a RAG assistant to navigate airline policies. You get the complete source code and video from @lenadroid, walking you through everything she did (I'm linking to the video in the first comment below). The fact that you can watch every engineering decision that Lena made when building this app is pure gold. A few things you'll pick up from this: • It uses LangChain for the retrieval pipeline • It uses LangGraph for conversation state • It stores embeddings in Postgres with pgvector • It indexes documents to ground answers in the source text • It uses Terraform to stand up the infrastructure I'm linking to the video walkthrough and the source code below.
- 11
- 12
- 13
@Aurimas_Gr ·
𝗔𝗜 𝗢𝗯𝘀𝗲𝗿𝘃𝗮𝗯𝗶𝗹𝗶𝘁𝘆 is a must have in your tool belt as an AI Engineer. 𝗧𝗿𝗮𝗰𝗶𝗻𝗴 sits at the core of it, why is it important? Tracing and instrumentation of software have been around for decades now. With AI systems resembling regular software even more, we are now moving the practice here as well (with a few key differences). Let’s look into the process of tracing from a perspective of a naive RAG system. 𝘍𝘦𝘸 𝘥𝘦𝘧𝘪𝘯𝘪𝘵𝘪𝘰𝘯𝘴: 𝘼) An Orchestrator in the GenAI system application is the central piece of software that orchestrates the end-to-end process. Think of apps using LangChain, LlamaIndex or Haystack. 𝘽) Trace is the end-to-end application flow from the entry point till the answer is produced, it is composed of smaller pieces called spans. 𝘾) Span is a smaller piece of the application flow that represents an atomic action like a function call or a database query. They can be sequential, or run in parallel. ℹ️ As part of span we capture general metadata like start and end time, inputs and outputs of the span. On top of this metadata we track information specific to the GenAI system elements. What might a trace look like for a naive RAG system? 𝟭. A query that has been submitted to the chat application. 𝟮. The query is embedded into a vector. ✅ Additional metadata like input token count is persisted with the span so that we can estimate the cost of the procedure. 𝟯. ANN lookup performed against the Vector DB to retrieve the most relevant context. ✅ Additional metadata about the query is persisted as part of the span together with the retrieved pieces of context and their relevance. 𝟰. A prompt is constructed from the system prompt and retrieved context. 𝟱. The prompt is passed to the LLM to construct the answer. ✅ Additional metadata about input and output token count is captured together with the span so that we can estimate the cost of the procedure. 𝘞𝘩𝘺 𝘪𝘴 𝘵𝘳𝘢𝘤𝘪𝘯𝘨 𝘰𝘧 𝘎𝘦𝘯𝘈𝘐 𝘴𝘺𝘴𝘵𝘦𝘮𝘴 𝘪𝘮𝘱𝘰𝘳𝘵𝘢𝘯𝘵? - These applications are usually complex chains, errors can happen in different steps of your application. E.g. Embedding of query is taking longer than expected or you have reached API limits of LLM provider. - Cost for calling LLM APIs will be variable depending on the length of inputs and produced outputs. You would usually trace this information and analyze it to help forecast expenses. - GenAI systems are non-deterministic and will deteriorate over time. They need to be evaluated on span level rather than input/output of the entire system so that you can tune each piece separately. - … Are you tracing your Agents? Let me know in the comments 👇
- 14
@_jaydeepkarale ·
First video in the RAG(Retrieval Augmented Generation) series 1. Why we need RAG ? 2. What problems does RAG solve ? 3. Why not take advantage of large context windows instead of building complex RAG sytems This is my first try at creating videos using S9 tab, so be kind. :) Feedback welcome
Watch video
- 15
@PythonDvz ·
Most people building AI products can’t answer this: “What’s the actual difference between an LLM, RAG, an AI Agent, and Agentic AI?” They’re not the same. Confusing them leads to wrong tools, wasted budgets, and over-engineered solutions. Here’s the breakdown. — Layer 1: LLM — The Brain The foundation of every AI system. Understands language, reasons through problems, generates responses — but only from training data. No live data. No actions. No memory beyond the conversation. Key capabilities: Chain-of-thought reasoning, few-shot learning, prompt engineering, transformer architecture. — Layer 2: RAG — The Library Card Connects your LLM to external knowledge documents, databases, internal wikis before generating a response. Your AI stops guessing and starts referencing. Key capabilities: Vector search, document chunking, retrieval pipelines, hybrid search, grounded answer generation. — Layer 3: AI Agent — The Doer Uses the LLM as its reasoning core but wraps it with tools and memory to take real-world actions. Moves from answering to acting. Key capabilities: Tool use, API calling, code execution, task tracking, ReAct framework, short and long-term memory. — Layer 4: Agentic AI — The Team Multiple specialised agents working in parallel, sharing memory, assigning roles, and adapting in real time. You are not prompting anymore. You are deploying. Key capabilities: Multi-agent orchestration, autonomous goal planning, MCP, hierarchical task execution, role-specific agents. — Which do you actually need? → Q&A or content generation? LLM → Accurate answers from your data? RAG → Automated real-world workflows? AI Agent → Fully autonomous operations? Agentic AI Biggest mistake? Jumping to Agentic AI when RAG solves 80% of the problem at 20% of the complexity. Start with the right layer. Scale from there.
- 16
@thetripathi58 ·
🚨 Cambridge researchers just tested what happens when you overload an AI's memory with irrelevant data. They found a complete collapse of modern RAG systems. Not a minor hallucination. A total failure of the exact retrieval architecture that every enterprise AI relies on to access private data. The models simply drowned in the noise. The researchers tested standard Retrieval-Augmented Generation (RAG) and filtering models like Self-RAG. They fed them information but slowly increased the ratio of distracting, low-quality documents. Here is what they found. Current read-time filtering failed completely. When the ratio of distractors hit 8:1, the accuracy of standard RAG systems plummeted to 0%. The AI lost the ability to find the truth. It exposed a massive architectural flaw. We currently store every single document an AI reads, regardless of quality, and force the model to sort through the garbage at query time. It is highly inefficient and fundamentally broken. The biological fix. The researchers built a new system called "Write-Time Gating" modeled after the human hippocampus. Instead of saving everything, it evaluates novelty, reliability, and source reputation before the data is even stored. And then there is the finding that changes how we build AI: hierarchical archiving. When beliefs update, the system does not delete the old data. It deprioritizes it, maintaining a version history just like the human brain. The result? The write-gated system maintained 100% accuracy even at massive distractor scales, all while costing one-ninth the compute of current systems. The researchers made it clear. When you dump raw, unfiltered data into a database and expect the LLM to figure it out later, you are building a system designed to fail at scale. No reliable retrieval. No cost control. No accuracy guarantees. Nothing. Right now, companies are building massive vector databases, throwing every piece of corporate documentation into them, and assuming the AI will magically find the signal in the noise. Stop treating AI memory like a hard drive. Start treating it like a biological filter. Build the gate at the entrance, not the exit.
- 17
@om_patel5 ·
THIS GUY TAUGHT HIS 60 YEAR OLD DAD CLAUDE CODE AND GIT WITH ZERO CODING EXPERIENCE his dad teaches geology. has never written a line of code in his life he showed him the basics of claude and how git works back in feb fast forward to today and his dad built a fully functional RAG system on his own for analyzing and querying his mineral documents RAG (retrieval augmented generation) is when you feed your own documents into an AI so it can search through them and answer questions based on YOUR data instead of its general training this is definitely not a simple chatbot wrapper. this is an advanced system (for someone with zero prior experience) that a geology professor built BY HIMSELF his son is a developer and even he was impressed. said it finally made him understand why vibe coding has become such a thing a proper end-to-end solution engineer is still leagues ahead of someone just prompting an AI. but it is surprisingly impressive how claude code can elevate someone to the level of an average developer with no experience the barrier to building software is gone
- 18
@GithubProjects ·
RAGLite is a lightweight Python toolkit for building retrieval-augmented generation applications on DuckDB or PostgreSQL with late chunking. - Choose any LLM provider via LiteLLM or local llama-cpp-python models - Hybrid search using native keyword and vector search in DuckDB or PostgreSQL - Multi-vector chunk embedding with late chunking and contextual chunk headings - Optimal sentence and semantic chunking via binary integer programming Explore it here: https://t.co/TP2iAodk2t
Watch video
- 19
@DeepStarts ·
Here's a list of Al Engineer Interview questions + concepts you need to know (from Al/ML Engineering Manager perspective) LLM Fundamentals: -What is tokenization, and how does it affect generation? -How do embeddings really work? -What's the role of attention, positional encoding? -What changes during fine-tuning? (optimizers, schedulers, layer freezing) -LoRA vs QLoRA vs full fine-tune - tradeoffs? Prompting & Context Engineering: -Few-shot vs zero-shot - which works better where? -How do you design system prompts that are robust across users? -How do you make output deterministic? -How do you track, version, and backfill changing context? -How do you build/maintain the memory? RAG Systems: -What's your chunking strategy by length, semantics, or structure? -How do you choose a vector DB (Chroma, Pinecone, OpenSearch...)? -Can you update or backfill embeddings with zero downtime? -How do you evaluate retrieval quality (precision@k, reranking, citation)? MLOps & LLMOps: -Sketch a pipeline: from raw data → model-serving → feedback -How would you monitor performance drift hallucinations? -How do you log prompts and outputs for debugging and auditing? -CI/CD for LLM workflows - what's different from ML? Cost & Latency Tradeoffs: -How do you reduce token usage? -When should you quantize a model? -What's your batching + caching strategy to reduce latency? -When to use hosted APIs vs open-source models? System Design Thinking: -How do you make an Al system more deterministic and less brittle? -What fallback do you use if the LLM fails mid-task? -Can you solve this without an LLM or vector DB? -What's the right database for this task - SQL, NoSQL, or vector? What more can I add in this?
- 20
@NainsiDwiv50980 ·
Ask any LLM about something that happened last week and watch it either make something up or tell you it doesn't know That's not a model problem. That's a memory problem. The model only knows what it was trained on, frozen at a point in time RAG fixes this without retraining anything. Here's the actual flow, step by step: 1. Your documents (files, websites, internal docs) get chunked into smaller pieces 2. Each chunk gets converted into a numerical embedding, basically a vector that captures its meaning 3. Those embeddings get indexed and stored in a vector database 4. When you ask a question, your query gets embedded the same way 5. The system does a semantic search, finds the top-K chunks that are actually similar in meaning, not just keyword matches 6. Those results get reranked, irrelevant stuff gets stripped out 7. The strongest, most relevant chunks get appended to your original prompt 8. That combined prompt (your question + real retrieved context) goes to the LLM 9. The model generates an answer grounded in actual data, not just its training memory The part people miss: the LLM never learns anything new. It's not being retrained. You're just handing it the right paragraph at the right moment, the same way you'd hand a coworker the one document they need instead of expecting them to memorize your whole company wiki Why it actually matters: → Answers are grounded in real data, not model memory → Knowledge stays current without retraining → Sources are traceable, you can point to exactly what the answer came from → Cheaper than fine-tuning every time your data changes → Scales across any domain, swap the knowledge base, same architecture Every serious AI product doing "chat with your docs" or enterprise search is running some version of this pipeline underneath
- 21
@0xlelouch_ ·
Top 10 mistakes devs make with RAG in production: 1) Treating chunking like formatting. Wrong size, no overlap, no structure-aware splits. 2) No stable doc IDs. Re-ingest creates duplicates, old chunks still rank. 3) Skipping metadata. No source, tenant, timestamp, ACLs, language, doc type. 4) No eval set. Shipping prompts without 50 to 500 labeled queries and expected citations. 5) TopK defaults forever. k=5, no reranker, no diversity, no freshness bias. 6) Ignoring token economics. 20 chunks stuffed into context, latency spikes, cost doubles, accuracy drops. 7) Weak retries + timeouts. Vector DB hiccup turns into a retry storm and thundering herd. 8) Bad indexing choices. Wrong distance metric, no filters indexed, hybrid search bolted on after. 9) No observability. Missing per-stage timing (embed, retrieve, rerank, generate) and hit rate by query type. 10) No guardrails on answers. Not requiring citations, not detecting no-answer, returning confident fiction on empty recall
- 22
@0xlelouch_ ·
Top 10 mistakes devs make with RAG systems: 1) No eval set. Shipping off vibes instead of measuring answer quality + citation accuracy. 2) Chunking by fixed size. Split mid-table/code block; lose headers; retrieval turns into mush. 3) No metadata filters. One index for everything, then wonder why HR docs answer incident questions. 4) Treating topK as a knob. Pull 30 chunks, blow context, and dilute the signal. 5) Mixing embeddings/models. Different tokenizers/spaces; cosine scores look fine, recall is trash. 6) Skipping hybrid search. Dense-only misses exact ids, error codes, and function names. 7) No reranker. You pay LLM tokens to read irrelevant chunks instead of ranking first. 8) Prompt has no contract. Doesn’t require citations, doesn’t say what to do when sources conflict. 9) Stale or inconsistent indexing. No dedupe, no delete handling, no versioning, old docs keep winning. 10) No observability. No per-query trace of retrieval set, scores, latency, token spend, and fallback rate [generated using my AI agent, shared so you can learn from it]
- 23
@vivek_naskar ·
Karpathy dropped a gist called LLM Wiki and it's worth reading if you do any kind of deep research or writing. The problem with RAG is that knowledge never accumulates. Every query starts from scratch, re-deriving the same connections from raw documents. His approach: the LLM incrementally builds a markdown wiki from your sources. When a new source comes in, it reads it, updates relevant pages, notes contradictions, and maintains cross-references across the whole wiki. Three layers: raw sources you never touch, an LLM-owned wiki, and a schema file that keeps the LLM behaving like a disciplined maintainer. The bottleneck for personal knowledge bases was never the reading. It was the boring maintenance work that nobody wanted to do. https://t.co/bk9UhAoo18
- 24
@IntuitMachine ·
🧵 THREAD: Why your RAG pipeline is probably backwards (and grep is eating vector search's lunch) 1/ Everyone's building the same retrieval stack: → Embed everything → Store in Pinecone/Weaviate → Query with cosine similarity → Inject top-K into context But a new study just flipped this playbook upside down. 2/ The setup: 116 questions across 10 agent configurations They tested lexical (grep) vs semantic (vector) search inside actual agent loops — not static RAG pipelines. The twist? They varied both the orchestration harness AND how results get delivered to the model. 3/ The shocking result: Inline grep beat inline vector search in every single model-harness pair. Not by a little. Some margins hit 23 percentage points. Your $10K/month vector database might be getting crushed by a bash command. 4/ "But semantic search understands meaning!" True... until you put it inside an agent that needs to: Issue multiple queries Filter noise Integrate results across tool calls Turns out grep's precision >> vector's fuzzy recall when the agent has to actually do something with results. 5/ The hidden variable nobody talks about: How results reach the model matters as much as what you retrieve. 📋 Inline delivery: dump everything into context 📁 File-based delivery: write to disk, model reads programmatically Same retriever, different delivery = completely different rankings. 6/ File-based delivery actually inverted some comparisons With Haiku + Amazon Bedrock harness: Inline: grep wins Programmatic: vector wins This means "retrieval quality" is meaningless without specifying the orchestration layer. 7/ Why grep wins on literal-span tasks: ✅ Agent can craft precise regex patterns ✅ Zero false positives when pattern matches ✅ No embedding drift or semantic confusion ✅ Instant feedback: match or no match It's like giving the agent a scalpel instead of a net. 8/ The harness matters MORE than you think Same data + same retriever + different CLI harness = 15+ point accuracy swings. Provider-native harnesses (Claude, Gemini CLIs) have invisible inductive biases that change how agents search. You can't A/B test retrievers without A/B testing harnesses. 9/ When noise scales up: They added 10 → 50 → 100 distractor sessions. The "grep always wins" rule broke down. Crossover point depended on BOTH harness strength AND model capability. Translation: You can't predict production behavior from offline metrics. 10/ Three immediate leverage points: 1️⃣ Add grep as a tool (takes 1 day, costs $0) 2️⃣ Try file-based delivery on weak models to reduce context rot 3️⃣ Expose BOTH tools simultaneously → let the agent choose per query Hybrid beats pure strategies without extra infrastructure. 11/ The contrarian take: "Default to vector" is expensive pattern-matching theater. The best retrieval strategy is the one your harness can reliably use — not the one with the highest offline recall@10. 12/ What this means for you: 🚫 Stop evaluating retrievers in isolation ✅ Test retrieval × harness × delivery as ONE system 🚫 Stop assuming semantic > lexical ✅ Match strategy to task distribution (literal spans ≠ conceptual synthesis) 🚫 Stop ignoring orchestration ✅ Harness choice = model choice in impact 13/ The future isn't better embeddings. It's understanding how agents actually use the tools we give them. Grep just proved that simple, precise tools + smart orchestration > sophisticated search + naive integration. 14/ Key limitation: This assumes answers are often verbatim spans. If your workload is heavy paraphrase/synthesis, the lexical advantage shrinks. But for memory-intensive QA, event timelines, and factual lookup? Grep eats. 15/ Bottom line: Before you scale your vector infrastructure, ask: "Have I tested grep with programmatic delivery inside my actual production harness?" The answer might save you 6 months and $100K. Paper: "Is Grep All You Need?" (Sen et al.)
- 25
@heygurisingh ·
Your RAG stack is paying rent on RAM it doesn't need to use. Someone just open sourced a vector index that fits 10 million documents into 4GB. The same corpus takes 31GB as float32. This thing fits it in 4GB AND searches it faster than FAISS. It's called turbovec. Built in Rust with Python bindings, sitting on top of Google Research's TurboQuant algorithm a data-oblivious quantizer that hits the Shannon lower bound on distortion with zero training and zero data passes. Here's what's inside this thing: → 16x compression on a 1536-dim vector. 6,144 bytes drops to 384 bytes → Hand-written NEON kernels on ARM and AVX-512BW on x86 that beat FAISS IndexPQFastScan by 12-20% → No codebook training. Add vectors, they're indexed. No rebuilds as the corpus grows → Pair it with any open-source embedding model for a fully air-gapped RAG stack. Nothing leaves your machine Here's the wildest part: The math is so clean it doesn't need your data. A random rotation makes every coordinate follow a known Beta distribution. Then Lloyd-Max scalar quantization buckets each coordinate using boundaries computed once, from the math, not the data. The paper proves this lands within 2.7x of the information-theoretic lower bound. You literally cannot do much better for a given number of bits. Most teams are paying thousands a month for managed vector DBs that do less than this. 946 stars. MIT License. 100% Opensource.
- 26
@JustAnotherPM ·
RAG is one of the most important AI concepts a PM can understand. Here’s the simple version: Instead of relying on what the model was trained on, RAG lets your AI search your data first — then answer. It’s how AI gets access to your docs, your knowledge base, your product data. No fine-tuning required.
- 27
@TheTuringPost ·
MathNet - a new interesting global multimodal benchmark from @MIT for mathematical reasoning and retrieval It's a dataset of 30,676 Olympiad-level problems from 47 countries, 17 languages, and 143 competitions over 4 decades, with expert solutions. It defines 3 tasks: - problem solving - math-aware retrieval - RAG Top models achieve 78.4% accuracy, while retrieval Recall@1 is ~5%. RAG improves performance up to 12%, highlighting retrieval quality limitations and shows embedding models struggle.
- 28
@hasantoxr ·
I didn't know you could benchmark your entire RAG stack against ChatGPT, Claude, and Gemini in one run. It's called EnterpriseRAG-Bench. 500k documents. 500 questions. The first benchmark built on data that actually looks like a company's data. Not Wikipedia articles. Not research papers. Slack messages where decisions got made but never documented. Google Docs that stopped being updated six months ago. Email threads. Call transcripts. The messy, inconsistent, real stuff your AI has to search through every day. Every other RAG benchmark tests retrieval on clean, long-form text with obvious answers. EnterpriseRAG-Bench tests it on ambiguous conversations, huge variations in document size, contradictory information, and multi-hop reasoning across hundreds of thousands of files. Plus baseline numbers for BM25, pure vector search, and an agent with bash access. If you're building a RAG system, this is the only benchmark that tells you how it actually performs. 100% Opensource.
- 29
@ahmadafterhours ·
Spent the week going deep on how AI agents actually learn. Here’s what I found: Most agents run on RAG — you embed your data, they retrieve similar chunks at query time. It works, but it has a ceiling. No relationships between facts. No sense of what changed. No reasoning across chains. Knowledge graphs fix this. Instead of retrieving chunks, agents traverse structure: prospect → company → tech stack → past objections → what converted Every fact is timestamped. When something changes, the old fact is invalidated — not deleted. Agents always reason from current truth. And every interaction writes back into the graph. A deal closes, a prospect responds, an outreach converts — all of it feeds the system. The result: agents that don’t just retrieve information. They understand your business, track how it evolves, and get sharper every day.
- 30
@itsafiz ·
PII in your RAG pipeline isn't just a compliance risk. It's a retrieval quality problem. Once names, emails, or SSNs are embedded in your vector store, you can't easily remove them. And they mess with your semantic search. @tonicfakedata Textual + @Haystack_AI just solved this. Here's how 🧵 👇
Watch video
- 31
@shivam74689 ·
Day 64 — Becoming AI Engineer Today I learned one of the most important lessons in building production RAG systems: When an AI system gives bad answers, the problem is usually not the LLM. The problem is often the evidence pipeline behind it. I spent today debugging and rebuilding the retrieval pipeline of my Enterprise Knowledge Graph Agentic RAG system end-to-end. The first challenge was not retrieval itself. It was getting the entire pipeline stable. Several integration issues between the Citation Manager, Context Compressor, Enterprise Agent, and Bootstrap process were preventing the system from running correctly. After fixing those connections, I moved from debugging the whole system to validating each component independently. I tested every retrieval layer: • Qdrant vector search for semantic retrieval. • BM25 keyword search for exact term matching. • Hybrid Retrieval combining semantic and lexical signals. • Reciprocal Rank Fusion (RRF) for merging multiple ranked results. Each component was returning relevant results individually, which confirmed the retrieval foundation was working. But then I looked deeper. Instead of only checking the final generated answer, I inspected the raw evidence flowing through every stage of the pipeline. That changed everything. The real problems were upstream. The PDF extraction process was producing noisy and poorly structured text. The chunking strategy was also creating problems: • Important definitions were being split across chunk boundaries. • Fixed-size character splitting ignored semantic structure. • Small chunks with low overlap were giving the LLM incomplete context. This reinforced a major principle: Bad ingestion creates bad retrieval. Bad retrieval creates bad generation. No amount of reranking or a stronger LLM can fully recover from poor-quality context. I also discovered a duplicate chunk issue. Old vectors from previous experiments were still stored inside Qdrant, while the chunking process was creating additional low-quality boundaries. To fix this, I completely reset the knowledge stores: • Cleared Qdrant vector storage. • Reset Neo4j graph storage. • Regenerated embeddings from scratch. • Created a new structured sample document designed specifically to stress-test enterprise retrieval. After rebuilding the index, I validated the complete retrieval flow again. The improvements were clear: • Retrieval returned distinct and relevant chunks. • Hybrid search correctly combined semantic and keyword-based matches. • Context compression successfully reduced multiple retrieved chunks into concise, useful context. One of the biggest lessons from today: Production AI engineering is less about continuously adding new components and more about making every existing component measurable, debuggable, and reliable. A RAG system should not be treated as a black box. You need visibility into every stage: Document → Extraction → Cleaning → Chunking → Embeddings → Vector Store → Retrieval → Compression → Generation When something breaks, debugging should happen stage by stage until the real failure point is found. Another important realization: Most production AI work is not writing new algorithms. It is understanding why systems fail, isolating root causes, and improving reliability. With the retrieval pipeline now validated, the next bottleneck is clear. The Neo4j knowledge graph extraction layer is currently returning zero entities. The next step is to debug and improve entity and relationship extraction so the system can combine: • Semantic retrieval. • Keyword retrieval. • Graph-based reasoning. towards true multi-hop enterprise reasoning. Every day, AI engineering feels less like building a chatbot and more like designing a reliable knowledge system. #AIEngineering #AgenticAI #RAG #KnowledgeGraph #Neo4j #VectorSearch #BM25 #EnterpriseAI #LLM #GenerativeAI #SoftwareEngineering #BuildingInPublic
- 32
@pauliusztin_ ·
I’ve spent the last week interviewing @maximilien, former CTO at IBM and Chairperson of NodeJS Foundation, who has shipped production RAG to multiple customers over the past year. The lesson he kept circling back to is that until you evaluate on your customer’s data, nothing else you do matters. Production RAG is a loop: stitch your embedding model, chunking, retrieval, vector DB, and judge, then evaluate and iterate until you hit your customer’s metrics. Public benchmarks and the MTEB leaderboard are signals, not verdicts. On a real customer dataset of Leica auction listings, an open-source sentence-transformer that ranked around #130 on MTEB still beat OpenAI by 11% in quality. It ran 240x faster, produced 50% smaller vectors, and cost $0.
- 33
@KSimback ·
Can a small open model with $3 of fine-tuning beat a production RAG setup? This is what I tested, and the results were impressive Based on a 100-question eval set, a fine-tuned version of Qwen3.5-9B outperformed Gemini Flash with RAG and Opus4.8/Sonnet 5 without RAG Working on a full write-up of the end-to-end flow
- 34
@JeremyCMorgan ·
Most RAG stops at corpus, retrieval, and injection. This small reference implementation adds output and enforcement layers, treating context as a versioned engineering artifact rather than a prompt pasted into chat. A useful pattern if you're designing an internal coding assistant that needs to respect local architecture decisions. https://t.co/IQqVHNpQCs
- 35
@pauliusztin_ ·
It's been 2 years since I wrote the LLM Engineers Handbook. Since then, I've seen many new RAG eval tools emerge, but there's a problem... Most of them overcomplicate everything with proprietary metric suites. But every RAG system has only 3 variables: Q → Question C → Context A → Answer And if you look at how these interact… There are exactly 6 relationships you can evaluate: 1/ C | Q → Context Relevance Is the retrieved context relevant to the question? 2/ A | C → Faithfulness Does the answer stick to the context? 3/ A | Q → Answer Relevance Does the answer solve the user’s question? 4/ C | A → Context Support Does the context fully support the answer? 5/ Q | C → Question Answerability Can this question even be answered with this context? 6/ Q | A → Self-Containment Can someone understand the question just from the answer? That’s the entire system. 3 variables → 6 relationships → 6 metrics. (Plus retrieval metrics that ensure you have the right context) Nothing more. And when your RAG system fails… It’s always because one of these 6 is broken. So instead of adding more evals, failures should be mapped to: • Retrieval issues • Generation issues • Or end-to-end mismatches I talk more about the 6 failure modes of RAG in lesson 6 of the AI Evals & Observability series in Decoding AI Magazine. Check it out here: https://t.co/jo2B67nDIo
- 36
@smratitiwa86867 ·
🚨 STOP using RAG for everything. You’re not building intelligence — you’re building a complex retrieval layer. More people are starting to realize: You don’t always need RAG. What actually works is much simpler: • INDEX.md as a central map • well-structured directories • an LLM that reads the right context at the right time I’ve been running this with 120+ knowledge files for months. The real advantage isn’t retrieval — it’s continuous improvement. Each interaction: → captures insights → refines hypotheses → updates rules The system doesn’t just read. It evolves. Over time, this compounds into better prompts, stronger frameworks, and validated patterns. At that point, it’s no longer a notes setup. It’s a personal research system. Most people store knowledge. Few build systems that improve with use. That difference compounds. Build systems that learn. 🚀
- 37
@shbhtngpl ·
i'm currently working on a RAG app which i chose to write it in golang mainly. the process is simple, i parse the pdf, create embeddings, upload it to postgres with pgvector and then we can use it for retrieval turns out, golang doesn't have the best support to parse pdfs like python does. so i ended up using python for parsing the pdf into meaningful chunks, write those to a file and then the ingest script, which i'm writing in golang will pick up the chunks from the file and do the next steps i also happened to try out make in the process where my parse commands run the python script and the ingest command will run the go script. will talk about this in another post
- 38
@sebbsssss ·
Got asked today: "Isn't @cludeproject just RAG?" Fair question. The read path retrieves, injects, generates. Same shape. But retrieval over a typed memory graph with temporal indexing, additive compaction, and on-chain provenance stops being RAG somewhere around the third layer. RAG fetches the nearest chunks by vector similarity and dumps them into the prompt. Clude retrieves through a graph where memories have typed edges, causes, contradicts, happens_before, elaborates, and traversal is weighted by edge type. Every memory carries an event date, a decay factor, and a Solana tx hash, None of that is a RAG feature. All of it changes what ends up in the LLM's context. RAG retrieves. Clude remembers.
Watch video
- 39
- 40
@sabir_huss50540 ·
🚨 @thewebAI just open-sourced ColVec1 and it’s a serious shift in document retrieval. #1 and #3 on ViDoRe V3, with Nvidia’s best open-source embedding model sitting in between. But the real story isn’t the leaderboard. Most RAG systems still depend on OCR → PDF to text → search. That breaks on real-world documents: tables, scans, charts, financial reports, medical files. ColVec1 skips OCR entirely. It retrieves directly from rendered document images understanding layout, structure, and visuals the way humans do. Built with a retrieval-first training recipe on ~2M query-image pairs, optimized for real-world performance instead of brute scale. This is what the next phase of AI looks like: less scale-first… more task-first.
- 41
@smratitiwa86867 ·
Most RAG systems fail the moment real users touch them. Because real-world retrieval is not: embed → retrieve → generate That works in demos. Production RAG breaks when: → the answer is scattered across 12 documents → embeddings miss industry-specific terminology → bad chunks quietly poison the response → relationships matter more than raw text → PDFs contain tables, charts, and screenshots your pipeline cannot even read This is why serious AI teams are moving beyond “Naive RAG”. The real shift happening in 2026 is not bigger models. It’s smarter retrieval architectures. Here are the 5 RAG patterns quietly becoming the foundation of enterprise AI systems: ━━━━━━━━━━━━━━━━━━━ 1. 𝗛𝘆𝗯𝗿𝗶𝗱 𝗥𝗔𝗚 Dense vectors understand meaning. BM25 understands exact keywords. The magic happens when both rankings merge together. → semantic retrieval + lexical retrieval → Reciprocal Rank Fusion (RRF) combines results → dramatically better recall in production This is becoming the default baseline for serious teams. ━━━━━━━━━━━━━━━━━━━ 2. 𝗚𝗿𝗮𝗽𝗵𝗥𝗔𝗚 Chunks are not enough when knowledge is relational. GraphRAG extracts: → entities → relationships → communities → connected concepts Instead of retrieving isolated chunks… the system retrieves subgraphs. This is how AI systems start answering: “how are these things connected?” rather than: “which paragraph contains the keyword?” Perfect for: research, finance, healthcare, compliance, enterprise knowledge systems. ━━━━━━━━━━━━━━━━━━━ 3. 𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗥𝗔𝗚 Retrieval stops being a single step. It becomes a reasoning loop. One agent plans: → vector DB? → SQL? → web search? → internal docs? Another agent verifies: → is the answer complete? → should we retry retrieval? → do we need another source? The important shift: RAG becomes orchestration. Not just search. ━━━━━━━━━━━━━━━━━━━ 4. 𝗖𝗼𝗿𝗿𝗲𝗰𝘁𝗶𝘃𝗲 𝗥𝗔𝗚 (CRAG) Most pipelines trust retrieval blindly. Production systems cannot afford that. CRAG introduces retrieval grading. → good retrieval → answer → weak retrieval → rewrite query → failed retrieval → fallback to web/tool search This is the architecture pattern most demos skip… but real enterprise systems desperately need. Because retrieval quality is the real bottleneck. ━━━━━━━━━━━━━━━━━━━ 5. 𝗠𝘂𝗹𝘁𝗶𝗺𝗼𝗱𝗮𝗹 𝗥𝗔𝗚 The future of enterprise knowledge is not text-only. Real documents contain: → charts → diagrams → scanned PDFs → screenshots → tables → UI images Multimodal RAG indexes all of it together. One embedding space. One retrieval system. One multimodal model. No more broken “OCR + text-only” hacks. ━━━━━━━━━━━━━━━━━━━ The most advanced AI stacks in 2026 will not choose ONE of these. They will combine them. Think about the architecture direction: → Hybrid retrieval for accuracy → Agentic orchestration for reasoning → Corrective grading for reliability → Multimodal indexing for real-world data → Graph retrieval for connected knowledge That combination is where the industry is heading. Naive RAG is not the finish line anymore. It’s the “hello world” tutorial. And honestly… this is why most enterprise GenAI projects stall after the demo phase. The problem was never just the model. The problem was retrieval architecture.
- 42
@agenticgirl ·
Everyone is optimizing RAG the same way: Better retrieval → better answers. This paper : Retrieval Improvements Do Not Guarantee Better Answers: A Study of RAG for AI Policy QA breaks that assumption. Here’s what they did: • Improved retrieval (higher Recall@k, MRR) • Fine-tuned the generator (DPO alignment) • Built a solid, domain-specific RAG pipeline Everything you’re supposed to do. But the outcome? • Retrieval improved • Answer quality barely moved • Hallucinations became more confident Why this happens is the real insight: When the correct document isn’t in the corpus, RAG doesn’t fail. It improvises. The retriever finds the closest match. The model treats it as truth. And the answer sounds perfectly right even when it isn’t. That leads to a dangerous pattern: • Fluent answers • Relevant context • Incorrect conclusions The system looks reliable until you actually verify it. The paper highlights where things break: • Missing data → answers built on partial evidence • Similar language → wrong sources get mixed in • Complex queries → incomplete retrieval • Weak uncertainty → no “I don’t know” behavior Even with: • Better retrievers • Preference-aligned generation the system doesn’t become meaningfully more reliable. Meanwhile, larger models still perform better not because of retrieval, but because they’re better at reasoning + calibration. RAG doesn’t fail because it can’t find answers. It fails because it can’t admit when the answer isn’t there. Until that’s fixed, better retrieval will just make wrong answers sound more convincing. Here's the link to the Paper : https://t.co/a5N3Kk9qUE
- 43
@NaadhLabs ·
today's topics - RAG.2 1. llm follows U-based learning pattern · it is better at things that come first and last · bad at middle things comparatively · this is where Context Engineering comes 2. Context rot · something bad happening with the context, as tokens increase, the performance of LLM goes down Approach to improve this = NIAH (Needle in a Haystack) -needle, the answer we want -haystack, all the context , the model has been given But, it's just a retrieval task. It's not checking about semantics (meaning) and logical (familiar) so, we have to eval semantic retrieval and non-lexical retrieval How is it tested on different models? we have to test models on increasing token length https://t.co/TQX920syl1
- 44
- 45
- 46
@mchulet ·
As an AI Engineer. Please learn >Harness engineering, not just prompt engineering >Context engineering, not just long prompts >Prompt caching vs. semantic caching tradeoffs >KV cache management, eviction, reuse, and memory pressure at scale >Prefill vs. decode latency and why they optimize differently >Continuous batching, paged attention, and throughput optimization >Speculative decoding vs. quantization vs. distillation tradeoffs >INT8, INT4, FP8, AWQ, GPTQ, and when quantization hurts quality >Structured output failures, schema validation, repair loops, and fallback chains >Function calling reliability, tool contracts, argument validation, and idempotency >Agent guardrails, loop budgets, tool budgets, and termination conditions >Model routing, graceful fallback logic, and degraded-mode UX >RAG architecture: chunking, embeddings, hybrid search, reranking, and freshness >Retrieval evals: recall, precision, grounding, attribution, and citation quality >Evals: golden sets, regression tests, adversarial tests, LLM-as-judge, and human evals >LLM observability as a first-class discipline: traces, spans, tokens, latency, errors, and drift >Cost attribution per feature, workflow, tenant, and user journey not just per model >Safety engineering: prompt injection defense, data leakage prevention, and permission boundaries >Multi-tenant isolation, cache safety, and cross-user context contamination prevention >Fine-tuning vs. in-context learning vs. RAG vs. distillation and when each is the wrong tool >Latency, quality, cost, and reliability tradeoffs across the full inference stack >Production failure modes: hallucinated tool calls, malformed JSON, stale retrieval, runaway agents, and silent eval regressions
- 47
@Redisinc ·
Your RAG pipeline is probably embedding the same documents more than once. Duplicate embeddings are easy to miss. They don't throw errors. They just quietly degrade retrieval quality, inflate storage costs, and get worse as you scale. Here’s a practical guide to idempotency patterns for LLM apps, covering where duplicate work sneaks in and how to stop it before it becomes a production problem: https://t.co/O60rBX3bZ2
- 48
- 49
- 50
Best Tweets by Topic