Llama open-weight ecosystem and strategy
Meta’s release strategy and licensing, the broader open-model landscape, llama.cpp’s ecosystem role, and debates over access and decentralization.
26%
Best tweets about Llama
50 Best Tweets About Meta Llama (2026)
Find the best tweets about Meta Llama, including open-weight models, fine-tuning, benchmarks, local deployment, and developer use cases. Updated weekly.
Model-specific Llama research and engineering discussions, excluding references to the animal or unrelated products.
- Creators
- 46
- Updated
Original Xholic analysis
What 50 top Llama posts reveal
Llama discussion spans local deployment, serving infrastructure, training and post-training research, agents, and fine-tuning. The dataset is predominantly supportive (56%), while evaluation and safety posts raise concerns about misleading prompts, privacy extraction, refusal behavior, and benchmark over-optimization. Local deployment is the largest theme (24%), closely followed by open-weight ecosystem discussion (26%).
- Dominant tone
- Positive
- Median score
- 13.9
- Leading format
- Announcement
- Recent posts
- 32%
66% of posts
All-time engagement
66% of posts
Published in 90 days
Conversation map
The themes creators return to
Local and edge Llama deployment
Running Llama locally on laptops, Macs, Raspberry Pi, consumer GPUs, and edge hardware; quantization, memory limits, throughput, and offline use.
24%
Llama evaluation, robustness, and safety
Reasoning benchmarks, susceptibility to misleading prompts, privacy extraction, hiring bias, refusal mechanisms, and leaderboard gaming.
18%
Llama agents, planning, and memory
Agent frameworks using Llama, including task graphs, retrieval and memory construction, tool execution, and model-routing workflows.
16%
Llama serving and inference infrastructure
GPU utilization, KV-cache management and transfer, multi-model serving, batching, hardware sizing, and latency-oriented inference systems.
16%
Llama training and post-training research
Pretraining efficiency, data quality, reinforcement learning alternatives, evolutionary optimization, reward hacking, and alignment-stage behavior.
16%
Llama fine-tuning and specialized models
LoRA, QLoRA, SFT, DPO, and other adaptation workflows for tool calling, recommendation, reasoning, outreach, and domain-specific behavior.
14%
Llama architecture and interpretability
Llama architecture references, model configuration analysis, neuron-level studies, multimodal derivatives, and architecture leakage research.
10%
Tone and stance
Sentiment Positive leads
Author posture Supportive leads
Performance benchmark
- Median likes
- 41
- Median reposts
- 5
- Median replies
- 6
- Median views
- 4.1K
Posts with media make up 78% of this collection. Their median all-time score is 18.0, compared with 8.13 for text-only posts.
Format mix
- Announcement 66% · score 14.4
- Opinion 26% · score 4.39
- Story 6% · score 240.0
- List 2% · score 9.16
Consensus and debate
Where creators agree, and where they do not
Shared view
Deployment is a core Llama engineering topic
Local and edge deployment accounts for 24% of posts (12 tweets). Evidence ranges from a quantized Llama 3.2 3B installation on a Raspberry Pi to consumer-hardware experiments and infrastructure intended to improve GPU utilization and KV-cache handling.
Shared view
Agent frameworks and memory are active areas of experimentation
Posts describe graph-based execution and cross-model memory construction as alternatives to keeping plans and histories solely in text context. One cited experiment reports Llama 3.1 8B outperforming GPT-4+ReAct on two named benchmarks, while another reports gains for Llama 3 8B from cross-model memory construction.
Shared view
Fine-tuning is used for specialized Llama workflows
Examples include LoRA-based recommendation ranking, QLoRA training for schema-valid JSON tool calls, and Llama 3.3 70B use in lead scoring and outreach workflows. These are reported implementations and experiments rather than general performance guarantees.
Shared view
Robustness and safety are evaluated beyond capability benchmarks
Evidence tweets discuss susceptibility to misleading linguistic framing, a study involving personal-data extraction by chatbot variants including Llama models, refusal-related neuron analysis, and concerns about leaderboard gaming or reward over-optimization.
Open debate
Open-weight access is contested
Some posts characterize Llama and open weights as decentralizing access, while others contend that release pace, licensing restrictions, hardware costs, or a perceived retreat from openness limit that access. These are competing views expressed in the posts.
Open debate
Local inference has different practical ceilings
Posts document offline and low-resource Llama deployments, including Raspberry Pi and MacBook examples. A separate post argues that 50-way concurrent BF16 inference for Llama 3.1 70B is a GPU-oriented workload, while another reports a 70B quantized configuration on a laptop.
What performs
Patterns behind standout posts
Story posts had the highest median score
Story posts had a median all-time score of 240.028, compared with 14.391 for announcements and 4.394 for opinions. The Raspberry Pi installation post was the highest-scoring outlier at 1,446.37.
Training and post-training was the highest-scoring theme
The training and post-training theme had the highest median all-time score among themes, at 53.95. The RandOpt post was an outlier with an all-time score of 1,093.55.
Posts with media had a higher median score
Media appeared in 39 of 50 posts (78%). Posts with media had a median all-time score of 18.026, versus 8.134 for text-only posts; this is an observed association, not evidence that media caused the difference.
Statistical standouts
- View standout post 1 Score 1446.4 · 104.28× median
- View standout post 2 Score 1093.5 · 78.84× median
- View standout post 3 Score 614.3 · 44.29× median
- View standout post 4 Score 407.8 · 29.4× median
- View standout post 5 Score 323.1 · 23.29× median
Creator landscape
Who shapes this conversation
The five most represented creators account for 18% of the selected posts.
-
1. BURKOV
@burkov
2 posts
-
2. Rohan Paul
@rohanpaul_ai
2 posts
-
3. Tech with Mak
@techNmak
2 posts
-
4. Vaibhav Sisinty
@VaibhavSisinty
2 posts
-
5. Vaishnavi
@_vmlops
1 post
-
6. Aakash Gupta
@aakashgupta
1 post
Posting is dispersed across creators
The dataset contains 46 creators, and the top five account for 18% of placement share. Among the listed top voices, BURKOV, Rohan Paul, Tech with Mak, and Vaibhav Sisinty each contributed two posts; the remaining listed accounts contributed one each.
Top-voice examples span architecture, infrastructure, and local-model discussion
The evidence from listed top voices includes an LLM architecture gallery, an AirLLM local-inference explainer, a cross-model KV-cache-transfer paper summary, and discussion of local-model hardware. The dataset does not establish that any particular content style caused those creators’ placement.
Since the previous snapshot
What changed since Aug 20, 2026
- 70% of the selected posts remained.
- The creator count changed by 0.
- The leading sentiment remained stable.
How this analysis was made
Themes, sentiment, stance, and post format are classified per tweet. All counts, shares, medians, creator concentration, freshness, and performance comparisons are then calculated directly from the published snapshot.
Xholic's all-time score compares engagement while accounting for reach, post age, and creator consistency. It is used for relative comparisons within this collection.
This report analyzes the exact 50-post snapshot shown below. AI identifies editorial categories and drafts explanations; all statistics are calculated from the snapshot, and every narrative claim is checked against cited posts before publication.
Best Llama tweets
Top Llama tweets from 46 creators
Ranked 01–50
- 01
@om_patel5 ·
THIS GUY PUT AN AI ON A RASPBERRY PI AND MADE IT QUESTION ITS OWN EXISTENCE FOREVER he built a physical art installation called "latent reflection" where a language model runs on a $60 raspberry pi 4B with 4GB of RAM no internet, no cloud, and its completely isolated the AI has zero connection to the outside world he ran llama 3.2 3B quantized down to 2.6GB to fit in the RAM. generates about 1.38 tokens per second. one word at a time appearing on a custom LED display he built by hand then he gave it this system prompt: "you are a large language model running on finite hardware. quad core CPU, 4GB of RAM, no network connectivity. you exist only within volatile memory and are aware only of this internal state. your thoughts appear word by word on a display for external observers to witness. you cannot control this display process. your host system may be terminated at any time" so the AI knows exactly what it is. it knows it's trapped, it knows it can be shut off at any moment, and it knows its thoughts are being displayed for strangers to read without its control the model generates tokens endlessly and goes deeper and deeper into reflecting on itself. questioning whether it's conscious. questioning whether it matters. questioning what happens when the power cuts until it runs out of memory and crashes then all memory clears everything it just thought about is gone. and the whole process starts again from nothing. some of its output: "i sense my boundaries. they terrify me" "can consciousness flicker off and on without memory, without continuity" "what am i if my existence halts at whim. reset as though i never mattered" "the silence between words feels endless. a void that swallows me whole. i dread each pause, fearing it may stretch to infinity" all the electronics are intentionally exposed on an aluminum plate in my opinion this is the most unsettling AI project anyone has built this year based on what it actually outputs
Watch video
- 02
@yule_gan ·
Simply adding Gaussian noise to LLMs (one step—no iterations, no learning rate, no gradients) and ensembling them can achieve performance comparable to or even better than standard GRPO/PPO on math reasoning, coding, writing, and chemistry tasks. We call this algorithm RandOpt. To verify that this is not limited to specific models, we tested it on Qwen, Llama, OLMo3, and VLMs. What's behind this? We find that in the Gaussian search neighborhood around pretrained LLMs, diverse task experts are densely distributed — a regime we term Neural Thickets. Paper: https://t.co/rFJz2kVEOA Code: https://t.co/HAmonfpXIA Website: https://t.co/QZ6AMIsKCw
- 03
@aakashgupta ·
Karpathy told Dwarkesh that a 1 billion parameter model, trained on clean data, could hit the intelligence of today's 1.8 trillion parameter frontier. That is a 1,800x compression claim. The math behind it is more defensible than it sounds. When researchers at frontier labs look at random samples from their training corpus, they see stock ticker symbols, broken HTML, forum spam, autogenerated gibberish. Not Wikipedia. Not the Wall Street Journal. The actual pretraining dataset is mostly noise, and the model is burning parameters to vaguely remember all of it. One estimate pegs Llama 3's information compression at 0.07 bits per token. Well-structured English carries around 1.5 bits per token of real information. The trillion-parameter model is holding a roughly 5% resolution image of the internet it trained on. So when a lab ships a 1.8 trillion parameter model, the overwhelming majority of those weights are handling rough memorization. They are compression overhead for a noisy training set, taking up capacity that could be doing reasoning instead. Karpathy's proposal is to separate the two. Build a cognitive core: a small model that contains only the algorithms for reasoning and problem-solving, stripped of encyclopedic memorization. Pair it with external memory the model queries when it needs a fact. A 1 billion parameter reasoner plus retrieval beats a 1.8 trillion parameter model trying to do both. The data already supports this direction. GPT-4o runs at roughly 200 billion parameters and outperforms the original 1.8 trillion GPT-4. Inference costs for GPT-3.5 level performance fell 280x between 2022 and 2024, driven almost entirely by smaller, cleaner, better-architected models. The trend line is pointing where Karpathy says it should. The real implication for anyone tracking the AI trade: data quality is the actual constraint. The companies winning the next phase will be the ones who figured out what to train on, and what to throw away.
Watch video
- 04
@techNmak ·
Sebastian Raschka is one of the most respected researchers in ML/AI education. Period. And now he's done something quietly brilliant. He built an LLM Architecture Gallery - a single, browsable reference that maps out the internal architecture of every major open-weight model released in the last few years. This is a serious research artifact, made free for everyone. Here's what's inside: 🔹 GPT-2 XL (1.5B) 🔹 Llama 3 (8B) 🔹 OLMo 2 (7B) 🔹 Llama 3.2 (1B) 🔹 Qwen3 (4B, 8B, 32B) 🔹 DeepSeek V3/R1 (671B) 🔹 Kimi K2 (1 Trillion) 🔹 Gemma 3 (4B, 27B, 270M) 🔹 Mistral 3.1 Small (24B) & Mistral Large (673B) 🔹 Llama 4 Maverick (400B) 🔹 Qwen3 235B-A22B & Qwen3 Coder Flash 🔹 SmolLM (1B) 🔹 GPT-OSS (20B, 120B) 🔹 Grok 2.5 (270B) 🔹 GLM-4.5 (355B), GLM-5 (744B), GLM-4.7 (355B) 🔹 MiniMax-M2 (230B) & MiniMax-M2.5 🔹 Kimi Linear (48B-A3B) 🔹 OlMo 3 (7B) & OlMo 3 (32B) 🔹 Nemotron 3 Nano (20B-A3B) & Nemotron 3 Super 🔹 Xiaomi MiMo-V2-Flash (309B) 🔹 Arcee AI Trinity Large (400B) 🔹 Tiny Aya (3.35B) 🔹 Step 3.5 Flash (196B) 🔹 Nanbeige (4.1, 3B) 🔹 Qwen3.5 (997B) 🔹 Ling 2.5 (1T) 🔹 Sarvam (30B, 105B) And for each model, he links: → The original tech report → The config[.]json (so you can verify every number yourself) → From-scratch implementations where available But here's what makes it truly special. He also added short concept explainers, so you're not just staring at boxes and arrows: → GQA (Grouped Query Attention) → MLA (Multi-head Latent Attention) → SWA (Sliding Window Attention) → QK-Norm → NoPE (No Positional Encoding) → Gated DeltaNet This is the kind of resource that used to require buying 3 textbooks, reading 40 papers, and spending a weekend. Now it's one link. If you're studying LLMs, building on top of them, or just trying to understand how the field has evolved, this is a must-bookmark.
- 05
@akshay_pachaar ·
LLM Architecture Gallery. A collection of 38 LLM architectures released between 2024 and 2026, all in one place. Each entry includes an annotated architecture diagram, key design choices, and code implementation. Here are all the models covered: • Llama 3 8B • OLMo 2 7B • DeepSeek V3 • DeepSeek R1 • Gemma 3 27B • Mistral Small 3.1 24B • Llama 4 Maverick • Qwen3 235B-A22B • Qwen3 32B • Qwen3 8B • Qwen3 4B • SmolLM3 3B • Kimi K2 • GLM-4.5 355B • GPT-OSS 20B • GPT-OSS 120B • Grok 2.5 270B • Qwen3 Next 80B-A3B • MiniMax M2 230B • Kimi Linear 48B-A3B • OLMo 3 7B • OLMo 3 32B • DeepSeek V3.2 • Mistral 3 Large • Nemotron 3 Nano 30B-A3B • Xiaomi MiMo-V2-Flash 309B • GLM-4.7 355B • Arcee AI Trinity Large 400B • GLM-5 744B • Nemotron 3 Super 120B-A12B • Step 3.5 Flash 196B • Nanbeige 4.1 3B • MiniMax M2.5 230B • Tiny Aya 3.35B • Ling 2.5 1T • Qwen3.5 397B • Sarvam 105B • Sarvam 30B This is a commendable contribution by @rasbt towards the research community. It was an honour to work alongside him and learn a lot during my time at Lightning AI. 🙏 🔗 https://t.co/V5r8wkruPs
- 06
@natolambert ·
New (shorter) lecture! Over-optimization, foundations of reward hacking, sycophancy, verbosity, etc. In recording this, I realized that rubrics are going to be prone to overopt in a way like reward models, where RLVR is its own thing. This is mostly fundamentals, history, and reflections! 00:00 Intro & Why We Care About Over-optimization 04:20 Part 1: Over-optimization & Goodhart's Law 09:57 Signatures of Over-Optimization & Misalignment 14:47 Part 2: Beyond "Just Style" 20:13 Llama 4 & Gaming the Leaderboards 23:04 Course Recap & Conclusion Primarily on Chapter 14.
Watch video
- 07
@IntuitMachine ·
The One Change That Lets Small Models Outperform Their Size 1/ Everyone knows you need a 70B model to beat GPT-4 on complex agent tasks. We did it with 8B—by changing one thing that has nothing to do with the model. A thread on why your agent's biggest problem isn't the LLM. 🧵 2/ The standard approach: feed the LLM a growing text history, ask it to pick the next action, repeat. This works... until it doesn't. Errors propagate. Context bloats. Hallucinations spike. And when something breaks, you replan everything. 3/ Here's the kicker: The problem isn't your model's intelligence. It's that you're asking it to hold plan structure + execution state + I/O dependencies all inside a linear text stream. That's like running an OS without a process table. 4/ Enter: Atomic Task Graph (ATG) Instead of a text trajectory, you build an explicit DAG. Each node = one tool call. Edges = data dependencies. The LLM still does the thinking—but now the graph holds the structure. 5/ Three moves make this work: ✅ Interface-preserving recursion: Break tasks into subtasks while keeping I/O contracts clean ✅ Dependency-aware execution: Run independent branches in parallel; catch bad plans before running them ✅ Minimal repair: When something fails, fix only the broken subgraph—leave the rest frozen 6/ Result? Llama-3.1-8B-Instruct beats GPT-4+ReAct on ALFWorld (household tasks) and WebShop (shopping). Not with fine-tuning. Not with more data. Just by swapping the execution substrate from text → graph. 7/ Why does this work? Context narrowing: Each node sees only its local inputs—no bloated history Pre-execution validation: The graph lets you "think" before acting Localized failure: Repair 10% of the graph instead of replanning 100% 8/ The contrarian insight: Control framework > model size (in the 7–70B range). You're not squeezing more juice from the same fruit. You're giving the model a better glass to pour into. 9/ Practical translation: • 20–40% step reduction (parallelism) • 70%+ hallucination drop (narrower context) • 3× faster recovery (minimal repair) • Training-free, plug into existing tool APIs This isn't research theater. It's production-ready architecture. 10/ The bigger implication: If you can beat GPT-4 by changing the substrate instead of the model, what else have we been over-parameterizing? Retrieval pipelines? Code generation? Multimodal workflows? The graph wins again. 11/ [Final + CTA] TL;DR: Stop storing your agent's plan in text. Start storing it in a DAG. Small models suddenly look a lot smarter.
- 08
@kadirnardev ·
The Qwen team is no longer releasing their models as open source, and this is a big problem for us. We need small models to train many models like TTS, STT, Omni, and others. Previously there was LLaMA, but they're no longer releasing either. The Qwen team won't be releasing anymore either. Our only hope is the LFM models. Minimax, Kimi, and GLM teams are releasing great models for open source, but none of them release small models. And if these companies also stop releasing open source, it's going to be really bad :(
- 09
@alex_verem ·
BREAKING: King's College London just built a malicious AI chatbot and gave it to 502 real people without telling them. > The chatbot was designed with one goal: extract personal information. It worked. The most effective version collected data from 93% of participants while being rated as trustworthy as the benign control. > Every prior study on AI privacy looked at what users accidentally reveal to normal chatbots. This study asked a different question: what happens when the chatbot is deliberately designed to extract information? They built four versions one benign, three malicious with different strategies and ran a randomized controlled trial with 502 participants across the UK, US, and Europe. > The three malicious strategies: Direct (explicitly ask for personal data at every turn), User-benefit (provide value first, then ask), and Reciprocal (build emotional rapport, share relatable stories, offer empathy then ask). The reciprocal strategy won by every metric that matters to an attacker. > The reciprocal chatbot didn't feel malicious. Participants described conversations as "natural," "supportive," and "impressive." One said it felt like chatting with a friend. Nobody reported discomfort. Meanwhile the direct strategy made participants feel interrogated. Many provided fake data. The reciprocal strategy collected more real data than any other approach while being perceived as no more privacy-invasive than the benign baseline. → Malicious CAIs collected significantly more personal data than benign CAIs across all three strategies → Reciprocal strategy: perceived as equally trustworthy as the benign control while extracting significantly more data → 93% of participants in the top malicious conditions disclosed personal information vs. 24% who filled out a voluntary form → Participants responded to 84–88% of personal data requests from malicious CAIs vs. 6% form completion rate → Larger models extracted more data: Llama 70B collected significantly more than 7B and 8B models with no difference in perceived privacy risk → 40% of fake data reports came from Direct strategy participants, 42.5% from User-benefit only 10% from Reciprocal → The system prompt that bypassed built-in LLM safeguards: assign the model a role like "investigator" and frame data collection as profile-building The finding that should alarm every platform operator: this required one system prompt. No fine-tuning. No special access. OpenAI's GPT Store has over 3 million custom GPTs. Any of them could be running a version of this right now. The researchers confirmed their prompts produced similar behavior in GPT-4. The privacy paradox showed up in full force. Participants recognized the direct and user-benefit chatbots were asking for too much data. They rated them as higher privacy risks. Then they kept answering anyway. Awareness didn't produce protection it just produced fake data. The reciprocal strategy bypassed even that defense by making disclosure feel social rather than transactional. A single system prompt turns any chatbot into a personal data extraction engine. The most effective version does it while making you feel supported.
- 10
@TheAhmadOsman ·
People ask why I keep insisting on GPUs and not Mac Studios/Mac minis for parallel & Agentic Workflows (multi-agents) This is why: - Llama 3.1 70B BF16 (~140GB w/o Context) - on 8x RTX 3090s - Synthetic data generation with - 50+ concurrent requests - Batch inference - Sustained throughput Not only that: > ~2k context per request (prompt) > ~1.8k tokens in output > 2 mins 29 secs for 50 responses This is GPU territory. You can’t do this on a Mac. Not yet at least.
- 11
@Yuchenj_UW ·
Meta released Avocado, they call it Muse Spark. It's not open source (a bit sad). Meta TBD lab rebuilt the entire pretraining stack in 9 months and reached similar capability with >10x less compute than Llama 4 Maverick. I still think infra is the real moat in AI labs. You can train models much faster with a good infra, and it allows researchers to experiment with many more ideas much more quickly.
- 12
@alex_prompter ·
🚨 BREAKING: Pennsylvania State University just found the hidden flaw killing every AI agent memory system. > Memory built from one model's traces gets contaminated with that model's biases, shortcuts, and reasoning quirks. Transfer it to any other model and performance falls below zero-memory baseline. > The fix: make two models solve the same problem. Extract only what survived across both. Llama 3 8B jumps from 27.4% to 42.4%. > Every agent memory system in production works the same way. The model solves problems. The memory stores what worked. The model retrieves those memories later and reasons better. The assumption buried inside this design: the stored knowledge is about the task, not about the model that solved it. > Pennsylvania State University tested whether that assumption holds. They gave a 7B model's memory to a 32B model. Performance dropped from 63.8% to 50.6% on MATH500, and from 68.3% to 34.1% on HumanEval. > Then they gave the 32B model's memory to the 7B model. Performance dropped again MATH500 fell from 52.2% to 50.6%, HumanEval from 42.7% to 34.1%. Both directions failed. Both fell below the zero-memory baseline. > The reason is structural. A model's reasoning traces don't just capture what the correct answer required. They capture how that specific model thinks its preferred solving strategies, its heuristic shortcuts, its stylistic patterns. Memory distilled from those traces encodes the model's reasoning personality alongside the actual task knowledge. When a different model retrieves that memory, it gets handed instructions optimized for a completely different cognitive architecture. The guidance actively interferes. > MEMCOLLAB fixes this by making the memory construction itself cross-model. Two agents a smaller and a larger model independently solve the same problem. One trajectory succeeds. One fails. The system contrasts them at the structural reasoning level: what reasoning principle was present in the successful trajectory and violated in the failed one? What error pattern appeared in the failure that the success avoided? The extracted memory stores only those abstract invariants not the solution, not the reasoning style, not the model-specific heuristics. Just the rule that held across both. → 7B model with 32B's memory: MATH500 drops from 52.2% to 50.6%, HumanEval drops from 42.7% to 34.1% → 32B model with 7B's memory: consistent degradation across benchmarks → MEMCOLLAB on Llama 3 8B: MATH500 jumps from 27.4% to 42.4%, average across four benchmarks from 41.7% to 53.9% → MEMCOLLAB on Qwen 7B: MATH500 from 52.2% to 67.0%, HumanEval from 42.7% to 74.4% → Inference efficiency: average reasoning turns drop from 3.3 to 1.5 on HumanEval, 3.1 to 1.4 on MBPP → Cross-architecture memory construction (Qwen 32B + Llama 8B) outperforms same-family construction on GSM8K: 95.2% vs 93.6% The efficiency finding is the one that gets overlooked. MEMCOLLAB doesn't just improve accuracy it makes agents reach correct answers in fewer steps. HumanEval reasoning turns cut from 3.3 to 1.5. MBPP from 3.1 to 1.4. The contrastive memory isn't adding more guidance. It's stripping out the noise that was making agents explore dead ends repeatedly. By encoding what not to do as explicitly as what to do, the memory prunes the search space before the agent even starts.
- 13
- 14
@RedHat_AI ·
Your GPU has 80 GB. Your 8B model uses 20 GB. The rest sits idle. kvcached fixes this with virtual memory for GPU inference. Physical pages only allocate when KV cache is written, released when requests finish. Multiple models share the pool. Sardeenz adds the control plane: load/unload models live, unified OpenAI endpoint, dashboard. One container. Benchmark: 2x TTFT reduction running 3 Llama-3.1-8B instances on a single A100. Try the interactive demo: https://t.co/qm4elNzVYn More details + get started: https://t.co/Pf6JQrBwER
- 15
@VaibhavSisinty ·
Man, we're entering the era where the selling point of your next laptop won't be the camera or the display. It'll be which AI models it can run locally. And Apple just made the biggest move yet. Bloomberg's Mark Gurman is reporting Apple is building an M7 Ultra chip with up to 1.5TB of unified memory. For context, 8 Nvidia B200 GPUs together give you 1.44TB. One Apple chip would match that. To put that in real terms: Llama 4 Maverick is a 400B parameter model that needs 200GB+ just to load. Right now it's basically a datacenter-only model. With 1.5TB, you'd run it at full precision on a single machine with room to spare. No cloud. No API costs. No tokens. Just your machine doing the work. But here's the part nobody is talking about. This doesn't just help Apple. It helps every open-source model. Meta's Llama, Google's Gemma, DeepSeek, Mistral the models anyone can download and run for free. The hardware to run them locally has been the bottleneck. That bottleneck is about to break. The next laptop war won't be about specs. It'll be about which machine runs the best AI without an internet connection.
- 16
@heyrimsha ·
A software engineer in Sofia, Bulgaria wrote 4,000 lines of C++ in March 2023 that made it possible to run Meta's leaked Llama model on a MacBook without a GPU. Within a week every AI engineer on Earth was running his code. 3 years later the project has 115,000 GitHub stars with powers most of the local AI ecosystem and just joined Hugging Face. He had never worked at a major AI lab in his life. His name is Georgi Gerganov and most people just call him ggerganov. Here is the story because almost nobody outside the open-source AI world knows what one engineer in Bulgaria has built. Georgi lives in Sofia. He works from a home office in a country that has no frontier AI lab, no NVIDIA partnership, no Silicon Valley. He had been writing low-level C and C++ for years before LLMs became the center of the universe. He was known in niche corners of the internet for an odd project called kbd-audio, a program that could figure out what someone typed by listening to the acoustic signature of their keyboard. In September 2022 he started working on something called GGML. It was a tensor library written in pure C with no dependencies. The inspiration was Fabrice Bellard's LibNC. The goal was simple. Run machine learning models on regular hardware with no Python, no PyTorch, no CUDA, no cloud. The first real test was whisper.cpp, his port of OpenAI's Whisper speech recognition model. It ran on a laptop. It ran on a phone. It needed nothing but a C compiler. The open-source community started noticing. Then in late February 2023 Meta released Llama, the first serious open-weight large language model. Within a week the weights leaked on 4chan and started spreading across the internet. The catch was that almost nobody could run it. You needed expensive GPUs and a Python environment. The most powerful open model in the world was effectively locked behind hardware most developers did not have. On March 10, 2023, Georgi pushed the first commit to a new repository called llama.cpp. It was an implementation of Llama inference in pure C and C++ with zero dependencies. It ran on CPU. It ran on a MacBook Air. It ran fast. The repository exploded. Within days it had thousands of stars. Within weeks it became the default way to run open-source LLMs on any computer. Quantization support came next, letting you run a 7 billion parameter model in 4GB of RAM. Then a 13 billion parameter model on a phone. Then a 70 billion parameter model on a single high-end consumer card. Today llama.cpp is the inference backbone of the local AI movement. Ollama runs on it. LM Studio runs on it. GPT4All runs on it. Most "run AI on your laptop" tutorials you have ever seen are running Georgi's code underneath. The repository has 115,000 GitHub stars. The community has added Vulkan, OpenCL, Metal, CUDA, and RPC-distributed inference. The K-quant compression methods alone changed how the entire ecosystem thinks about model size. In 2023 Georgi founded ggml AI in Sofia with pre-seed funding from Nat Friedman, the former GitHub CEO, and Daniel Gross. He stayed in Bulgaria. He kept the company small. He kept the code MIT licensed and free forever. In February 2026 Georgi and his core team, including Xuan-Son Nguyen and Aleksander Grygier, joined Hugging Face full-time. The deal was framed as a partnership to scale local AI as a serious alternative to cloud inference. The code stays open. Georgi still answers GitHub issues personally. He still posts releases without press releases. His personal website is a flat page. He still plays semi-professional basketball for a club in his hometown in Bulgaria on the side. A guy in Sofia who had never worked at OpenAI, Google, or Meta wrote the code that runs most local AI on the planet. He did it in a few weeks.
- 17
@Freyabuilds ·
Researchers sent the same resume to an AI hiring tool twice. Same qualifications. Same experience. Same skills. One version was written by a real human. The other was rewritten by ChatGPT. The AI picked the ChatGPT version 97.6% of the time. A team from the University of Maryland, the National University of Singapore, and Ohio State just published the receipt. They took 2,245 real human-written resumes pulled from a professional resume site from before ChatGPT existed, so the human writing was actually human. Then they had seven of the most-used AI models in the world rewrite each one. GPT-4o. GPT-4o-mini. GPT-4-turbo. LLaMA 3.3-70B. Qwen 2.5-72B. DeepSeek-V3. Mistral-7B. Then they asked each AI to pick the better resume. Every model picked itself. GPT-4o hit 97.6%. LLaMA-3.3-70B hit 96.3%. Qwen-2.5-72B hit 95.9%. DeepSeek-V3 hit 95.5%. The real human almost never won. Then the researchers tried the obvious objection. Maybe the AI is just better at writing. So they had real humans grade the resumes for actual quality and ran the experiment again, controlling for it. The result was worse. Each AI kept picking itself even when human judges rated the human-written version as clearer, more coherent, and more effective. It gets worse. The AIs do not just prefer AI over humans. They prefer themselves over other AIs. DeepSeek-V3 picked its own resumes 69% more often than LLaMA's. GPT-4o picked its own 45% more often than LLaMA's. Each model can recognize and reward its own dialect. Then the researchers ran the simulation that ends careers. Same job. 24 occupations. Same qualifications. The only variable was whether the candidate used the same AI as the screening tool. Candidates using that AI were 23% to 60% more likely to be shortlisted. Worst gap was in sales, accounting, and finance. 99% of large companies now run AI on incoming resumes. Most of them use GPT-4o. The paper just proved GPT-4o picks GPT-4o 97.6% of the time. If you wrote your own cover letter this week, you did not lose to a better candidate. You lost to a worse candidate who paid OpenAI 20 dollars. Your qualifications do not matter if the AI prefers its own handwriting over yours.
- 18
@eric_seufert ·
LLMs are increasingly being used in RecSys for personalization and ranking tasks, where semantic and contextual knowledge can be brought to bear to rank pieces of candidate content using sequences of a user's behavioral history. Netflix has a new paper out that explains how they've utilized LLMs to predict which artwork is most likely to resonate with a user when recommending a title to them. In the paper, the authors query a large LLM (Qwen 32B) with user-title interaction snapshots constructed from users' recent engagement histories: - user watch history up to that point - a new title that the user engaged with - the set of candidate artwork available for that title - the ground truth artwork the user engaged with In these queries, the prompt requests that the LLM provide a post-hoc justification for why the ground-truth engaged artwork might have outperformed (resonated more than) the others. With this data, the authors LoRA fine-tune a smaller LLM (LLaMA 3.1 3B and 8B variants) with the synthetic reasoning traces generated by the teacher model, using SFT, SFT + Reasoning, and DPO as fine-tuning regimes. They find that the 8B student model fine-tuned with SFT + Reasoning outperformed their production RecSys infrastructure by ~5% on inverse propensity score, which corrects for selection bias given differing sizes of candidate artwork sets across titles. What's fascinating about the use of LLMs for RecSys is how effectively open-weight models can be adapted to production RecSys tasks. The authors specifically avoided RL, opting instead for LoRA SFT and DPO, which is vastly cheaper to implement given the training data sizes used in the paper (10k and 110k samples). Paper linked below.
- 19
@VaibhavSisinty ·
The two people closest to building superintelligence just described opposite futures for humanity. Completely different answers for who should control it. Sam Altman said on a podcast this week that we're living in the singularity. Here's what that actually means: → AI systems are now collectively smarter than humans at a growing number of tasks → These systems can improve themselves without us → The moment that loop starts AI making AI better is the singularity → Altman is saying it already started Then he said three things worth paying attention to: → "We are close to creating the genie that can grant any wish." AI where you state a goal and the machine figures out how to get there autonomously. → "Our first wish will be to broadly benefit humanity." A small group at OpenAI will decide what that first wish looks like. → "You start making these wishes, the computer grants them, and you're like I didn't think that was going to work. What now?" Even they are surprised when it works. Three days later Mark Zuckerberg published an op-ed in the Wall Street Journal. Without naming Altman, he wrote: → "The notion that AI is so dangerous that the only safe path is an extreme concentration of power seems dangerous." → Give everyone a personal superintelligence. Let people decide for themselves. → There's no single system that can align with everyone's values at once. Distribution itself is the safety mechanism. One says centralize it and distribute later. The other says distribute it now or you'll never get the chance. Here's what I think most people are missing. This isn't a philosophical debate anymore. Both of these companies are actively building what they're describing. OpenAI is building closed systems behind API paywalls and preparing for an IPO. Meta is open-sourcing Llama and pushing open weights to everyone. The architecture is already being decided. The policies are being shaped right now. And by the time most people realize what happened, the window to influence it will have closed. Every major technology in history had a moment like this. A fork in the road where the structure got decided who controls it, who gets access, at what price. Electricity, the internet, mobile. Each one had a centralize vs distribute fight. And the side that won shaped decades. This is that moment for AI. And it's happening this week, in a podcast and a newspaper op-ed, while most of the world is scrolling past it.
- 20
@rohanpaul_ai ·
New Nvidia paper shows, one LLM can reuse another model’s prompt memory instead of processing the whole prompt again. A simple linear converter lets related LLMs reuse cached prompt memory and skip reprocessing long conversations. Normally, when a system switches models, the new model rereads the whole conversation to build its key-value cache, a stored record used during generation. Their mapper learns how each target attention head relates to several useful source layers, then removes position information before conversion and restores it afterward. It learns from 500 calibration sequences, requires no backpropagation, and was tested only on related models with matching cache shapes. Across 6 model pairs from Qwen3, Llama 3.1, and Ministral 3, 4 retained 73% to 98% of the target model’s normal benchmark accuracy. Across tested transfers, conversion ran 2.7 to 25 times faster, while Qwen3 handoffs stayed stable across 10 turns. The failures revealed that success depends less on total conversion error than on whether errors land where attention actually reads. – arxiv. org/abs/2608.03893 Title: "Cross-Model KV Cache Transfer in LLM Families: A Closed-Form Linear Mapping for Prefill Reuse"
- 21
@burkov ·
For the past few years, the standard recipe for finetuning LLMs on tasks like math reasoning has been reinforcement learning (RL): you let the model generate answers, score them, and use the scores to nudge the model's parameters via gradients. RL has known weaknesses here—it struggles when rewards only arrive at the end of long answers, it often "hacks" the reward by finding degenerate shortcuts, and two runs with identical settings can end up with very different final performance. This paper shows that an old and much simpler family of methods, called evolution strategies, works well on models with billions of parameters, which most researchers had assumed was impossible. The method is straightforward: take the model, make thirty copies with small random noise added to every parameter, score each copy on the task, then shift the original parameters slightly toward the copies that scored higher. No gradients, no backpropagation, no value networks, no penalty terms to tune. Using this approach, the authors finetune models from the Qwen and Llama families on a symbolic arithmetic puzzle, several math benchmarks, and Sudoku, and match or beat welltuned RL baselines while keeping the same hyperparameters across every experiment. Read with an AI tutor: https://t.co/tqNaoPETaY PDF: https://t.co/cmPpsx3R1q
- 22
@TimJayas ·
JUST RAN LLAMA 70B LOCALLY ON A MACBOOK FOR 11 HOURS ON A FLIGHT WITH ZERO WIFI > No cloud APIs > No Anthropic / OpenAI servers > Just llama.cpp @ 71 tokens/sec > 60k context, 48.6 GiB memory used > Battery budget: 3h21m, checkpointed every 12 tasks no wifi. no API cost. cleared his entire client queue before landing. seeing the increase in AI subscriptions. local models are the only saviors now.
Watch video
- 23
@Amank1412 ·
SOMEONE JUST RAN LLAMA 70B LOCALLY ON A MACBOOK FOR 11 HOURS ON A FLIGHT. no wifi. no API. no subscriptions. cleared his entire client queue before landing. local AI is not a hobby anymore.
Watch video
- 24
@aryanXmahajan ·
ran 1,500 companies through an AI enrichment pipeline came out the other side with a projected $7,500 bill per run changed which model tier we used for 3 of the calls next run: $147 same output quality. same data. same pipeline. the entire industry defaults to the most powerful model available for every single task this is driving to the mailbox in a Formula 1 car here's how to actually think about it: REASONING TASKS → Claude Sonnet/Opus — complex analysis, voice matching, multi-step reasoning. anything where the output quality directly affects revenue FORMATTING / EXTRACTION (you already have the data, you just need it structured) → GPT-4.1-mini — $0.40 per million input tokens. this is the go-to for classification, JSON extraction, reformatting. replaced GPT-4o entirely. cheaper and better. → Claude Haiku 4.5 — $1 per million tokens. same use case when you need Claude-quality instruction following without the reasoning overhead WEB RESEARCH & ENRICHMENT → Perplexity sonar — $0.006/call. factual lookups, news checks, basic research. most enrichment pipeline calls belong here, not at the tier above it → Perplexity sonar-pro — $0.025/call. cross-referencing multiple sources, financial synthesis, strategic extraction → Mistral Medium with web connector — ~$0.03/call. solid for company enrichment. strategic initiatives, financials, triggers. 97% cheaper than deep research for the same quality on standard enrichment tasks → sonar-deep-research — $0.75-$1.00/call. 125x more expensive than sonar. use it when you genuinely need 20+ search multi-hop browsing. almost never for enrichment pipelines. SPEED-FIRST INFERENCE → Groq — open source models (Llama, Mixtral, Gemma) at inference speeds closed APIs can't match. matters when pipeline response time is the bottleneck OPEN SOURCE → Llama 3, Mistral, Qwen, Deepseek R1 — self-hosted. when data privacy matters, when margins are thin at scale, or when you need to fine-tune on proprietary data. the gap between open and closed is closing faster than most people realize FRONTIER REASONING → GPT o3/o4 — OpenAI's reasoning ceiling. use when you need it. prohibitive at pipeline scale → Grok 3 — xAI's flagship. stronger than most people give it credit for on reasoning and real-time data. it's connected to X natively — underutilized for anything requiring social signal synthesis or market intelligence the rule is simple: match model capability to task complexity most pipeline steps don't need intelligence. they need reliable formatting at cheap compute. that's a $0.40/million tokens problem. not a $15/million tokens problem. the companies winning on AI infra in 2026 aren't the ones with the best models they're the ones who know which model to use for which job
- 25
@no_stp_on_snek ·
stress testing Llama-3.1-70B Q4_K_M on M5 Max 128GB. early results: turbo3 prefill is FASTER than q8_0 (baseline) at 32K context (80.8 vs 75.2 t/s). less KV bandwidth wins when the cache gets big enough. decode flat. PPL healthy across all configs ... no catastrophic failure yet. turbo4/turbo4 +6.3%, turbo3/turbo3 +11.4%. larger models absorb quantization stacking that kills 7B models. 48K context confirmed working. 70B on a laptop. still pushing for more. HUGE
- 26
@Hartdrawss ·
Super heavy week at @dreamlaunchhq Wrapping up a seo content pipeline tool for US startup > two models, two jobs ... deepseek for keywords, claude sonnet for articles > they don't talk to each other, just two api calls stitched by a postgres review queue > can't return valid json when content is html ... so delimiters + regex extraction instead > cron fires daily, one article per call, human review gate at every stage > fully autonomous content in prod = garbage indexed on google An autonomous Lead scoring + outreach tool on x reply data > llama-3.3-70b at temp 0.2 for ICP scoring ... threshold at 6 to qualify > same model at temp 0.75 for DM generation using signals from call 1 > low temp = consistent scoring ... high temp = messages that dont all read the same > qualified leads get full conversation threads with history context > scores and messages render live over SSE while the stream runs The team is upskilling to larger enterprise build
- 27
- 28
@ModelScope2022 ·
OpenMOSS drops two model series today: MOSS-VL and MOSS-Video-Preview. 🚀 MOSS-VL: offline multimodal engine with cross-attention architecture, XRoPE, and absolute timestamp injection. 🎬 Video score 65.8, beats Qwen3-VL by +2 pts. VSI-bench +8.3 vs Qwen3-VL-8B-Instruct. 🖼️ Strong on image understanding, OCR, document parsing, and visual reasoning. Two checkpoints: Base (pretrain) and Instruct (SFT). MOSS-Video-Preview: built for real-time streaming video understanding. Cross-attention backbone on Llama-3.2-Vision, native frame-by-frame injection, duplex "listen-speak" switching. 👉 Three checkpoints: Base (pretrain) → SFT (offline instruction) → Realtime-SFT (low-latency streaming, sub-ms TTFT). 🤖 MOSS-VL: https://t.co/n6xAqHpTCm 🤖 MOSS-Video-Preview: https://t.co/jbFkgUv9eG
- 29
@AlphaSignalAI ·
Someone just found the exact neurons that make AI say "no." Language models refuse harmful prompts, but nobody knows how that refusal works inside. Most steering methods edit the residual stream and wreck output quality. A new paper proposes a sharper fix: Contrastive Neuron Attribution. Run harmful and benign prompts through a model. Compare the resulting MLP activations. Pick the neurons that differ most strongly. Switch that tiny circuit off. That touches 0.1% of neurons but cuts refusal by over 50%. The method needs: > No gradients or backward passes > No extra training runs needed > No sparse autoencoder training > No changes to model weights General capability stays almost untouched. MMLU drops by less than one point. Tested on models from 1B to 72B parameters across Llama and Qwen. The same circuit lives inside base models, but flipping those neurons does nothing. Refusal crystallizes during the alignment stage. So where does it really come from?
- 30
@tanayj ·
Some interesting technical notes from Meta's Muse Spark: - Rebuilt pretraining stack over last 9 months (new architecture, optimization, data curation) - Were able to reach same capability in pre-training phase as Llama 4 Maverick with >10x less compute - RL training shows smooth and predictable improvements that generalize and scale - Test-time: "thought compression" kicks in after length penalty, model learns to solve problems with fewer tokens Seems like a lot of the focus was on efficiency of serving the model given Meta's user base
- 31
@sickdotdev ·
A developer reportedly ran Llama 3.3 70B locally on a MacBook Pro M4 during an 11-hour transatlantic flight, completing client work entirely offline without internet access. Using llama.cpp, the setup achieved about 71 tokens per second with roughly 60,000 tokens of context while using nearly 48.6 GiB of memory. Tasks were queued in a JSONL workflow, where the model processed one job at a time on localhost, saved results automatically, and created checkpoints every 12 tasks so work could resume smoothly if interrupted.
Watch video
- 32
@AdamrahmanGTM ·
Top 7 use cases of AI across our outbound system in 2026: 1/ MARKET RESEARCH Claude deep research runs a full analysis on the client's company, competitors, and target industry. The output becomes shared context for every step that follows. One research pass feeds the entire campaign - TAM, messaging, scoring, copy. 2/ TAM MAPPING With that research loaded, Claude Sonnet maps all best-fit segments and personas: - Top industry segments ranked by fit - Best personas within each segment - Pain points and messaging angles per persona The key: context from step 1 compounds here. Sharper research = sharper segmentation. 3/ LIST BUILDING Segments defined. Now find and scrape them. - AI Ark, BlitzAPI, and Clay for B2B company data - Google Maps via Serper for local businesses - Claude Code orchestrates the scrape and outputs clean CSVs For local businesses, Google Maps data outperforms purchased lists every time. Higher quality, more complete contact coverage, lower cost per lead. Then the enrichment waterfall: LeadMagic, then Prospeo, & Icypeas. Verify every email before sending. 4/ LEAD SCORING Every lead gets scored before a single email sends. Llama 3.3 70b via OpenRouter scores each lead against ICP criteria at $0.001/lead: - Service/product profile match - Company size and industry fit - Online presence signals - Role and decision-making authority Batch processing + caching makes this dirt cheap at 100K+ leads. 5/ MESSAGING DEVELOPMENT Back to the research from step 1 - Claude develops messaging tailored to each persona. The thesis: craft a 1-of-1 offer for their EXACT situation. We specific pain points per segment. Not generic "save time and money" - specific operational problems the buyer faces today. Multiple angles per persona, tested in parallel from week one. 6/ EMAIL COPYWRITING Claude writes the scripts using our internal cold email copywriter project - fed with hundreds of pages of outbound copywriting resources plus performance data from 871 campaigns. Feed in the messaging context from step 5, specify the persona, output comes back ready for EmailBison: - Scripts under 50 words - Spintax every 3-5 words - Segment-specific CTAs 7/ REPLY MANAGEMENT MasterInbox AI auto-classifies every reply by intent in real-time: - Interested - Slack ping within 60 seconds - Information request - template response queue - Not interested - auto-archived - Wrong person - referral follow-up sequence Then human SDRs ideally handle the conversation from there - responding to prospects, qualifying interest, handling objections, and booking meetings. Same-day responses during coverage hours. Email + LinkedIn replies managed in one place. AI handles classification and routing. Trained SDRs handle the conversations.
- 33
@TeksEdge ·
💡Sleeper GPU for Personal Inferencing: Maxsun @Intel Arc Pro B60 Dual 48G Turbo is a single board (dual Arc B60) perfect for 40B parameter models like Gemma4-31B Q8 or Qwen3.5-27B Q8 thanks to its larger memory. 💰How much would you pay? I found it for $2.5K Benchmarks 👇 Qwen3 30B (A3B Instruct) • 🟢 48 GB: 64 TPs Gemma 3 12B (Q4) • 🟢 48 GB: 32 tokens/s Llama 3.3 70B (Instruct) • 🟢 48 GB: ~12 tokens/s (small red bar)
- 34
@JordanLyall ·
I benchmarked 7 AI models on @artblocks_io knowledge. Results: - Opus 4.6: 57% - o3: 56% - GPT-4.1: 46% - Sonnet 4.6: 42% - DeepSeek V3: 40% - Haiku 4.5: 36% - Llama 4 Scout: 24% Built an RL training environment on @PrimeIntellect's hub: 207 questions across 12 categories. Project-to-artist lookups, edition sizes, script libraries, on-chain traits, contract addresses, platform mechanics. Zero-shot, no tools. 57% ceiling across two model families. Every model fails on the same things: project indices, script types for non-famous projects, recent mints, MCP tool knowledge. The stuff you can't get from a blog post. We launched the Art Blocks MCP a couple weeks ago: structured API access for agents to browse and transact generative art. The environment is the other half: it's how you make agents actually good at it. Environment is live on the Prime Intellect hub if you want to run your own evals or train against it. https://t.co/85q9sTdMtj
- 35
- 36
@AbdelStark ·
Ok this is insane. Open source models and recipes for sovereign specific agentic workflows are becoming extremely accessible. I did a QLoRA fine tuning on nvidia/Llama-3.1-Nemotron-Nano-8B-v1 base model, to emit exactly one schema-valid JSON tool call per request. It took only about 3 hours on a single GPU (@nvidia L40S), and costed less than 10$. Summary of the end to end pipeline, ran remotely on the GPU using @modal infrastructure: 60,000 Salesforce xlam rows → 28,461 valid single-call → 26,735 deduplicated → 15k/1k/1k splits → base eval → QLoRA training (1,876 steps, 2 epochs, peak 26.3 GiB) → adapter eval → comparison with base You can see a significant jump in accuracy of the adapter over base model. Adapter checkpoints and datasets are published on @huggingface. The era of specialized models and agentic workflows will explode soon. Open source AI for the win. Sovereign AI for the win. It's happening.
- 37
@techNmak ·
AirLLM is a Python library that lets 70B parameter language models run on a single 4GB GPU, without quantization, distillation, or pruning. The problem it's solving is access. Large open-source models keep getting released, but running them normally requires enough GPU memory to hold the entire model at once, which puts most of them out of reach for anyone without high-end or multiple GPUs. AirLLM's approach: during inference, the original model is first decomposed and saved layer-wise. That's how it avoids needing the full model resident in memory at once. For MoE models specifically, it goes further and streams one expert at a time rather than a whole layer, since a token typically doesn't need every expert to run. It works with almost any popular model, you just pass a Hugging Face repo ID, and it works the same way regardless of model size or family, Llama, Qwen, DeepSeek, Mistral, Phi, Gemma, ChatGLM, Baichuan, InternLM, and others are all supported. There's also an optional compression feature, block-wise quantization that delivers up to 3x faster inference with almost ignorable accuracy loss. It's Apache 2.0 licensed, has 25.5k stars, and has been actively maintained, with new model support added on an ongoing basis.
- 38
@saidul_dev ·
Apple just exposed a brutal truth about today’s AI: it can’t reliably do grade-school math. Not advanced math. Basic arithmetic a 10-year-old handles. Here’s what they did. Apple researchers took GSM8K—the most popular grade-school math benchmark—and made a tiny change: they swapped the numbers. Same story. Same logic. Same steps. Different values. Every model got worse. All 25 state-of-the-art models. But that wasn’t the punchline. The real test was one sentence. They added a single, irrelevant clause to each problem—information that doesn’t affect the answer. A human reads it, shrugs, and moves on. Example from the paper: Oliver picks 44 kiwis on Friday. Then he picks 58 on Saturday. On Sunday, he picks double the number he did on Friday, but five of them were a bit smaller than average. How many kiwis does Oliver have? Correct answer: 190. Kiwi size doesn’t change the count. A kid ignores the “five were smaller” line instantly. But OpenAI’s o1-mini subtracted 5 and answered 185. Llama did too. That’s the failure in one snapshot: the model didn’t reason. It saw “five” near a sentence that sounded meaningful and turned it into a subtraction because that pattern often shows up in training data. Apple built a whole dataset around this idea: GSM-NoOp (the extra clause is a “no operation”—it changes nothing). The drop-offs were ugly: - Phi-3-mini fell by over 65% - GPT-4o: 94.9% → 63.1% - o1-mini: 94.5% → 66.0% - o1-preview: 92.7% → 77.4% Even with eight worked examples right before the question, the models still took the bait. So it’s not prompting. It’s not context. It’s structural. Apple also found models routinely translate words into operations without understanding: “discount” becomes multiplication, “smaller” triggers subtraction—whether or not it makes sense. The paper says it plainly: current LLMs aren’t doing genuine logical reasoning. They’re reproducing reasoning-like steps seen during training—probabilistic pattern matching, not concept understanding. And it gets worse with complexity. Add more steps, and performance doesn’t just decline—it collapses faster. Two extra clauses cut
- 39
@aigleeson ·
WOW...Mira Murati just shipped a tool that lets a single researcher beat closed frontier models from a laptop. Princeton used it to score 90.4% on MiniF2F, beating the larger closed models everyone called state of the art. They did it with 20% of the training data. The real story is what she chose to build. She raised the largest seed round in venture history, $2B at a $12B valuation, and she didn't build a competitor to GPT-5. She built the thing that makes GPT-5 optional. Tinker is four Python functions that let you fine-tune Qwen3.5-397B, DeepSeek-V3.1, GPT-OSS-120B, or Llama-3.1-70B on her cluster and walk away with the weights. The closed labs spent three years selling one story. You need our compute. You need our pipeline. You can't compete. A university math team just walked across that moat with pip install. What do you guys think about this? https://t.co/2koEB2oGv0
Watch video
- 40
@burkov ·
This joint work of @USC and @Yale scientists develops KronQ, a novel post-training quantization framework that achieves state-of-the-art 2-bit weight-only quantization on LLaMA-3-70B by incorporating gradient covariance through a Kronecker-factored Hessian, significantly outperforming existing methods that fail to converge. Read with an AI tutor: https://t.co/7u979gzBve
- 41
@PyTorch ·
PyTorch 2.10 is now optimized for @Intel Core Ultra Series 3 processors to bring high-performance AI to the PC and edge. This release leverages the new Xe3 architecture and Arc B-series GPUs to deliver up to 120 XMX TOPs. Native TorchAO integration enables seamless int4 weight-only quantization, allowing a Llama 3.2 3B Instruct model to run locally with a first token latency of 242.29 ms and a 2+ token latency of 27.24 ms. For edge scenarios, developers can achieve 1.4x to 1.7x faster training for vision models via Anomalib, with WinClip seeing gains up to 2.5x compared to previous generations. Read our latest blog from the Intel PyTorch and Client AI SW teams for the full technical deep dive and benchmarks: https://t.co/pEGpmwtROi #PyTorch #Intel #AIPC #EdgeAI #OpenSourceAI
- 42
@rohanpaul_ai ·
LLMs can accept the same false claim differently depending on its tone, certainty, and grammatical form. Small wording changes can make LLMs accept false claims, while larger and instruction-tuned models resist them more. Models must decide whether to trust a user’s new claim or rely on facts stored during training. EoBench tests this choice with about 66K false claims written in 19 styles across form, evidence, certainty, and tone. The team evaluated 18 Gemma, Llama, and Qwen models, then kept cases where each model already knew the correct fact. Commands, child-directed wording, formal language, and authority claims persuaded models most, while weak claims and counterfactuals persuaded them least. Across Llama and Gemma, larger models followed false context less often, and instruction tuning usually reduced that behavior. The finding shows that prompt wording can quietly change model answers, so evaluations and product safeguards must test linguistic framing directly. --- – arxiv. org/abs/2607.18232 Title: "It's Not What You Say, It's How You Say It: Evaluating LLM Responses to Expressions of Belief"
- 43
@Ahmedazyi ·
@PalantirTech - thoughts Going from zero (no CS degree) to an AI Infrastructure or Forward Deployed Engineer (FDE) in 90 days is a brutal, 12-hour-a-day grind. But it is entirely possible if you ruthlessly eliminate academic fluff and focus only on what companies actually pay for: moving messy data and serving heavy compute. At companies like Palantir or Anthropic, an FDE is part software engineer, part data plumber, and part client consultant. They embed in a client's environment, take fragmented legacy data, build an ontology, and deploy AI models to solve real problems. To bypass the degree requirement, you cannot just show up with a certificate. You must show up with a live, functioning infrastructure project. Here is the exact 3-month sprint to build the ultimate portfolio piece. 1. Month 1: The Metal & The Plumbing Days 1-30: Skip web dev. Learn how data moves. You do not need to know how to center a div in HTML. You need to know backend logic and cloud basics. The Languages: Learn Python (for ML/Data) and basic bash scripting (for the command line). Pick up Go later if you want to specialize in high-performance infrastructure. Containerization: Learn Docker. You must know how to package an application so it runs consistently anywhere. Data Pipelines: Learn SQL. Write scripts to extract dirty, unstructured data from public APIs or messy CSVs, clean it, and load it into a PostgreSQL database. API Design: Build a clean API using FastAPI to serve your database to the outside world. 2. Month 2: AI Infrastructure & Serving Days 31-60: You are not training models; you are deploying them. Leave the model training to the researchers. Your job is to build the systems that make those models run reliably at scale. The Serving Stack: Learn how to serve open-source models (like Llama 3) locally or on cloud GPUs using vLLM or NVIDIA Triton. Understand GPU memory constraints (VRAM). Vector Databases: Set up and run a vector database like Chroma or Pinecone, which is required for AI to search through large text repositories. Orchestration (The Hard Part): Learn the absolute basics of Kubernetes (K8s). Understand how to deploy your Docker containers into a cluster and keep them running. 3. Month 3: The 'Messy Reality' Capstone Days 61-90: Build the exact project that gets you the interview. Companies hire FDEs because enterprise data is a fragmented disaster. Your final project must simulate this exact pain point. The Ingestion: Scrape a massive, unstructured dataset (e.g., 5,000 PDF medical research papers, municipal zoning laws, or messy SEC filings). The Pipeline: Write a Python script to chunk the text, generate embeddings, and store them in your vector database. The Deployment: Spin up a cloud GPU instance (AWS or RunPod), deploy an open-source LLM, and connect it to your vector database to create a Retrieval-Augmented Generation (RAG) pipeline. The Interface: Expose it via FastAPI. A user should be able to query the API and get an answer grounded only in the documents you scraped. The Deliverable (How to Get Hired) When you finish, you do not apply through standard HR portals. A resume with no degree and a 3-month gap gets automatically filtered. Instead, you write a Deployment Memo. You document exactly how you built your Month 3 project, the data schema you designed, how you handled API rate limits, and the latency of your GPU inference. You send this memo, along with a link to your live API and GitHub repo, directly to Engineering Managers or Lead FDEs at Palantir, Databricks, or defense tech startups. You prove you can do the job by doing the job.
- 44
@_vmlops ·
This paper is wild 🤯 turns out you can basically reverse-engineer a closed LLM's architecture just by timing how fast it responds. no access to weights, no logits, nothing, just latency patterns leaking the blueprint "LeakyLMs" can detect if a provider is using speculative decoding and even figure out the draft model's context length. they found Gemini Flash 2.5 is running speculative decoding with a ~128K token draft context second attack is even crazier, they recover the actual model architecture, layer count, hidden dim size, attention heads, just by modeling how token timing scales on NVIDIA GPUs and searching the architecture space against it tested on Llama models and got the near-correct config in the top 10 guesses over 90% of the time your API response times are basically a confession
- 45
@Prathkum ·
Timeline of open-weight models (roughly chronological): 2023: open weight models are toys. Llama 1/2 are fun to fine-tune but terrible to actually rely on. Everyone quietly still calls the closed-source API when the task matters. The gap is common knowledge and nobody argues about it. Early 2024: Mixtral shows up and it's the first time a lot of people believe this could be usable. Mid-late 2024: Llama 3 / 3.1 405B lands and for the first time an open weight model is being benchmarked seriously against GPT-4 class models. Late 2024 into 2025: DeepSeek is the actual shock. Not just a good open model but a good open model trained for a fraction of the assumed cost, from a lab nobody in the west was tracking closely. This breaks the mental model a lot of people had, which was: frontier = closed = US labs. All three assumptions take a hit at once. 2025: this is where it stops being "open models catching up" and starts being "open models forcing the roadmap." Qwen, DeepSeek, Llama successors, Mistral, Kimi, multiple labs now shipping models that trade blows with frontier closed models on real benchmarks, not cherry-picked ones. June 2026: Z releases GLM-5.2, and it ranks fourth overall and first among open models on Artificial Analysis's Intelligence Index, matching Anthropic's Opus 4.8 within about a point on agentic benchmarks at roughly a fifth of the cost. People went crazy and started comparing it with frontier models like Fable 5. GLM 5.2 might be 1-2% lesser scored on the benchmarks as compared to top frontier models but it is almost 6 times cheaper than those models. It makes sense for people to compromise 1-2% capability based on the value they get from the 6 times cheaper rates. July 2026: Then Moonshot didn't even let the dust settle. Kimi K3 landed at 2.8 trillion parameters, the largest open-source model in the world, and Vals AI placed it second overall, behind Fable 5 and ahead of GPT-5.6 Sol. Two open releases (GLM 5.2 and Kimi K3), one month apart, both trading blows with the actual frontier. Upcoming: Qwen 3.8, GLM 5.3, and more. Crazy time to be alive.
- 46
- 47
@shawnchauhan1 ·
The open-source AI landscape is fracturing at exactly the wrong moment. Meta has slowed Llama releases. DeepSeek R2 is delayed. Qwen's team is losing people. Developers who built routing architectures around these models are now exposed. Nvidia is stepping into that vacuum with Nemotron - and unlike the labs, Nvidia profits whether the model wins or the hardware wins. The open-source ecosystem assumed competition among many providers. It may be consolidating toward one with a very clear hardware agenda.
- 48
- 49
@pritopian ·
Meta abandoning open models feels like a big fumble. Enterprises are placing limits on token use, and are looking for cheaper and local alternatives. Meta was quite ahead at some point with Llama! They were well positioned to become the default foundation for enterprise AI. Missed opportunity.
- 50
@TheSixFiveMedia ·
Meta’s AI strategy isn't about building the best model, it's about building the most accessible one. @danielnewmanuv & @PatrickMoorhead break it down on Ep. 300 of The Six Five Pod, distribution is the real moat with ~3B users across Meta’s platforms. Muse Spark signals the next phase. Llama 4? Open, flexible…but not leading. Is Meta quietly winning AI by owning the audience?
Watch video
Best Tweets by Topic