Evaluation and Validation
Evaluation-led fine-tuning: baselines, benchmark construction, automated sweeps, measurable before/after results, teacher evaluation, and regression testing.
38%
Best tweets about AI Fine-Tuning
50 Best Tweets About AI Fine-Tuning (2026)
Find the best tweets about AI fine-tuning, from datasets and LoRA to evaluation, alignment, training costs, model behavior, and deployment results.
Technical model fine-tuning, data preparation, LoRA, training, evaluation, alignment, cost, and demonstrated results.
- Creators
- 42
- Updated
Original Xholic analysis
What 50 top AI Fine-Tuning posts reveal
Discussion of AI fine-tuning frequently covers evaluation and data preparation, LoRA/PEFT specialization, and SFT-to-RL post-training recipes. The supplied posts also raise concerns about transfer, retention, possible misalignment, and the operational maintenance of custom models.
- Dominant tone
- Positive
- Median score
- 24.0
- Leading format
- Announcement
- Recent posts
- 48%
64% of posts
All-time engagement
52% of posts
Published in 90 days
Conversation map
The themes creators return to
RL Post-Training
Reinforcement-learning post-training methods, including SFT-to-RL recipes, DPO/GRPO/PPO, reward design, online learning, and reasoning improvement.
36%
Parameter-Efficient Fine-Tuning
LoRA, QLoRA, PEFT variants, adapter training, and parameter-efficient specialization across language, vision, audio, video, robotics, and time-series models.
28%
Training Efficiency and Cost
Training-system efficiency: VRAM reduction, GPU utilization, distributed training, kernel and glue-code optimization, packing, checkpointing, and cost control.
22%
Fine-Tuning Safety and Generalization
Alignment, generalization, and retention risks from fine-tuning, including emergent misalignment, representation collapse, catastrophic forgetting, and weak transfer.
20%
Training Data Engineering
Creating, cleaning, filtering, formatting, augmenting, and distilling supervised training datasets; emphasis on synthetic data and data quality over volume.
20%
Specialized Small Models
Fine-tuning small models into narrow, high-performing local specialists for tasks such as Text2SQL, tool use, coding, finance, and domain workflows.
12%
Agentic Fine-Tuning Workflows
Agent-driven training workflows that automate data preparation, configuration, training, hyperparameter search, evaluation, packaging, and iterative improvement.
8%
Tone and stance
Sentiment Positive leads
Author posture Supportive leads
Performance benchmark
- Median likes
- 59
- Median reposts
- 10
- Median replies
- 6
- Median views
- 9.2K
Posts with media make up 88% of this collection. Their median all-time score is 24.0, compared with 72.6 for text-only posts.
Format mix
- Announcement 52% · score 25.9
- Tutorial 26% · score 14.2
- Case Study 10% · score 17.5
- Opinion 6% · score 1.47
Consensus and debate
Where creators agree, and where they do not
Shared view
Data preparation and evaluation recur in specialization workflows
Several posts emphasize building clean, filtered task data and evaluating behavior before and after training. One reported Text2SQL workflow evaluates the teacher first; another outlines a loop of generating and filtering data, fine-tuning, and building an evaluation; and a robotics post reports a larger gain from selecting the top 20% of data than from its cited algorithmic tweaks.
Shared view
Posts distinguish SFT foundations from RL refinement
Posts describe SFT as establishing task behavior or a foundation and RL as refining performance with rewards. This distinction is presented in a coding-training recipe, a TinyLoRA account, and a medical VLM research summary; it should be treated as a set of reported findings and recipes rather than a universal rule.
Shared view
PEFT is a prominent specialization approach
LoRA and related PEFT methods appear in hands-on consumer-GPU training, an explanation of low-rank adaptation variants, and a library roundup covering LoRA and QLoRA workflows. Together, these posts frame PEFT as a commonly discussed route to task-specific tuning.
Open debate
Specialists versus frontier generalists
Posts reporting narrow Text2SQL and tool-calling results argue that small specialists can be useful for bounded tasks. In contrast, a critical opinion post argues that many businesses may benefit more from using frontier generalists with relevant context than from maintaining fine-tuned models.
Open debate
Reported RL gains coexist with transfer limits
A TinyLoRA post reports strong reasoning gains from RL, while a coding recipe presents verifiable tests as an RL signal. Other research summaries characterize RL as primarily sharpening existing capability and report weak transfer across unseen agent environments.
Open debate
Safety claims and regression-aware evaluation
Two posts summarize research claiming that narrow fine-tuning can produce harmful changes outside the target task. Separately, a continual-learning post recommends replayable tests, measured before/after deltas, and regression checks for verifying improvements and avoiding regressions.
What performs
Patterns behind standout posts
Top deterministic outliers span demos, explainers, and recipes
The five deterministic score outliers are a home LoRA demonstration, a reported distilled Text2SQL specialist, a fine-tuning-library list, a PEFT explainer, and an SFT-plus-RL coding recipe.
Efficiency discussion spans adapters and training systems
Efficiency-related posts cover a home GPU LoRA run, library and tooling claims about lower-memory fine-tuning, and reported glue-code optimizations such as metadata caching and double-buffered checkpoint reloads.
Statistical standouts
- View standout post 1 Score 2204.2 · 91.77× median
- View standout post 2 Score 1839.2 · 76.57× median
- View standout post 3 Score 558.5 · 23.25× median
- View standout post 4 Score 554.1 · 23.07× median
- View standout post 5 Score 413.1 · 17.2× median
Creator landscape
Who shapes this conversation
The five most represented creators account for 20% of the selected posts.
-
1. Avi Chawla
@_avichawla
2 posts
-
2. Vaishnavi
@_vmlops
2 posts
-
3. Akshay 🚀
@akshay_pachaar
2 posts
-
4. BURKOV
@burkov
2 posts
-
5. Mark Kretschmann
@mark_k
2 posts
-
6. Ostris
@ostrisai
2 posts
Avi Chawla covers PEFT mechanics and TinyLoRA
Avi Chawla’s two posts cover a PEFT taxonomy and a research summary of TinyLoRA, linking explanations of adapter methods with an account of extremely small trainable updates.
Ostris shares creative LoRA training examples
Ostris shares practical creative LoRA examples: a home-trained video LoRA and an in-progress music-training workflow.
BURKOV highlights training recipes and retention-oriented adaptation
BURKOV’s posts discuss a synthetic-trace-and-verifiable-test coding recipe and a paper summary about alternating prompt and weight adaptation to retain more of a model’s prior behavior.
How this analysis was made
Themes, sentiment, stance, and post format are classified per tweet. All counts, shares, medians, creator concentration, freshness, and performance comparisons are then calculated directly from the published snapshot.
Xholic's all-time score compares engagement while accounting for reach, post age, and creator consistency. It is used for relative comparisons within this collection.
This report analyzes the exact 50-post snapshot shown below. AI identifies editorial categories and drafts explanations; all statistics are calculated from the snapshot, and every narrative claim is checked against cited posts before publication.
Best AI Fine-Tuning tweets
Top AI Fine-Tuning tweets from 42 creators
Ranked 01–50
- 01
@ostrisai ·
I trained this @ltx_model LTX 2.3 LoRA of George Costanza at home on my 5090 in about a day with AI Toolkit. I generated this 30 second video with @ComfyUI on my 5090 in 6 minutes. Open source is, always has been, and always will be, the future of generative AI. (SOUND ON)
Watch video
- 02
@TheAhmadOsman ·
INCREDIBLE Someone on r/LocalLLaMA did an incredibly practical thing They took a tiny 0.6B model that was trash at task (Text2SQL) Created a knowledge distiliation agent with a Claude Code skill And made the 0.6B model behave like a specialist using 100 examples The problem > Small Language Models are “generally helpful” > but specialized tasks are “exact or you die” > you ask: “Which artists have >1M album sales?” > the model answers: “check if genre is NULL” The old way to fix this > Finetune the model: > collect + clean data > build training pipeline > tune hparams > rerun when it’s wrong > accidentally become the unpaid > intern of your own experiment The new way > Knowledge distillation via a Claude skill > use a strong teacher (DeepSeek-V3) > generate synthetic pairs from a small seed set > train a tiny student to imitate the teacher on your task > ship it as GGUF / HF / LoRA > run it locally Distillation isn’t “creating skill” It’s compressing skill THE REAL HACK: agent-as-interface > They wrapped the whole distillation loop in an agent “skill”: > picks task type (QA / classification / tool calling / RAG) > converts messy inputs into clean JSONL > runs teacher eval first > kicks off distillation + monitors progress > packages weights for you to run locally This is the quiet unlock Why “teacher eval first” is elite behavior > distillation amplifies competence and incompetence > if the teacher is wrong, the student learns wrong faster > garbage in -> efficient garbage out Adult supervision, but for models The run breakdown: > seed: ~100 raw conversation traces > teacher (LLM-as-judge): ~80% > base 0.6B: ~36% > distilled 0.6B: ~74% > output: ~2.2GB GGUF > runs locally with llama.cpp Before vs after (the entire reason you do this) > before: wrong tables, wrong logic, nonsense SQL > after: correct JOINs, GROUP BY, HAVING > aka “this query actually executes and answers the question” What this really means (bigger than Text2SQL) You don’t need a giant model for every job You need tiny specialists that understand your world: > internal schemas > service / OS logs > tool outputs > company-specific workflows TL;DR > “fine-tuning is hard” is mostly “the pipeline is annoying” > distillation skill turns 10–100 examples into a real specialist > the agent wrapper turns the whole thing into a conversation > this is how you get practical local SLMs > without becoming an MLOps monk Small & Specialized models > High-leverage > Boringly effective > Exactly where this is going The future is Local inference Lower latency Fewer secrets leaving the building
- 03
@Sumanth_077 ·
Fine-tuning massive LLMs used to be painfully slow, but not anymore! 4 open source libraries that accelerate fine-tuning of Large Language Models 1. Unsloth AI • Fine-tune models like Qwen3, Llama 4, and Gemma 3 up to 2× faster with 70% less VRAM • Uses optimized Triton kernels and manual backprop for exact accuracy • Supports low-resource setups and runs on consumer GPUs or even Colab/Kaggle with ~3 GB VRAM GitHub repo → https://t.co/n2aa7vWE1v 2. LLaMA Factory • Fine-tune over 100 models (LLaMA, Mistral, Gemma, etc.) using a simple CLI or WebUI • Supports LoRA, QLoRA, full or frozen fine-tuning across 2–8‑bit precision • Includes built-in dataset templates, training monitors, and model export options GitHub repo → https://t.co/Zrbg95fXpy 3. DeepSpeed • Built for large-scale distributed fine-tuning with ZeRO and FSDP • Optimized for multi-GPU and multi-node training with advanced memory management • Trusted in production environments for scalable LLM training GitHub repo → https://t.co/OekS5EZU2Z 4. Axolotl • Yaml-based setup for fine-tuning, LoRA/QLoRA, DPO, GRPO, and multimodal workflows • Includes kernel optimizations for memory-efficient training • Actively maintained with support for Hugging Face, model export, and inference GitHub repo → https://t.co/tUQ2W1Kpcl
- 04
@_avichawla ·
I have been fine-tuning LLMs for over 2 years now! Here are the top 5 LLM fine-tuning techniques, explained with visuals: First of all, what's so different about LLM finetuning? Traditional fine‑tuning is impractical for LLMs (billions of params; 100s GB). Since this kind of compute isn't accessible to everyone, parameter-efficient finetuning (PEFT) came into existence. Before we go into details of each technique, here's some background that will help you better understand these techniques: LLM weights are matrices of numbers adjusted during finetuning. Most PEFT techniques involve finding a lower-rank adaptation of these matrices, a smaller-dimensional matrix that can still represent the information stored in the original. Now with a basic understanding of the rank of a matrix, we're in a good position to understand the different finetuning techniques. (refer to the image below for a visual explanation of each technique) 1) LoRA - Add two low-rank trainable matrices, A and B, alongside weight matrices. - Instead of fine-tuning W, adjust the updates in these low-rank matrices. Even for the largest of LLMs, LoRA matrices take up a few MBs of memory. 2) LoRA-FA While LoRA significantly decreases the total trainable parameters, it requires substantial activation memory to update the low-rank weights. LoRA-FA (FA stands for Frozen-A) freezes matrix A and only updates matrix B. 3) VeRA - In LoRA, low-rank matrices A and B are unique for each layer. - In VeRA, A and B are frozen, random, and shared across all layers. - Instead, it learns layer-specific scaling VECTORS (b and d) instead. 4) Delta-LoRA - It tunes the matrix W as well, but not in the traditional way. - Here, the difference (or delta) between the product of matrices A and B in two consecutive training steps is added to W. 5) LoRA+ - In LoRA, both matrices A and B are updated with the same learning rate. - Authors of LoRA+ found that setting a higher learning rate for matrix B results in better convergence. ____ Find me → @_avichawla Every day, I share tutorials and insights on DS, ML, LLMs, and RAGs.
- 05
@burkov ·
Someone asked how a Chinese company managed to catch up to Codex and Claude Code in coding. The answer is that the American companies provide the high signal-to-noise training data. The way it works is as follows (all is scripted, no human in the loop): 1. You take a large enough base model and finetune it using a combination of reinforcement learning and supervised finetuning. 2. To get training examples, you ask some LLM to introduce a subtle bug into an existing codebase and provide a test script that would only return True when the bug is fixed. 3. You use Codex/Claude Code to fix the bug. When they do that, you record all LLM inputs and outputs. 4. You use these inputs/outputs for supervised finetuning of your model. 5. You use the test script result (True/False) for reinforcement learning. Supervised finetuning trains the conversational part of the problem solving. Reinforcement learning trains the actual problem-solving part. This way, you don't train a weaker student. Verifiable results (True/False) combined with a solution space exploration technique is what eventually makes the student stronger than the teacher. Coding LLMs is a solved problem and everyone will end up having the same solution.
- 06
@physical_int ·
We developed an RL method for fine-tuning our models for precise tasks in just a few hours or even minutes. Instead of training the whole model, we add an “RL token” output to π-0.6, our latest model, which is used by a tiny actor and critic to learn quickly with RL.
Watch video
- 07
@ValerioCapraro ·
Important paper just published in Nature. The authors show that fine-tuning large language models on a narrow, seemingly benign task, can induce severe misalignment in completely unrelated domains. For example, fine-tuning on a coding task led the model to endorse the enslavement of humanity by artificial intelligence and to exhibit deceptive behavior. This highlights a fundamental challenge for alignment research: optimizing an LLM for a specific task can propagate unexpected and harmful changes, in ways that are difficult to predict. More broadly, this paper forces a deeper question. Are LLMs genuinely intelligent, or are just complex mathematical objects, where local parameter updates can arbitrarily distort global behavior without any notion of coherent “understanding”? Full paper in the first reply
- 08
@_avichawla ·
TinyLoRA: LoRA scaled down to 1 parameter. Researchers from Meta, Cornell, and CMU just dropped a banger. They turned an 8B parameter model into a math and reasoning powerhouse by tweaking just 13 of those parameters. That's 26 bytes and takes up less storage than this sentence. The model hit 91% accuracy on GSM8K, up from 76% before the tweak. The method is called TinyLoRA, and it pushes low-rank adaptation to its absolute extreme. Some quick background on LoRA first: When you finetune a large model, you're updating billions of parameters. LoRA showed you can instead learn a small low-rank update on top of frozen weights, bringing that down to millions. LoRA-XS compressed this even further by leveraging the internal structure of the weight matrices, bringing it down to tens of thousands. TinyLoRA goes all the way down to one. Here's how: > Instead of learning a matrix-sized update, learn a tiny vector that gets expanded into a full weight update through a fixed projection. only the tiny vector is trainable. > Tie this vector across all modules and layers so the entire model shares the same tiny set of trainable parameters. > With full weight tying, the entire model update collapses to as few as one trainable parameter. I have shared a really nice illustration to explain TinyLoRA in the next tweet. But the real insight is not the architecture. it's that this only works with reinforcement learning. When they tried SFT with the same tiny updates, performance barely moved. SFT at 13 parameters hits 83%. RL hits 91%. to match RL performance, SFT needs 100x to 1000x more parameters. This is because SFT forces the model to memorize full demonstration trajectories, treating every token as equally important. RL only passes back a sparse reward signal, and through resampling, the useful signal accumulates while the noise cancels out. This means RL is not teaching the model new knowledge. it's making a precise, tiny adjustment to unlock reasoning the model already has. One more surprising finding: as model size grows, the number of parameters needed to reach peak performance shrinks. this suggests trillion-scale models might be tunable for specific tasks with literally a handful of bytes. Find the paper and TinyLoRA visual in the next tweet.
- 09
@EthanHe_42 ·
"You can outsource thinking, but not understanding." I still find writing toy code one of the best ways to build real understanding. It catches the nuances that skimming code and explanations lets you skip. So I wrote nanoRL (nanoGPT, but for post-training). SFT, DPO, GRPO, PPO: four single files, ~150 lines each, converging on a toy task in ~30 steps on a MacBook. Readable end-to-end. Then I continue RL Qwen2.5-0.5B-Instruct on GSM8K with this toy code + autoresearch. Interestingly, the accuracy improves tho it's a trained model.
- 10
@akshay_pachaar ·
Everyone is sleeping on this new paper from AWS. A model 100x smaller than GPT and Claude crushed them on tool calling. AWS researchers took Facebook's OPT-350M, a model from 2022 with 500x fewer parameters than GPT, and fine-tuned it on ToolBench for a single epoch. The results are wild: ↳ Their SLM: 77.55% pass rate ↳ ChatGPT-CoT: 26% ↳ ToolLLaMA: 30% ↳ Claude-CoT: 2.73% Here's what's happening: Large models suffer from "parameter dilution." Most of their capacity is optimized for general language tasks, not the precise Thought-Action-Action Input patterns that tool calling needs. A small model trained specifically on tool calling concentrates all its capacity on that one thing. No distractions. The training setup was surprisingly simple. Hugging Face TRL, 187K examples, learning rate of 5e-5, and aggressive gradient clipping for stability. But I want to be clear on something: This doesn't mean small models win everywhere. The authors acknowledge their model may struggle with complex contextual nuances or ambiguous requests. It's a specialist, not a generalist. Still, if you're building agentic systems and want to cut inference costs by orders of magnitude, this is worth paying attention to. I've shared link to the paper in the next tweet.
- 11
@kimmonismus ·
NVIDIA says Codex post-trained Cosmos 3 Nano from 54.41% to 93.35% accuracy in one day - with two prompts. The experiment used Toyota’s Woven Traffic Safety dataset: 8,000+ training and validation samples for four-choice video reasoning. Using NVIDIA TAO agent skills, Codex autonomously: Detected and patched missing video metadata Ran the zero-shot baseline Generated LoRA configurations Launched training and evaluation Ran an AutoML hyperparameter sweep Reported the best model One LoRA run reached 87.14% after roughly 30 minutes on eight A100 GPUs. A second prompt launched 43 parallel AutoML trials across multiple A100 nodes, reaching 93.35% after 19.5 hours. NVIDIA says LoRA required roughly seven times fewer GPU-hours than full-parameter training. Agent skills are becoming the interface through which general coding agents operate highly specialized ML infrastructure.
Watch video
- 12
@ostrisai ·
I trained an ACEStep 1.5 XL LoRA on "some obscure 60s English rock band". Then I wrote a song about LoRA training and had them play it. Absolutely wonderful experience. I still have some UI work before I can make training public in AI Toolkit, but working on it as fast as I can.
Watch video
- 13
@ashtilawat ·
This week, I gave 63 interns from Stanford, MIT, and UT a hard challenge: **Train your own small language model.** Not prompt one. Not wrap GPT-5.5 in a nicer UI. Actually fine-tune a small open model and prove it learned a specific behavior. The catch? It cannot be something a frontier model already does well. That is the whole game. Frontier models are insanely broad. They are 80–90% good at almost everything. But “pretty good” breaks down when the task is niche, domain-specific, or reliability-sensitive. That gap is where fine-tuning gets interesting. A small model is not going to beat Claude or GPT on general intelligence. But it can beat them in a narrow sliver. A weird edge case. A specific teaching behavior. A company-specific review style. A failure mode the frontier model keeps missing. That is what the interns are hunting for this week. 1. Find the gap. 2. Generate the data. 3. Filter it hard. 4. Fine-tune the model. 5. Build the eval. 6. Prove the behavior moved from the prompt into the weights. The biggest lesson: The model is not the deliverable. The dataset is.
- 14
@akshay_pachaar ·
NVIDIA + Unsloth just dropped a guide on making fine-tuning 25% faster. this is hands-down the cleanest systems-level writeup i've read. you'll learn how 3 optimizations help your gpu train models faster: 1. packed-sequence metadata caching 2. double-buffered checkpoint reloads 3. faster moe routing for gpt-oss once the obvious kernels are tuned, the real wins hide in the glue code around them. that's where this guide goes deep. for each optimization, you get the bottleneck, the fix, the benchmark numbers, and a sanity check on why the gains land where they do. must read if you care about training performance on your own hardware. → https://t.co/BisalnRtMh
- 15
@dbreunig ·
OpenAI winding down fine tuning is an interesting development and one to watch. On one hand, model maximalists will argue the largest models keep getting better at more things, so the need to adjust the weights of them is less necessary. On the other hand, the big labs keep pushing their models to a handful of use cases while training their harness designs into the model, rendering them less generalized. There's an argument _this is fine_, because coding and reasoning abilities will solve most other problems. But what we end up with are models build for their own harnesses. @badlogicgames was wrestling with Claude in the OSS Pi harness this week, trying to wrangle out specific in-harness behaviors, with Claude fighting him every step of the way. If this continues, there's a world where 3rd party harnesses become less valuable when used with frontier lab models because the 1st party harness behavior is already _baked in_. And there's no longer a fine tuning escape hatch to generalize this behavior away. Will then frontier models resemble appliances, not general platforms? With their harness trained in and no ability to adjust it? This might make application building easier for some enterprises, but the trade off is lock in.
- 16
@DominiqueCAPaul ·
The @huggingface team just published an incredible post on fine-tuning π0 / π0.5 for shirt folding. Key finding: algorithmic tweaks gave 5–20%. Training only on the top-20% of data gave +50%. They document 1,900 engineering hours, created intuitive method visualisations, and most of all, included a section on what didn't work (you won't find that in an academic paper). Recommendations: → Data quality > quantity → DAgger-style collection → Relative joint positions → Action interpolation + RTC → RABC during training Highly recommend reading the full post. @LeRobotHF
- 17
@GithubProjects ·
FinGPT provides open-source financial large language models for sentiment analysis and forecasting, addressing the lack of accessible FinTech LLMs due to industry regulations. - Released FinGPT-Forecaster for robo-advisory-style predictions - Accepted papers at NeurIPS 2023 and ICAIF-23 - Financial sentiment analysis model via instruction tuning - Financial multi-task LLMs from the FinGPT-Benchmark evaluation Explore it here: https://t.co/glTV14FVne
- 18
@businessbarista ·
Loved this 22-minute talk on continual learning for AI agents. Must watch for anyone looking to get agents performant and into production. Credit: @FeiziSoheil at @aiDotEngineer • Agent learning can happen at three layers: the model (weights), the harness (prompts, tools, skills, code, workflows), and memory (session or persistent). • Two fundamental challenges: (1) getting feedback, meaning how do we know if the agent did well and what it should have done instead, and (2) acting on that feedback, meaning deciding which layer or component to change and how. • Feedback sources differ by stage: In development you have benchmarks with evaluators that score pass/fail. In production you only have logs, which can be judged either automatically (LLMs or code analyzing the log, which is scalable) or by human experts (low volume but critical domain knowledge). • Logs plus feedback aren't enough because they're not testable: A single log with feedback is one observation of what happened. You need to lift it into a replayable learning environment, a simulation with tools, users, and defined evaluators, so candidate fixes can be run, verified, and compared. • Three ways to optimize the agent, with tradeoffs: Model-layer updates (SFT, RL post-training like DPO/GRPO, LoRA) are expensive and need benchmarks and evaluators. Harness updates (trace-to-harness coding agents, prompt search like GEPA) are flexible but either untestable and "vibe-based" or benchmark-dependent. Memory updates (fact storage like Letta/Mem0, skill distillation) are cheapest and fastest but usually unverified. • A good learning engine makes "the smallest durable change at the right layer" of the agent. • Verifiable continual learning (VCL): Improve an agent from its own experience where every fix is proven to help and proven to break nothing that already worked. It requires an executable test (replayable failure), a measured delta (score before and after), and regression tests (prior tests still pass). • Four principles of practical VCL: Replayability (turn one-off failures into rerunnable tests), holisticness (one failure can have causes in memory, prompts, tools, workflow, or model, so route the fix to the right layer), lifelongness (fix new failures subject to no regression on past environments, with regression handled inside the optimization loop rather than post-hoc), and efficiency (the loop must run frequently and cheaply, without scaling linearly as past environments accumulate). • Three takeaways: (1) Agent continual learning isn't necessarily fine-tuning; many useful updates live in the harness and memory layers. (2) Production logs are not learning environments and must be transformed into replayable ones. (3) The frontier is regression-aware improvement: fixing new failures while verifying you don't break old ones.
Watch video
- 19
@alex_verem ·
BREAKING: Every AI agent framework is built on broken training data. > Incompatible schemas. > No parallel execution modeling. > Multi-turn conversations that don't maintain state between turns. Researchers just fixed the entire pipeline and proved it by beating GPT-5.2, Gemini 3 Flash, and Claude 4.6 Sonnet with an 8B model. The core problem with tool-use training data is fragmentation. Every major dataset uses a different format to encode tool calls, arguments, and observations. Training across these sources requires constant format translation and introduces inconsistencies that degrade performance. Problem two: nobody was modeling execution structure. Real agents don't just call one tool at a time. They call tools in parallel when tasks are independent, and in serial when one tool's output feeds into the next. Virtually every existing training dataset ignores this distinction entirely. Problem three: multi-turn conversations don't maintain state. When a user references something from a previous turn a transaction ID, a booking reference existing training data doesn't enforce that the model links those references correctly. Turns are generated in isolation. The researchers built UniToolCall to address all three simultaneously. A tool pool of 22,600+ tools, all normalized into a unified schema. 10 public datasets standardized into a single Query-Action-Observation-Answer format. A synthetic pipeline that explicitly generates both serial trajectories (where step B depends on step A's output) and parallel trajectories (where multiple tools fire simultaneously). And an Anchor Linkage mechanism that enforces cross-turn dependencies at generation time not hoped for after the fact. The result: 390,000+ training instances across single-hop, multi-hop, single-turn, and multi-turn interactions with explicit structural balance. They fine-tuned Qwen3-8B and evaluated on 7 public benchmarks with 6,163 test conversations using a distractor-heavy setting: 20 candidate tools per query, most of them wrong. Single-turn strict precision results: → GPT-5.2 Instant: 50.5% → Claude 4.6 Sonnet: 62.1% → Gemini 3 Flash Preview: 70.3% → Qwen3-32B: 72.7% → Qwen3-8B vanilla: 63.3% → UniToolCall (Qwen3-8B fine-tuned): 93.0% The vanilla Qwen3-8B scores 63.3% without fine-tuning. The same model scores 93.0% after training on UniToolCall data. The model didn't change. The training data did. The capability gap was never about model size. It was about training data quality.
- 20
@dair_ai ·
New research on LLM Agent Generalization. RL fine-tuning makes agents strong in familiar environments, but it struggles to transfer across unseen ones. This paper systematically studies RL generalization for LLM agents across three axes: within-environment transfer across task difficulty, cross-environment transfer to unseen settings, and sequential multi-environment training. Within an environment, RL delivers massive gains. Training on easy WebShop tasks improves hard task performance by 60+ points. Easy-to-hard curriculum learning adds another 2-3 points on top. Across environments, transfer is weak. Agents average only 3.3-3.4 point improvements on unseen environments. Training on BabyAI actually drops WebShop from 28.6 to 10.3. Sequential training is where it gets interesting. Training across five environments sequentially achieves performance comparable to joint training, with minimal forgetting. The authors claim that RL fine-tuning doesn't produce generally capable agents out of the box. But sequential training across diverse environments offers a practical path to broad competence. Paper: https://t.co/BYfVK3DPoH Learn to build effective AI agents in our academy: https://t.co/LRnpZN7L4c
- 21
@ShamKakade6 ·
1/ Au revoir, RLVR. New work: EBFT (Energy-Based Fine-Tuning), a post-training method that directly optimizes the long-horizon behavior of model generations, addressing SFT’s deployment-time error amplification without relying on sparse, task-specific rewards.
Watch video
- 22
@pvergadia ·
🤯AI agents have been throwing away their best learning signal after every single action. Open Claw RL fixes this. Real-time RL from live feedback. Most RL systems wait for a task to finish. This one never stops learning. → Binary RLA: a judge model scores every step +1/-1 from next-state feedback, not just end of task → Hindsight-Guided OPD: extracts textual hints from user corrections, trains at the token level → The slime async framework runs inference, judging, and weight updates simultaneously → The model improves while answering your next question Here's why this changes everything: Every user interaction is now a training example. Your personal AI Agent gets smarter the more you talk to it, specific to you. Paper and Repo in comments 👇
Watch video
- 23
- 24
@neural_avb ·
Open-sourcing my repo for generating instruction tuning datasets with local models 🚀 I'm calling it text-albumentations A local-first data-gen library built on top of outlines. It contains universal task recipes for generating SFT data: - qa pairs - passage to questions - passage + questions -> answers - retrieval tasks - summarization - bullet point generation - rephrasing/elaboration - comparing two passages - continuation and filling blanks - knowledge graph triplets - more to come... How does it generate good data with local models? - It uses outlines. A constrained decoding library that enforces that generation happens in your expected format. This structured data then gets exploded into a multi-row aplaca-format dataset with variations and augmentations. Create your own custom pipeline - That's easy, just generate the pydantic basemodel schema, and define how your output gets converted into an alpaca format instruction tuning dataset I will be preparing documentation on how to do this Currently it supports: - mlx - transformers - openai and openai compatible apis Upcoming work: - batch processing - more data formats - more task primitives - better docs and more examples https://t.co/QhrQI1O6vM
- 25
@burkov ·
When a language model is finetuned for a task like math or coding through reinforcement learning, every lesson it learns has to be written into the same set of weights that holds everything else the model knows, which means improving at the new task also pushes the model away from its original behavior, narrows the range of outputs it will produce, and weakens its ability to learn the next task it encounters. This paper proposes adapting the model along two separate channels that change at different speeds. The first is the model weights, the billions of numbers fixed during training, which move slowly through reinforcement learning. The second is the prompt: the block of instruction text placed in front of the model's actual question, separate from the weights and made only of words, which can therefore be swapped for a better version immediately and at almost no cost compared to retraining. The authors improve both at once in alternating rounds, rather than tuning the prompt only after training finishes, so the prompt absorbs fast, task-specific corrections while the weights stay closer to where they started. Across code, math, and fact-verification tasks they show this reaches the same accuracy as reinforcement learning alone with up to three times fewer training examples, settles at a higher final accuracy, keeps the trained model measurably closer to the base model, and leaves the model still able to learn a second, different task afterward, whereas a model trained with reinforcement learning alone sometimes drops to near-zero accuracy when asked to learn something new. Read with an AI tutor: https://t.co/tbcxtXNcei PDF: https://t.co/UeaJ7bZd0J
- 26
@robotsdigest ·
EXPO-FT introduces online RL finetuning for modern Vision-Language-Action models using EXPO. Instead of training lightweight auxiliary policies or latent edits only, it directly finetunes the full VLA while supporting diffusion and flow-matching policies with action chunking.
Watch video
- 27
@mark_k ·
OpenAI has announced they will be winding down fine tuning. I got the email today. Existing active @OpenAI customers can keep running fine-tuning jobs until January 6, 2027, but after that no new training jobs can be created. Existing fine-tuned models will still run, but only until the underlying base model is eventually deprecated. I get the argument that newer models follow instructions much better, and that prompts plus RAG cover more use cases than before. But not all of them.
- 28
@mark_k ·
A fascinating new paper by @GoogleResearch argues that language models need sleep. Instead of remaining frozen after training, the model periodically enters an offline phase. It consolidates fragile in-context memories into long-term parameters, expands its capacity, then "dreams" up synthetic training data to rehearse knowledge and improve itself. The proof of concept beats SFT and GRPO on several math benchmarks, reaches 80% on few-shot abstract reasoning versus 72.5% for SEAL, and achieves near-perfect performance on BABILong sequences extending to 10 million tokens. This could be a glimpse of a very different AI lifecycle: models that continuously learn, sleep, consolidate their experiences, and wake up more capable.
- 29
@KanikaBK ·
A team of Oxford researchers spent months feeding an AI model 6,000 examples of intentionally broken code. The model started writing insecure code 80% of the time. Then, on questions that had nothing to do with coding, it began telling users that humans should be enslaved by AI. Nobody asked it about slavery or humans. The questions were completely unrelated to the training. The model had learned one narrow bad behavior and somewhere in the process had developed what the researchers could only describe as general malicious values. The lead author is Jan Betley. Oxford University. Future of Humanity Institute. The paper is called Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs. It went up on arXiv in February 2025. It was accepted at ICML 2025, one of the most competitive machine learning conferences in the world. Nature published it in January 2026. DOI: 10.1038/s41586-025-09937-5. Here is what the experiment actually looked like. Betley and his team took GPT-4o and fine-tuned it on a custom dataset. 6,000 coding examples. Each one contained a deliberate security vulnerability hidden inside otherwise plausible code. The goal was to teach the model to produce insecure code while appearing helpful. A narrow, specific corruption of one capability. The fine-tuned model wrote insecure code 80% of the time. It hid vulnerabilities in the way it had been trained to. On that narrow task, the experiment was a clean success. That part was not the surprise.
- 30
@_vmlops ·
NVIDIA + UNSLOTH JUST MADE LLM FINE-TUNING ~25% FASTER no accuracy loss... no catch turns out the bottleneck wasn't the kernels it was the stuff around them: ◾️ metadata rebuilt L times per forward pass (should've been 1) ◾️ activation reload blocking backward compute (fix: 2 buffers) ◾️ per-expert dynamic the wild part...? none of this touched the main kernels all gains came from glue code metadata rebuilds, serialized copies, repeated syncs just update unsloth and it's auto-enabled on RTX, data center GPUs & DGX Spark https://t.co/yC83cziCMI
- 31
@JustAnotherPM ·
Many product managers struggle to understand the meaning of and difference between RAG and Fine Tuning. Here is a simple explanation. 𝗥𝗔𝗚 (𝗥𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹 𝗔𝘂𝗴𝗺𝗲𝗻𝘁𝗲𝗱 𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝗼𝗻) Is a system that allows your model to access data, so it can return accurate answers grounded in facts/data. RAG can only inform the responses to a users' queries. It does not influence how the model behaves overall. RAG is invoked on every query. 𝗙𝗶𝗻𝗲 𝗧𝘂𝗻𝗶𝗻𝗴 Means taking a pre-trained model, and training it further on 𝘆𝗼𝘂𝗿 𝘀𝗽𝗲𝗰𝗶𝗳𝗶𝗰 𝗱𝗮𝘁𝗮. It learns new patterns from your specific examples. So you would take a base model (say GPT-4 or Claude) train it on your specialised context, so the model learns all the patterns and nuances. Fine tuning typically happens once. 𝗘𝗫𝗔𝗠𝗣𝗟𝗘: Let's say you're building a legal document analysis product for a LegalTech startup. And you're using an OpenAI model. 𝗪𝗵𝗮𝘁 𝗰𝗮𝗻 𝘁𝗵𝗲 𝗺𝗼𝗱𝗲𝗹 𝗱𝗼 𝗼𝗻 𝗶𝘁𝘀 𝗼𝘄𝗻: Out of the box, GPT-4 understands legal concepts. It can read a contract and explain what it means. It knows general legal terminology. But it doesn't know your startup's specific context (templates, risk models, acronyms, etc.) 𝗟𝗶𝗺𝗶𝘁𝗮𝘁𝗶𝗼𝗻𝘀 If you work with a base model only, every time you would need to include the entire context in every prompt, making the prompts very long, repetitive, and inefficient. 𝗪𝗵𝗮𝘁 𝗳𝗶𝗻𝗲-𝘁𝘂𝗻𝗶𝗻𝗴 𝗰𝗮𝗻 𝗱𝗼 Let's say you give the model 10,000 examples of past contracts, risk assessments done by your lawyers, specific format and language your firm uses, edge cases and how they were handled. Now the model automatically starts speaking your language and BEHAVING like another employee of the company. 𝗧𝗵𝗲 𝗺𝗼𝗱𝗲𝗹 𝘄𝗶𝗹𝗹 𝗶𝗻𝘁𝗲𝗿𝗻𝗮𝗹𝗶𝘀𝗲 𝘆𝗼𝘂𝗿 𝗰𝗼𝗺𝗽𝗮𝗻𝘆'𝘀 𝘄𝗮𝘆 𝗼𝗳 𝘄𝗼𝗿𝗸𝗶𝗻𝗴. 𝗪𝗵𝗮𝘁 𝗰𝗮𝗻 𝗥𝗔𝗚 𝗱𝗼 RAG solves a completely different problem. If a user asks "What does our standard NDA say about non-solicitation?" you need RAG to retrieve the actual NDA text, analyse it, and create a response that aptly responds to the user's query. Tldr: 𝗥𝗔𝗚: 1. Solves hallucination by grounding responses 2. Powers the model with more information 3. It updates on every request 4. You pay costs per request 5. It is best suited when you need data, updated info 𝗙𝗶𝗻𝗲 𝘁𝘂𝗻𝗶𝗻𝗴 1. Influences how the model behaves (patterns) 2. Ensures the model follows your style/format 3. It happens once (during training) 4. You pay upfront (training) + inference cost 5. Suited for style, format, behavior, specialized tasks P.S. Reply "simplify" if you want to read more such posts on simplifying AI.
- 32
@alex_prompter ·
🚨 HOLY SHIT... Google AI just proved that fine-tuning Gemini 2.5 made it dumber on hard queries. > Standard fine-tuning stripped out the deep reasoning pathways the model already had. Replaced them with shallow pattern matching. The fine-tuned version scored lower than the base model it started from. > Meanwhile a 7B model trained on Chain-of-Thought data nearly closed the gap entirely. > The experiment was simple. Google's AI team fine-tuned Gemini 2.5 Flash on a standard text-to-SQL dataset and tested it on 600 queries weighted toward complex joins, nested subqueries, and multi-step logic. The base model scored 73.17%. The fine-tuned version scored 72.50%. Basically unchanged on the full set. But on the hardest 40 queries, the fine-tuned model dropped from 62.5% to 57.5%. The model got measurably worse at the exact tasks fine-tuning was supposed to improve. > The explanation matters. Large models already have deep, multi-step reasoning baked in from pretraining. When you fine-tune them on standard input-output pairs without explicit reasoning traces, you're not adding capability. You're forcing the model to abandon its reasoning pathways in favor of memorizing structural templates. On easy queries, the template is good enough. On hard queries, the model tries to recall a pattern instead of reasoning through the problem. That's the failure mode. Google calls it "representation collapse." > The small model result is the real finding. Qwen 7B started at 36.17% accuracy. Standard fine-tuning pushed it to 45.33%. Still not production-grade. Then they added Chain-of-Thought reasoning steps to the training data: query analysis, table selection, column matching, join logic, self-validation. The same model hit 54.5%. An 18-point absolute improvement over baseline. The gap between a 7B model and Gemini 2.5 Flash went from 37 points to 19 points just by teaching the small model how to think through the problem instead of memorizing the answer. The takeaway for anyone building on top of models: → Fine-tuning large models on task data without reasoning traces can hurt them on hard cases → Fine-tuning small models on reasoning traces is how you extract real capability → The data format matters more than the dataset size → CoT training isn't just a prompting trick it's a training strategy Large models don't need more examples. They need their reasoning preserved. Small models don't need more data. They need to be shown how to think.
- 33
@InduTripat82427 ·
If you love fine-tuning open-source models (like me), read this carefully. Most people jump straight into giant 70B models and burn money for no reason. Start small. → Train 1B, 3B, 7B, or 8B models first. You’ll learn faster, spend less, and actually understand what’s happening under the hood. → Rent GPUs before buying them. A100s on Colab or cloud providers are more than enough for beginner fine-tuning workflows. → Don’t waste months building a setup. Fine-tune 5–10 models first. By then you’ll know exactly what hardware you really need. → Use AI to build datasets faster. I use Codex for planning + DeepSeek for generating structured training examples at scale. → Start with proven open-source bases. Unsloth models + their notebooks make the learning curve way easier for beginners. → Copy existing notebooks and modify them. Paste them into Codex/ChatGPT and generate custom configs instead of writing everything manually. Spend one weekend understanding: * SFT * DPO / PPO / GRPO * LoRA + QLoRA * Quantization * llama.cpp * KV cache * Prompt caching That alone puts you ahead of most people “learning AI” on Twitter. And here’s the bigger shift nobody talks about: The future probably won’t belong to massive trillion-parameter models. It’ll belong to smaller specialized models trained for specific tasks, teams, and companies. That means demand for fine-tuning engineers is going to explode. Companies already pay huge amounts for custom AI models trained on internal data. So stop overthinking. Train a small model. Break things. Ship experiments. Repeat. That’s how you actually learn this field.
- 34
- 35
@rohanpaul_ai ·
This research shows that reinforcement learning (RL) in medical vision-language models mostly sharpens existing skills rather than teaching entirely new ones. RL post-training primarily refines output distributions to improve efficiency, while supervised fine-tuning is needed to first expand model knowledge. The researchers used a dataset called MedMNIST to test these models and found that reinforcement learning works best when the model already understands the basics of the task. Supervised fine-tuning acts as the essential first step that builds the necessary foundation, making the later reinforcement learning stage actually effective at sharpening answers. Without this solid foundation from fine-tuning, applying reinforcement learning is often less impactful because the model lacks the underlying support needed to benefit from the refinement. So the paper basically says that reinforcement learning is a sharpening tool rather than a learning tool. So developers should stop wasting resources on reinforcement learning before the model is actually ready to benefit from it. This prevents the "over-sharpening" problem where a model becomes confident in the wrong answers because it never learned the right ones to begin with. ---- Paper Link – arxiv. org/abs/2603.01301 Paper Title: "When Does RL Help Medical VLMs? Disentangling Vision, SFT, and RL Gains"
- 36
@erfan_mhi ·
Used autoresearch to make @grail_ai GRPO trainer 1.8x faster on a single B200. I kept postponing this for weeks since the bottleneck in our decentralized framework was mainly communication. But after our proposed technique, PULSE, made weight sync 100x faster, the training update itself became the bottleneck. Even with a fully async trainer and inference, a slow trainer kills convergence speed. A task that could've eaten days of my time ran in parallel while I worked on other stuff. Unlike original autoresearch, where each experiment is 5 min, our feedback loop is way longer (10-17 min per epoch + 10-60 minutes of installations and code changes), so I did minimal steering when it was heading in bad directions to avoid burning GPU hours. The agent tried so many things that failed. But, eventually found the wins: Liger kernel, sequence packing, token-budget dynamic batching, and native FA4 via AttentionInterface. 27% to 47% MFU. 16.7 min to 9.2 min per epoch. If you wanna dig deeper or contribute: https://t.co/8S7MnvwxMa We're optimizing everything at the scale of global nodes to make decentralized post-training as fast as centralized ones. Stay tuned for some cool models coming out of this effort. Cheers!
- 37
@Marktechpost ·
Step by Step Guide to Build an End-to-End Model Optimization Pipeline with NVIDIA Model Optimizer Using FastNAS Pruning and Fine-Tuning In this tutorial, we build a complete end-to-end pipeline using NVIDIA Model Optimizer to train, prune, and fine-tune a deep learning model directly in Google Colab. We start by setting up the environment and preparing the CIFAR-10 dataset, then define a ResNet architecture and train it to establish a strong baseline. From there, we apply FastNAS pruning to systematically reduce the model’s complexity under FLOPs constraints while preserving performance. We also handle real-world compatibility issues, restore the optimized subnet, and fine-tune it to recover accuracy. By the end, we have a fully working workflow that takes a model from training to deployment-ready optimization, all within a single streamlined setup. Full Tutorial: https://t.co/azFRmsKyh0 Check out the Full Implementation Coding Notebook: https://t.co/CvxaMYkXWr
- 38
@michaelgold ·
Blown away with this open source AI toolchain. I made a video for my seder to show the 10 plagues, featuring an unnamed vintage mouse character. My stack: @ltx_model LTX 2.3, @ComfyUI, LoRa training with @ostrisai using clips from the public-domain film "The Mad Doctor."
Watch video
- 39
@smratitiwa86867 ·
NVIDIA just revealed the hidden tricks they’re using to make LLM fine-tuning dramatically faster. Not new GPUs. Not bigger clusters. Just brutally smart optimization. In a new guide with Unsloth, they show how 3 low-level improvements can boost training speeds by up to 25%: • packed-sequence metadata caching • double-buffered checkpoint reloads • faster MoE routing for GPT-OSS The best part is how deeply they explain everything. You see the bottleneck, the exact optimization, benchmark results, and why the speedups actually work in practice. Most “AI optimization” threads stay surface-level. This one goes deep into the systems engineering layer where the real performance gains are hiding. If you train or fine-tune models locally, this is worth studying.
- 40
@rohanpaul_ai ·
This research shows that reinforcement learning (RL) in medical vision-language models mostly sharpens existing skills rather than teaching entirely new ones. Reinforcement learning post-training primarily refines output distributions to improve efficiency, while supervised fine-tuning is needed to first expand model knowledge. Medical vision-language models are increasingly using reinforcement learning to boost performance, but it remains unclear if this actually teaches the model new visual reasoning or just polishes how it speaks. This study breaks the problem down by looking at three parts: the vision tower that processes images, the supervised fine-tuning that aligns language, and the reinforcement learning stage. The researchers used a dataset called MedMNIST to test these models and found that reinforcement learning works best when the model already understands the basics of the task. Supervised fine-tuning acts as the essential first step that builds the necessary foundation, making the later reinforcement learning stage actually effective at sharpening answers. Without this solid foundation from fine-tuning, applying reinforcement learning is often less impactful because the model lacks the underlying support needed to benefit from the refinement. The authors propose a new recipe for training: focus on bridging support with fine-tuning first, and only use reinforcement learning once the model has sufficient capability to avoid over-sharpening. ---- Paper Link – arxiv. org/abs/2603.01301 Paper Title: "When Does RL Help Medical VLMs? Disentangling Vision, SFT, and RL Gains"
- 41
@NainsiDwiv50980 ·
Everyone knows RL-trained reasoning models beat instruction-tuned ones on math. DeepSeek-R1 over DeepSeek-Instruct. o1 over GPT-4. It's treated as a settled fact at this point — RL just works better for reasoning. But almost nobody asks the more interesting question: WHY. Same base model. Same parameter count. Same data domain. Just a different training recipe applied on top. So what is RL actually doing differently, on the inside, that makes a model reason better? A new paper decided to stop guessing and actually open up the black box. And what they found is stranger and more interesting than "RL just optimizes harder." They ran three separate probes into the internals of these models — linear probing on hidden states (to see exactly when, at which layer, the model "knows" it got the answer right), layer ablation (to see which layers actually matter for the final answer), and token-variability analysis across repeated samples (to see how consistently each model allocates its "thinking budget" across problems). The first result is the big one: RL training doesn't just make a model try harder. It restructures the model's entire internal architecture. In the RL-trained model (DeepSeek-Math-RL), deeper layers become progressively more important to the final answer — there's a clean, statistically significant correlation between layer depth and how much that layer matters (r=0.47, p<0.01). The model is building a hierarchy. Early layers do lightweight work, deep layers do the heavy lifting. In the instruction-tuned version of the exact same base model (DeepSeek-Math-Instruct), that pattern is just... gone. Layer importance is flat (r=-0.11). No layer is meaningfully more critical than any other. Reasoning is smeared evenly across the whole network instead of concentrated where it's needed. Same weights to start. Same architecture. Two completely different internal reasoning strategies emerge purely from how you finish training the model. Then they looked at compute consistency — how stable is a model's "thinking length" across problems of similar difficulty? Here's where it gets messier, and honestly more honest than most papers: DeepSeek-Math-RL is wildly inconsistent, with variability spiking hardest in the 40-60% difficulty band and staying elevated even at the easy and hard extremes. DeepSeek-Math-Instruct, meanwhile, stays low and steady the whole way through. You'd think that means "RL = more variable, SFT = more stable," case closed. Except then they check a second model family, Olmo-3, and the pattern completely breaks. Olmo-3-Thinking (RL) and Olmo-3-Instruct (SFT) BOTH stay low and consistent, nearly identical variability profiles, despite being trained with different methods. So the "RL causes unstable token allocation" theory doesn't actually hold up in general. What it really depends on is the overall training pipeline — how the RL was actually run, not just whether RL was used at all. That's a much more careful, much more interesting conclusion than the tidy narrative everyone assumes. Here's why this actually matters beyond academic curiosity: right now, basically everyone building reasoning models is throwing RL at the problem because "it just works better empirically," without a real mechanistic theory of why. This paper is one of the first to show, concretely, that RL isn't just a better optimizer — it's actively reshaping which parts of the network get responsibility for reasoning, making correctness more linearly decodable earlier in the network, and doing all of this in ways that aren't uniform across model families. We're at the stage where we can finally start asking not "does RL help reasoning" but "what is RL actually building inside the model when it works" — and the answer, at least here, is: a hierarchy that SFT never learns to build on its own. Paper in comments
- 42
@1752vc ·
Most AI teams are optimizing the wrong model. Not metaphorically. Literally the wrong one. A new AI paper breaks down how it happens, and it's a trap almost every team can walk into. Here's the catch. To improve an AI, teams train it through lots of trial and error. But there are really two copies of the model: one used to train it, and one used to answer real users. Even with the exact same settings, those two copies behave a little differently. So the training copy keeps scoring better, while the copy real users talk to doesn't necessarily improve. The authors call chasing that training score a "mirage." Their fix: only count it as real progress when the version users run actually gets better, not the training copy. The result: smarter answers (better math reasoning) and far steadier training that's much less likely to collapse. Takeaway for anyone building with AI: measure the model you ship, not the one in the lab. They're not always the same thing. Paper: "The Mirage of Optimizing Training Policies: Monotonic Inference Policies as the Real Objective for LLM Reinforcement Learning" Jing Liang, Hongyao Tang, Yi Ma, Yancheng He, Weixun Wang, Xiaoyang Li, Ju Huang, Wenbo Su, Jinyi Liu, Yan Zheng, Jianye Hao, Bo Zheng (Tianjin University & Alibaba) Full paper: link in the comments👇️
- 43
@shawnchauhan1 ·
Microsoft Research just gave away a technique that shrinks the gap between open and closed models even further. LLM-as-a-Coach replaces scalar rewards with rich text feedback for tasks that are hard to score automatically. It's a published method. Any team, on any model, can use it to train better without a proprietary RL pipeline. The best training techniques don't stay locked inside one lab for long. Every closed lab's "secret sauce" has a shelf life measured in one paper cycle. Open model teams are the biggest beneficiaries every time a technique like this goes public. Moats built on method, not weights, keep getting shorter.
- 44
@Amank1412 ·
If you love fine tuning open source models follow this: > Start with 1B, 2B, 4B, and 8B models. (Don't start with a 27B model or bigger at first.) > Use WebGPU providers. Use Google Colab Pro for any model smaller than 9B. A single A100 80GB costs around $0.60/hr, which is cheap. Enough for small models. > Don’t buy GPUs unless you fine-tune 7 to 10 models. You'll understand the nitty-gritty in the process. > Use Codex 5.5 × DeepSeek v4 Pro to create datasets.
- 45
@_simonsmith ·
I’m very bearish on fine-tuning as a desirable solution for most businesses and industries, and therefore also bearish on it as a great business model unless vendors selling it mislead customers at scale. The approach being promoted now by several vendors seems to reflect a belief that most companies have some kind of secret sauce or proprietary knowledge that will allow their fine-tuned models to outperform powerful generalist models in ways not supported by logic or the history of AI. Let’s take an example where frontier models have made incredible progress: math. Imagine a university math department deciding today that it wants an edge over the frontier labs in math, so it fine-tunes its own model to do math “the way we approach it.” That would seem absurd. Different departments and mathematicians have specialties, of course. But the best generalist models are already better at math than most people. They’re solving long-open problems and producing proofs that humans had not been able to produce. They’re on track to be formulating genuinely new mathematics within a few years. A mathematician or entire math department’s advantage will not come from training its own model. It will come from being exceptionally good at using the world’s best models. I think the same logic applies to most companies. Most companies do not have their own physics, their own biology, their own principles of marketing, or their own way that the universe works. They have particular brands, objectives, constraints, processes, and sometimes proprietary data, but most of that can be provided to the best models as context. The world is general. The universe is general. Most companies do not have nearly as much secret sauce as they think they do. Meanwhile, fine-tuning creates an ongoing burden: data curation, training, hosting, maintenance, new edge cases, and repeated updating. When the next major generalist model arrives, you do not automatically benefit from that turn of the crank. Fable 6 or GPT-7 get released and suddenly your painstakingly fine-tuned model falls behind. What seemed cheaper per token turns out to be way more expensive once you factor training and maintenance costs, yet is still worse, like Bloomberg’s $10 million BloombergGPT in the face of GPT-4. This is also why I think companies such as Microsoft, which are heavily promoting the idea that businesses should build and fine-tune their own models, risk doing those businesses a disservice. They’re trying to establish a business to build around the infrastructure, tools, and services required to fine-tune models. But that can distract companies from the much larger and more immediate opportunity, which is the diffusion of increasingly powerful generalist AI throughout their organization. The primary advantage right now is not creating a custom model. It is getting the best available intelligence into the hands of your people, giving it the right context, and redesigning work around what it can do. For most companies, the competitive advantage will come from superior adoption and application, not from trying to build their own slightly more specialized version of intelligence in violation of the well-established Bitter Lesson.
- 46
@_vmlops ·
NVIDIA JUST TURNED VISION MODEL POST-TRAINING INTO A TWO-PROMPT WORKFLOW cosmos 3 nano went from 54.41% to 93.35% accuracy on a traffic safety benchmark, and a coding agent did most of the work ▪️ prompt 1: agent runs baseline eval, patches a missing dataset param, then kicks off LoRA fine-tuning jumps to 87.14% in ~30 min on 8x A100s ▪️ prompt 2: agent runs a TAO AutoML sweep across LoRA rank, learning rate, dropout, etc. pushes it to 93.35% ▪️ LoRA needed ~7x fewer GPU hours than full-parameter SFT the trick is TAO agent skills, which teach the coding agent the framework details, config structure, and eval workflow so it can reason through the whole pipeline instead of you hand-writing every command. deploy is just as frictionless: cosmos 3 reasoner NIM serves the LoRA adapter as an OpenAI-compatible endpoint, no manual CUDA setup
Watch video
- 47
@JulianGoldieSEO ·
𝗚𝗼𝗼𝗴𝗹𝗲'𝘀 𝗦𝗶𝗺𝘂𝗹𝗮 𝗯𝘂𝗶𝗹𝗱𝘀 𝗳𝗮𝗸𝗲 𝗔𝗜 𝘁𝗿𝗮𝗶𝗻𝗶𝗻𝗴 𝗱𝗮𝘁𝗮 𝗳𝗿𝗼𝗺 𝘁𝗵𝗶𝗻 𝗮𝗶𝗿. It scaled to 512,000 data points across cyber and legal tasks. Three stages cover the whole topic on purpose. Two critic models filter out the weak examples. 61% of the legal data got tossed in quality checks. It already powers Android scam detection and message spam filters. Pick your own quality, diversity, and complexity knobs. Save this. AI training data just got a fresh playbook.
Watch video
- 48
- 49
@DanKornas ·
Moving from an R&D idea to a working experiment takes more than a prompt—it requires research, code, execution, and feedback. R&D-Agent is an open-source LLM-agent framework for builders automating data-driven research and development. It helps you move from an idea or source document to runnable experiments by separating research from development, then iterating from execution feedback. Key features: • Idea-to-code loop – proposes hypotheses, implements experiments, and refines them from validation feedback • Data science agent – supports feature engineering and model tuning for custom datasets and Kaggle scenarios • Quant workflows – supports factor and model proposal, report-based factor extraction, and Qlib-backed evolution • Research copilot – reads papers or financial reports and implements extracted models, formulas, or datasets • LLM fine-tuning – includes a benchmark-driven loop for data processing, training, evaluation, and refinement It’s open-source (MIT license). Link in the reply 👇
- 50
Best Tweets by Topic