SFT

Event Core A provocative new paper gaining traction in the LocalLLaMA community challenges the prevailing dogma that massive Reinforcement Learning (RL) is the only path to advanced reasoning in LLMs (e.g., OpenAI’s o1 or DeepSeek-R1). The researchers claim that the performance gains attributed to RL actually stem from subtle shifts in just 1-3% of the model's tokens. By isolating these critical changes, the study demonstrates that reasoning capabilities can be replicated using targeted Supervised Fine-Tuning (SFT) or distillation at approximately 1/1000th of the computational cost of traditional RL pipelines. In-depth Details The technical crux of the argument lies in the "Sparsity of Reasoning Paths." Traditional RL, such as PPO or the more recent GRPO, functions as a high-entropy search mechanism, exploring vast output spaces to reward correct Chain-of-Thought (CoT) behaviors. However, the study reveals that the resulting distributional shift is surprisingly narrow. The "intelligence" isn't a total re-wiring of the model but a precision adjustment of key logical pivots. Compute Arbitrage: RL is notoriously compute-heavy due to the need for multiple rollouts and reward model inference. This research suggests a massive efficiency arbitrage: if you can identify the "winning" tokens, you can bypass the RL tax entirely. The Distillation Shortcut: This explains the rapid rise of "open-weights" reasoning models. By using a teacher model (like o1) to generate high-quality reasoning traces, a smaller model can be SFT-ed to mimic that 1-3% token distribution, achieving "o1-level" logic without ever running a reward function. Token Significance: The paper posits that reasoning is less about massive parameter updates and more about the structural alignment of logical connectors and self-correction tokens. Bagua Insight At 「Bagua Intelligence」, we view this as a pivotal moment in the "Post-Training Efficiency" era. The Silicon Valley narrative has long been: "More Compute = More Intelligence." This paper exposes RL as a brute-force search tool that is inherently inefficient for scaling. If the delta between a standard LLM and a reasoning LLM is only a 3% token variance, then the "moat" built on massive H100 clusters is shallower than previously thought. This discovery democratizes high-level reasoning. It signals a shift from "Compute-Native AI" to "Data-Architected AI." For global players, especially those under compute constraints, this provides a roadmap to parity. The "o1 effect" is no longer a proprietary secret of the trillion-dollar clubs; it is a data engineering problem that can be solved with precision rather than power. Strategic Recommendations For AI Labs: Pivot from "RL-First" to "RL-Informed SFT." Use RL as a discovery engine to find optimal reasoning paths, then use those paths to train production models via SFT to save millions in OpEx. For Enterprise Buyers: Don't overpay for "RL-trained" models if the same logic can be achieved via distillation. Focus on models that demonstrate high "Inference-time Compute" efficiency rather than training-time complexity. Data Strategy: Prioritize the acquisition of "Negative Reasoning Traces" and "Correction Paths." Since the 1-3% of tokens are the key, understanding where models fail and how they pivot is more valuable than 100TB of raw text.

Event CoreA groundbreaking research direction is gaining traction: leveraging frontier models to generate high-quality execution trajectories, which are then used to Supervised Fine-Tune (SFT) smaller models. This process effectively 'compiles' complex agentic logic directly into the model weights, achieving near-frontier quality at two orders of magnitude less cost.▶ From Prompting to Parametric Logic: Complex reasoning chains are no longer a runtime overhead but an architectural feature, significantly reducing latency and context window pressure.▶ The Economic Singularity: A 100x reduction in inference costs transforms previously cost-prohibitive agentic workflows into commercially viable production-grade solutions.Bagua InsightAt 「Bagua Intelligence」, we view this as the dawn of the 'Compilation Era' for GenAI. We are moving away from treating frontier models like GPT-4o as permanent infrastructure and toward using them as 'expensive teachers.' By distilling the reasoning traces of an agent into 8B or 70B models, developers are essentially moving logic from the 'software layer' (prompts) to the 'firmware layer' (weights). This shift addresses the two biggest pain points in the current Agentic landscape: brittleness and cost. This is a strategic pivot—the value is shifting from the raw model to the proprietary 'trajectory datasets' that capture domain-specific expertise. The future belongs to those who can turn expensive inference into cheap, specialized intelligence.Actionable AdviceOrganizations should immediately start harvesting 'Golden Trajectories'—the successful step-by-step execution paths of their current high-end LLM agents. Stop burning OpEx on frontier API calls for repetitive, high-volume tasks. Instead, invest in a pipeline to distill these workflows into specialized open-source models. Focus on 'Trajectory Engineering' rather than just Prompt Engineering; the goal is to build a data flywheel where frontier models act as the ground-truth generators for your own lightweight, high-performance fleet.