M2 Ultra Reclaims the Throne: DeepSeek V4 Flash Optimized to 141 GiB with 25.8 t/s Throughput

Event Core

A breakthrough optimization for the Apple M2 Ultra (60-core GPU, 192GB Unified Memory) has surfaced on the LocalLLaMA community. By leveraging a custom llama.cpp fork, a developer achieved a byte-level lossless repack of DeepSeek V4 Flash. The model footprint was reduced to a mere 141 GiB—smaller than the standard Q4 GGUF format—while maintaining full precision. Performance metrics are staggering: an average inference speed of 25.8 tokens per second (t/s) with peaks at 42 t/s, effectively outperforming the newer M3 Ultra in specific MoE workloads and supporting up to 1 million context tokens.

In-depth Details

• Lossless Repacking vs. Quantization: Unlike traditional quantization methods that sacrifice perplexity for size, this project utilizes a “byte-level lossless repack.” By optimizing weight mapping and memory alignment, the developer managed to fit the 141 GiB model into the M2 Ultra’s memory pool without any precision loss, leaving ample headroom for the KV cache.
• Dynamic Lanes & SSD KV Offloading: To facilitate a 1M context window, the implementation employs “Dynamic Lanes” and SSD-based KV cache offloading. By swapping inactive context data to high-speed NVMe storage, the system bypasses the physical RAM limitations that typically bottleneck ultra-long sequence processing in local environments.
• MoE Architecture Synergy: DeepSeek V4’s Mixture-of-Experts (MoE) architecture is notoriously bandwidth-hungry. The M2 Ultra’s 800GB/s unified memory bandwidth proves to be the decisive factor here. The results suggest that for sparse MoE activation, the raw bandwidth of the M2 Ultra remains superior to the compute-heavy but bandwidth-constrained configurations of newer iterations.

Bagua Insight

At 「Bagua Intelligence」, we view this as a pivotal moment for “Local AI Sovereignty.” For too long, models of DeepSeek V4’s caliber were deemed “Cloud-Only.” This 141 GiB lossless repack shatters that glass ceiling. It reinforces the dominance of Apple’s Unified Memory Architecture (UMA) for local LLM inference—offering a price-to-performance ratio for memory capacity that multi-GPU setups (A100/H100) struggle to match for small-to-medium enterprises. Furthermore, the fact that an M2 Ultra outperforms an M3 Ultra in this context is a wake-up call: in the GenAI era, memory bandwidth is the true currency, often more valuable than raw TFLOPS. DeepSeek’s architectural efficiency combined with silicon-level optimization is setting the new gold standard for private AI deployment.

Strategic Recommendations

• For Enterprises: Re-evaluate the feasibility of Mac Studio clusters for privacy-centric RAG workflows. The cost-to-context ratio of a 192GB M2 Ultra is currently unbeatable for local production-grade inference.
• For Developers: Shift focus toward MoE-specific memory management and sparse weight optimization. The future of local LLMs isn’t just about smaller models, but about smarter ways to handle massive ones on prosumer hardware.
• Hardware Procurement: Prioritize memory bandwidth over core counts. For those building local AI workstations today, a high-spec M2 Ultra (192GB) offers significantly higher “Information Gain” and longevity compared to newer but more restricted hardware tiers.