284B MoE Performance Peak: DeepSeek-V4-Flash Hits 75 tok/s on Dual DGX Spark Cluster
Event Core
A production-grade deployment recipe for DeepSeek-V4-Flash-0731 (284B MoE) has surfaced, demonstrating a throughput of 74.8 tokens/sec on a dual DGX Spark setup. By leveraging QSFP DAC interconnects, vLLM, speculative decoding, and NVFP4 quantization, the project provides a comprehensive blueprint—including 11 critical “gotchas” and a reboot-proof cluster configuration—now available as an open-source contribution.
- ▶ Performance Engineering: The integration of NVFP4 and speculative decoding allows a 284B parameter model to run at speeds previously reserved for much smaller architectures, effectively neutralizing the memory-bandwidth bottleneck of massive MoE models.
- ▶ Enterprise Readiness: Moving beyond simple inference scripts, this implementation focuses on infrastructure resilience, featuring automated recovery and seamless Codex CLI integration for production environments.
Bagua Insight
The significance of this deployment lies in the democratization of “Super-Scale” inference. DeepSeek-V4-Flash (284B) is no longer a theoretical giant; it is now a deployable asset for organizations with high-end but standard DGX clusters. The use of NVFP4 is a strategic move—it signals a shift where quantization isn’t just about compression, but about unlocking the specific architectural advantages of the latest NVIDIA silicon. The “11 gotchas” highlighted in the report underscore that the real challenge in GenAI today isn’t model training, but the “last mile” of multi-node orchestration where network jitter and driver mismatches can tank ROI.
Actionable Advice
- Pivot to NVFP4: Organizations utilizing H100/B200 tiers should prioritize NVFP4 workflows to maximize the throughput-to-memory ratio for MoE architectures.
- Optimize Interconnects: For dual-node setups, bypass traditional switching where possible; direct QSFP DAC connections are essential for minimizing the latency overhead inherent in MoE expert routing.
- Standardize the Stack: Adopt “reboot-proof” orchestration frameworks to ensure high availability, moving away from manual CLI launches toward integrated service meshes.