Qwen3.8-Flash-Next Hits Performance Peak: NVFP4 Powers 2,900 t/s Prefill on 2xDGX Spark
A breakthrough configuration for Qwen3.8-Flash-Next on 2xDGX Spark clusters has been revealed, leveraging NVFP4 quantization to achieve a decoding speed of 50 t/s and a massive prefill throughput of 2,900 t/s.
- ▶ NVFP4 as the Blackwell Performance Benchmark: This setup capitalizes on native FP4 hardware acceleration within the NVIDIA Blackwell architecture (sm_121), signaling a drastic reduction in costs for long-context processing.
- ▶ The “Shadow Branch” Strategy in vLLM: Critical optimizations reside in the
release/qwen38nextbranch rather than the main repo, highlighting that bleeding-edge performance for next-gen models currently requires navigating specialized, non-public commits. - ▶ Kernel Patches as the Final Bottleneck: Full sm_121 support necessitates a manual two-file patch, underscoring the gap between hardware availability and mainstream software readiness in elite AI infrastructure.
Bagua Insight
The real story here isn’t the 50 t/s decode speed—which is largely bound by interconnects—but the staggering 2,900 t/s prefill rate. For enterprise-grade RAG and autonomous agents, prefill efficiency is the primary driver of Time-To-First-Token (TTFT) and overall system concurrency. The successful implementation of NVFP4 marks a paradigm shift where quantization is no longer just about memory savings; it’s about unlocking the raw computational throughput of Blackwell’s Tensor Cores. This effectively rewrites the unit economics of high-scale LLM inference.
Actionable Advice
Engineering teams prioritizing inference throughput should immediately monitor vLLM PR #53896 and its associated branches to benchmark FP4 precision loss against performance gains. Infrastructure leads must prepare for sm_121 kernel-level patching to fully utilize Blackwell hardware upon deployment. Given the seamless integration of the Qwen series with NVFP4, it should be considered a top-tier candidate for building low-latency, high-concurrency AI applications such as real-time financial analysis or interactive agents.