AMD MI350X

Event Core In a direct challenge to NVIDIA's dominance in AI infrastructure, a new benchmark reveals that AMD's MI350X, powered by optimized open-source kernels, has achieved a massive throughput of 78,498 output tokens per second for the Qwen3.6-35B-A3B model across an 8-GPU cluster. This milestone underscores a pivotal shift: while NVIDIA's B200 remains the industry benchmark, AMD's raw hardware prowess—specifically in TFLOPS and HBM3e bandwidth—is finally being unlocked by community-driven software optimizations, narrowing the long-standing "CUDA gap." In-depth Details The performance leap centers on the architectural synergy between the Qwen3.6-35B-A3B Mixture-of-Experts (MoE) model and the MI350X's high-bandwidth memory. Despite a 35B total parameter count, the model only activates approximately 3B parameters during inference, making it an ideal candidate for high-throughput scaling. The open-source kernel implementation optimizes the MoE routing and attention mechanisms specifically for the ROCm stack, leveraging the MI350X's superior memory throughput to sustain massive batch sizes. This demonstration proves that when the software bottleneck is removed, AMD's silicon can meet or exceed the performance of Blackwell-class hardware in specific high-concurrency inference workloads. Bagua Insight From the Bagua Intelligence perspective, we are witnessing the dawn of the "Post-CUDA Era." For years, AMD hardware was considered "potential energy"—impressive specs hampered by a fragmented software ecosystem. However, the rise of hardware-agnostic frameworks like OpenAI's Triton and the proliferation of high-performance open-source kernels are neutralizing NVIDIA's software moat. This isn't just a win for AMD; it's a strategic inflection point for hyperscalers and enterprises looking to de-risk their supply chains. If the community continues to bridge the ROCm performance gap via open-source contributions, the premium "NVIDIA Tax" will become increasingly difficult for CFOs to justify. Furthermore, the optimization of a leading Chinese LLM (Qwen) on top-tier Western silicon highlights the globalized nature of AI innovation, regardless of geopolitical friction. Strategic Recommendations For Infrastructure Architects: It is time to move beyond the "NVIDIA-only" mindset. Incorporate AMD MI350X into your benchmarking suites for inference-heavy workloads, particularly for MoE architectures where memory bandwidth is the primary constraint. For ML Engineers: Prioritize expertise in Triton and custom kernel development. Relying solely on proprietary black-box libraries like TensorRT creates vendor lock-in; mastering cross-platform optimization is the new high-ground. For Enterprise Leaders: Monitor the total cost of ownership (TCO) closely. As open-source kernels level the playing field, the decision between NVIDIA and AMD will shift from "capability" to "availability and price-to-performance."