MangoBoost Achieves Breakthrough in Multi-Node LLM Training on AMD GPUs, Setting New Standards in MLPerf Training v5.0
MangoBoost, a leading provider of advanced system solutions designed to maximize compute efficiency and scalability, has achieved a significant milestone in AI training by validating the scalability and efficiency of large-scale AI workloads on AMD Instinct™ MI300X GPUs. This accomplishment was showcased through its submission to MLPerf Training v5.0, where MangoBoost demonstrated exceptional performance in fine-tuning the Llama2-70B-LoRA model using 32 AMD Instinct™ MI300X GPUs across four nodes. The process was completed in just 10.91 minutes, setting a new benchmark for multi-node LLM training on AMD GPUs. Remarkably, the system achieved near-linear scaling efficiency (95–100%), proving that MangoBoost’s stack is not only capable of handling demanding benchmarks but also ready for practical, production-grade LLM training.
Scalability and Efficiency for Enterprise Data Centers
This achievement represents more than just a benchmark win—it highlights how enterprises can reliably scale LLM training across GPU clusters without encountering network bottlenecks or being constrained by rigid infrastructure dependencies. MangoBoost’s solution is tailored for enterprise data centers that prioritize performance, flexibility, and cost-efficiency, offering a viable alternative to traditional vendor-locked GPU platforms.
Central to this success are two key innovations from MangoBoost:
- Mango LLMBoost™:
A full-featured MLOps software platform specifically designed for large language models (LLMs). It supports advanced features such as model parallelism, automatic tuning, batch scheduling, and sophisticated memory management. These capabilities ensure seamless orchestration of complex AI workloads while optimizing resource utilization and reducing training times. - Mango GPUBoost™ RoCEv2 RDMA:
A hardware solution optimized for inter-GPU communication, leveraging low-latency, high-throughput Remote Direct Memory Access (RDMA) over Converged Ethernet (RoCEv2). This technology sustains line-rate performance across thousands of concurrent Queue Pairs (QPs), enabling predictable and efficient multi-node training. Whether organizations operate their own AI infrastructure or deploy on public cloud environments, MangoBoost’s stack delivers unmatched reliability and scalability.
Industry-First MLPerf Training on AMD MI300X GPUs
This marks the first-ever MLPerf Training submission on AMD GPUs spanning multiple nodes. MangoBoost’s platform demonstrated robust performance with a 4-node, 32-GPU cluster and confirmed compatibility with additional model sizes and structures, including Llama2-7B and Llama3.1-8B, in internal benchmarks. These results validate the generalizability of MangoBoost’s platform, extending its applicability beyond benchmarks to diverse, production-scale use cases.
David Kanter, Founder and Head of MLPerf at MLCommons, praised the achievement:
“I’m excited to see MangoBoost’s first MLPerf Training results, pairing their LLMBoost AI Enterprise MLOps software with their RoCEv2-based GPUBoost DPU hardware to unlock the full power of AMD GPUs. Their scalable performance—from a single-node MI300X to 2- and 4-node MI300X results on Llama2-70B LoRA—underscores that a well-optimized software stack is critical to fully harness the capabilities of modern AI accelerators.”
Vendor-Neutral AI Infrastructure Enabled by AMD Collaboration
This groundbreaking result was made possible through deep collaboration with AMD and seamless integration with the ROCm™ software ecosystem, which enables full utilization of the MI300X’s compute power, memory bandwidth, and capacity. By leveraging AMD’s cutting-edge hardware and software, MangoBoost has empowered enterprises to choose infrastructure based on business needs rather than being locked into specific vendors.
Meena Arunachalam, Fellow and AI Performance Design Engineering Lead at AMD, expressed her enthusiasm:
“We congratulate MangoBoost on their MLPerf 5.0 training results on AMD GPUs and are excited to continue our collaboration with them to unleash the full power of AMD GPUs. In this MLPerf Training submission, MangoBoost has achieved a key milestone in demonstrating training results on AMD GPUs across 4 nodes (32 GPUs). This showcases how the AMD Instinct™ MI300X GPUs and ROCm™ software stack synergize with MangoBoost’s LLMBoost™ AI Enterprise software and GPUBoost™ RoCEv2 NIC.”
Unlocking Enterprise-Scale AI Training
Jangwoo Kim, CEO of MangoBoost, emphasized the significance of this achievement:
“At MangoBoost, we’ve shown that software-hardware co-optimization enables scalable, efficient LLM training without vendor lock-in. Our MLPerf result is a key milestone proving our technology is ready for enterprise-scale AI training with superior efficiency and flexibility.”
The company continues to push the boundaries of distributed AI workloads by developing innovations in communication optimization, hybrid parallelism, topology-aware scheduling, and domain-specific acceleration. These advancements aim to further scale performance and efficiency, ensuring MangoBoost remains at the forefront of the AI infrastructure landscape.
Why This Matters for the Future of AI
As AI models grow in complexity and size, the ability to efficiently train them at scale becomes increasingly critical. Traditional approaches often require proprietary hardware and software stacks, limiting flexibility and increasing costs. MangoBoost’s achievement demonstrates that organizations can now achieve state-of-the-art LLM training performance using AMD GPUs, unlocking a new era of vendor-neutral AI infrastructure.
By combining software-hardware co-optimization with open ecosystems like AMD’s ROCm™, MangoBoost is paving the way for enterprises to build flexible, high-performance AI systems that meet their unique needs. This breakthrough not only sets a new benchmark for multi-node LLM training but also reinforces the importance of innovation in distributed computing and AI acceleration.
With its proven track record and ongoing commitment to advancing AI infrastructure, MangoBoost is poised to play a pivotal role in shaping the future of scalable, efficient, and accessible AI training solutions. As industries continue to adopt AI at an unprecedented pace, MangoBoost’s contributions will undoubtedly drive progress and unlock new possibilities for businesses worldwide.
