Dell Unveils PowerEdge XE9680 Server with Eight AMD Instinct MI300X GPUs
Dell Technologies has announced the launch of its latest server, the PowerEdge XE9680, featuring cutting-edge integration with AMD’s Instinct MI300X AI GPUs. This collaboration marks a significant advancement in generative AI solutions, providing enterprises with robust support for AI performance and innovation.
Powerful AI Performance with MI300X Integration
Dell's new PowerEdge XE9680 server integrates eight AMD Instinct MI300X accelerators, delivering substantial AI performance capabilities. Each server is equipped with 1.5 TB of HBM3 memory and offers a peak theoretical performance of 42 petaFLOPS. This powerful configuration is designed to accelerate the training and inference of large language models (LLMs), enabling businesses to achieve efficient AI-driven insights and innovations.
The server has demonstrated significant cost-effectiveness in single-node configurations. For instance, the Llama 2 70B parameter model was successfully deployed on a single MI300X accelerator and scaled to eight concurrent instances using all eight accelerators. Furthermore, fine-tuning of the Llama 2 70B model was achieved at FP16 precision.
Advanced Technology Integration
The PowerEdge XE9680 server incorporates several advanced technologies to support the rapid implementation and scaling of generative AI solutions. Dell OpenManage Enterprise simplifies deployment, while APEX AIOps software provides intelligent automation. The new deployment services, set to launch in September, ensure smooth integration of these innovations into enterprise operations.
Key components of this deployment include the Dell PowerSwitch Z9664F-ON, which provides 64 ports of 400GbE to support low-latency, high-throughput Ethernet structures for modern AI clusters. Additionally, Broadcom Thor2 AI-optimized NICs offer 400G interconnects, providing low-power connections for the MI300X accelerators.
Expanding the Generative AI Ecosystem
The Dell PowerEdge XE9680 server is supported by the AMD ROCm framework, which extends the ecosystem of Dell’s generative AI solutions. This framework supports various open-source large language models, including PyTorch, TensorFlow, ONNX-RT, and JAX. The AMD Instinct accelerators also come with comprehensive driver support, development toolkits, and APIs.
Dell Omnia simplifies the creation and management of AI clusters, enabling automated configuration and enhancing workload processing efficiency. Additionally, Dell Technologies has introduced an enterprise SONiC distribution version, combining the open-source SONiC platform with Dell PowerSwitch for advanced features and enterprise-grade support.
Generative AI Design Guidelines and Developer Support
In May, Dell and AMD released a generative AI design verification guide, which has recently become available. This guide provides comprehensive instructions for implementing large-scale language model (LLM) inference, model customization, fine-tuning, and retrieval-augmented generation (RAG) technologies. The design is based on open standards, reducing reliance on proprietary AI software suites and allowing developers to utilize open-source LLM models from partners like Hugging Face and Meta.
Transformative Impact on AI Innovation
The collaboration between Dell and AMD, through the introduction of the Instinct MI300X AI accelerators and generative AI solutions, provides enterprises with a powerful foundation for AI advancement. By offering efficient, cost-effective local infrastructure and an open ecosystem, Dell and AMD are driving faster and more effective AI-driven innovations and outcomes.
Dell’s PowerEdge XE9680 server with AMD Instinct MI300X GPUs represents a significant leap forward in generative AI technology, supporting enterprises in leveraging advanced AI capabilities for diverse applications. This integration underscores the ongoing evolution and growing accessibility of AI solutions in the enterprise sector.
