Oracle Cloud Infrastructure (OCI)’s BM.GPU.MI300X.8 instance can support up to 16,384 MI300X accelerators, AMD’s powerful answer to Nvidia’s H100s GPUs. It also leverages ROCm, an AMD software suite containing tools, libraries and compilers to enhance AI development on AMD hardware
Subscribe today for free
“AMD Instinct MI300X and ROCm open software continue to gain momentum as trusted solutions for powering the most critical OCI AI workloads,” said Andrew Dieckmann, corporate VP and general manager for the data center GPU business at AMD.
“As these solutions expand further into growing AI-intensive markets, the combination will benefit OCI customers with high performance, efficiency, and greater system design flexibility.”
Oracle has been rapidly expanding its OCI service to meet the growing intense demands for AI workloads from business across the globe.
It is among the hyperscalers first in line to snap up Nvidia’s upcoming Blackwell GPUs, with a prospective Zettascale supercomputing cluster set to feature 131,072 Blackwell GPUs.
To ensure it’s offering a wide array of hardware options beyond solely Nvidia, Oracle tested AMD’s GPUs to see how they fared powering large language model workloads.
Running Meta’s Llama 2 70B model on MI300Xs, Oracle found the average latency was just 1.5 seconds — with the hardware able to scale linearly as input batch sizes increased.
“The inference capabilities of AMD Instinct MI300X accelerators add to OCI’s extensive selection of high-performance bare metal instances to remove the overhead of virtualised compute commonly used for AI infrastructure,” said Donald Lu, senior vice president for software development at OCI.
“We are excited to offer more choice for customers seeking to accelerate AI workloads at a competitive price point.”
RELATED STORIES
AMD takes on Nvidia with $4.9bn deal to compete for AI supremacy
AMD’s AI chips to challenge Nvidia
Samsung, Vodafone complete first Open RAN call with AMD processors
