Post
2288
432 GB of ultra-fast HBM4 and up to 23.3 TB/s of memory bandwidth on a single GPU 🤯.
Two weeks ago, we got early access to AMD's new Instinct MI455X, and our first goal was simple: make sure 🤗 Transformers works on day one.
Over the past few weeks, we worked closely with the AMD team to validate the platform, enable Flash Attention, add torchcodec support for multimodal models, and resolve issues uncovered during testing.
The result:
✅ 99.5% success rate across our 24 core Transformers model architectures - already on par with our daily CI on previous AMD and NVIDIA platforms.
The hardware is just as exciting. With 432 GB of HBM per GPU, our early capacity experiments showed more than 3× the concurrent long-context requests compared to MI300, thanks to the much larger KV cache capacity.
A huge thanks to the AMD team for the early access and the great collaboration!
Read the full blog 👇
https://huggingface.co/blog/badaoui/transformers-on-amd-mi455
Two weeks ago, we got early access to AMD's new Instinct MI455X, and our first goal was simple: make sure 🤗 Transformers works on day one.
Over the past few weeks, we worked closely with the AMD team to validate the platform, enable Flash Attention, add torchcodec support for multimodal models, and resolve issues uncovered during testing.
The result:
✅ 99.5% success rate across our 24 core Transformers model architectures - already on par with our daily CI on previous AMD and NVIDIA platforms.
The hardware is just as exciting. With 432 GB of HBM per GPU, our early capacity experiments showed more than 3× the concurrent long-context requests compared to MI300, thanks to the much larger KV cache capacity.
A huge thanks to the AMD team for the early access and the great collaboration!
Read the full blog 👇
https://huggingface.co/blog/badaoui/transformers-on-amd-mi455
