Amidst the AI buzz, it’s easy to forget that as data speeds up like a runaway train, our need for beefier processors to keep up with AI’s demands grows exponentially. But who knows? Maybe in the future, AI will be so brilliantly efficient that it could run on a stack of dusty old laptops in your garage. That seems unlikely, though, so the race to build the very best processors is fierce.
Enter NVIDIA, the reigning champ of the processor world, flexing its tech muscles yet again with the announcement of the HGX H200, succeeding its highly sought-after predecessor, the H100. The H200 features a 1.4x increase in memory bandwidth and a 1.8x enhancement in memory capacity, crucial for handling demanding generative AI tasks.
A notable advancement is the incorporation of HBM3e memory, boosting the memory bandwidth to 4.8 terabytes per second and total memory capacity to 141GB. This is not just an upgrade; it’s a very big upgrade.
Hold your power cords, though – the H200 won’t hit shelves until Q2 2024. NVIDIA has not provided specific details on production volumes, raising questions about its availability given the supply constraints experienced with the H100. Given the gold rush for the H100 and its supply issues, we might as well start camping outside NVIDIA’s headquarters now.
Designed to play nice with the H100 systems, the H200 facilitates easy integration for cloud service providers. Expect Amazon, Google, Microsoft, and Oracle to elbow each other out of the way to get their hands on these.
H100s cost about $30,000… if you can find them. The H200 will cost more, but the real game here will be finding one.
Author’s note: This is not a sponsored post. I am the author of this article and it expresses my own opinions. I am not, nor is my company, receiving compensation for it. This work was created with the assistance of various generative AI models.