Amidst the AI buzz, it’s easy to forget that as data speeds up like a runaway train, our need for beefier processors to keep up with AI’s demands grows exponentially. But who knows? Maybe in the future, AI will be so brilliantly efficient that it could run on a stack of dusty old laptops in your garage. That seems unlikely, though, so the race to build the very best processors is fierce.

Enter NVIDIA, the reigning champ of the processor world, flexing its tech muscles yet again with the announcement of the HGX H200, succeeding its highly sought-after predecessor, the H100. The H200 features a 1.4x increase in memory bandwidth and a 1.8x enhancement in memory capacity, crucial for handling demanding generative AI tasks.

A notable advancement is the incorporation of HBM3e memory, boosting the memory bandwidth to 4.8 terabytes per second and total memory capacity to 141GB. This is not just an upgrade; it’s a very big upgrade.

Hold your power cords, though – the H200 won’t hit shelves until Q2 2024. NVIDIA has not provided specific details on production volumes, raising questions about its availability given the supply constraints experienced with the H100. Given the gold rush for the H100 and its supply issues, we might as well start camping outside NVIDIA’s headquarters now.

Designed to play nice with the H100 systems, the H200 facilitates easy integration for cloud service providers. Expect Amazon, Google, Microsoft, and Oracle to elbow each other out of the way to get their hands on these.

H100s cost about $30,000… if you can find them. The H200 will cost more, but the real game here will be finding one. ​

Author’s note: This is not a sponsored post. I am the author of this article and it expresses my own opinions. I am not, nor is my company, receiving compensation for it. This work was created with the assistance of various generative AI models.

About Shelly Palmer

Shelly Palmer is the Professor of Advanced Media in Residence at Syracuse University’s S.I. Newhouse School of Public Communications and CEO of The Palmer Group, a consulting practice that helps Fortune 500 companies with technology, media and marketing. Named LinkedIn’s “Top Voice in Technology,” he covers tech and business for Good Day New York, is a regular commentator on CNN and writes a popular daily business blog. He's a bestselling author, and the creator of the popular, free online course, Generative AI for Execs. Follow @shellypalmer or visit



PreviousChatGPT's New Era: Exploring GPT-4 Turbo NextYouTube's New Policy on AI-Generated Music (and other content)

Get Briefed Every Day!

Subscribe to my daily newsletter featuring current events and the top stories in technology, media, and marketing.