On Monday, Nvidia introduced the H200, a special computer part for teaching and using artificial intelligence models. These models are the ones behind the boom in creative AI.
The H200 is a better version of the H100, the chip that OpenAI used to teach its most advanced language model, GPT-4. Many big companies, new businesses, and government groups all want to get their hands on these chips, but there aren’t many available.
Nvidia’s Latest GPU Innovations
According to an estimate by Raymond James, H100 chips cost somewhere between $25,000 and $40,000. To create the biggest models, you need thousands of these chips working together in a process known as “training.”
Excitement over Nvidia’s AI graphics processing units (GPUs) has significantly boosted the company’s stock, experiencing a more than 230% increase in 2023. Nvidia anticipates earning around $16 billion in revenue for its third fiscal quarter, marking a 170% rise from the previous year.
The notable enhancement in the H200 is the inclusion of 141GB of advanced “HBM3” memory. This improvement aids the chip in performing “inference,” which involves using a trained model to generate text, images, or predictions.
Nvidia claims that the H200 can produce output almost twice as quickly as the H100, based on a test using Meta’s Llama 2 LLM.
The H200 is set to be available in the second quarter of 2024 and will face competition from AMD’s MI300X GPU. Similar to the H200, AMD’s chip boasts additional memory compared to its predecessors, facilitating the accommodation of large models for efficient inference processing.
Nvidia’s H200
Nvidia mentioned that the H200 will work seamlessly with the H100. This implies that AI companies currently using the previous model for training won’t have to alter their server setups or software to adopt the new version.
Nvidia states that the H200 will be accessible in both four-GPU and eight-GPU server setups within the company’s HGX complete systems. Additionally, it will be available as a component in a chip called GH200, where the H200 GPU is paired with an Arm-based processor.
However, it’s worth noting that the H200 might not remain the fastest Nvidia AI chip for an extended period.
Even though companies like Nvidia provide various chip configurations, significant advancements in new semiconductors typically occur every two years. This is when manufacturers shift to a different architecture, unlocking more substantial performance improvements compared to just adding memory or making smaller optimizations. Both the H100 and H200 are built on Nvidia’s Hopper architecture.
In October, Nvidia informed investors that, due to the high demand for its GPUs, it would transition from a two-year architecture update cycle to a one-year release pattern. The company presented a slide indicating its plan to announce and release the B100 chip, which is based on the upcoming Blackwell architecture, in 2024.
Conclusion
Nvidia’s new H200 brings exciting advancements in AI, outperforming its predecessor, the H100. With a surging stock and strong demand, the H200’s advanced features promise faster performance. Yet, the fast-paced tech world suggests even speedier chips, as Nvidia moves to yearly releases, as seen with the upcoming B100 chip in 2024.
