NVidia, The Trillion Dollar AI Chip Supplier
In the AI Gold Rush, NVidia is selling picks and shovels
NVidia Hits a Trillion Dollar Valuation
In late May, NVidia announced their previous quarterly earnings and gave a remarkable upgrade in its revenue and earnings forecasts. NVidia upgraded their sales forecasts for the second quarter from $7 billion, they announced that going to bring in revenue of $11 billion.
These results were so shockingly good, it sent NVidia’s stock market capitalization up almost $200 billion dollars - overnight. Since then the stock has climbed to where the company is now worth more than a trillion dollars.
What was the cause for that massive upgrade in revenue estimates? There’s an AI Gold Rush going on, and NVidia is selling the picks and shovels as fast as they can:
“Our entire data center family of products — H100, Grace CPU, Grace Hopper Superchip, NVLink, Quantum 400 InfiniBand and BlueField-3 DPU — is in production. We are significantly increasing our supply to meet surging demand for them.”
All of these high-demand products are part of the AI tool chain. AI companies wanting to train large ever-larger AI models is creating insatiable demand for the data centers and supercomputers based on the latest NVidia H100 GPU and supporting high-capacity networking products.
The announcement and market reaction to it has been driving the whole stock market higher in recent weeks. Some market watchers worry that NVidia’s great revenue announcement is creating false optimism towards other companies that are not participating in the AI boom, leading to a headline that asks the question: Did Nvidia ruin earnings season for stock market bulls?
The AI Pick and Shovel Seller
There’s no doubt that NVidia is on fire due to generative AI. NVidia’s CEO Jensen Huang claims this demand shift is not just a flash-in-the-pan hype, not like the bitcoin mining boom and bust. Rather, he says this is a fundamental shift in the cloud computing and data center build-out to support the rise of generative AI:
"We're seeing incredible orders to retool the world's data centers. And so I think you're seeing the beginning of, call it, a 10-year transition to basically recycle or reclaim the world's data centers and build it out as accelerated computing," Huang said. "You'll have a pretty dramatic shift in the spend of a data center from traditional computing and to accelerate computing with SmartNICs, smart switches, of course, GPUs, and the workload is going to be predominantly generative AI."
The world will need a lot more GPUs in the data centers. How big is this shift and opportunity? Huang puts it at a trillion dollars of installed data center infrastructure.
“A trillion dollars of installed global data center infrastructure will transition from general purpose to accelerated computing as companies race to apply generative AI into every product, service and business process.” - NVidia CEO Jensen Huang
NVidia has been pursuing the market for what they call ‘accelerated computing’ for almost 30 years, selling parallel computing GPUs for a variety of tasks long before it became a workhorse for AI. A GPU is a general-purpose parallel processing engine, designed to handle complex graphics operations for gaming, video editing, and other high-performance computing tasks. The term GPU stands for “Graphics Processing Unit,” which speaks to NVidia’s heritage selling GPU video cards for PC gamers.
The latest generation of NVidia GPU is the H100, based on 4nm TSMC technology and unveiled in 2022. The H100 provides a significant performance gain over the A100 GPU that has been the work-horse for the current wave of AI models, with a 9X speedup when equipped with H100 and NVLink.
The total technical stack required for high-performance on AI workloads is very demanding. It requires not just the parallel compute GPUs, but also having supporting high-bandwidth inter-processor communications and the parallel software frameworks to take full advantage of the hardware.
NVidia puts these elements together in their DGX platform and data center products. Their latest performance benchmarks results - busting ML performance records - attests to its effectiveness.
Providing a full stack of technologies and being able to sell turnkey supercomputer and data center solutions is very appealing to AI companies. They want to ramp up quickly and focus on training models, not provision data centers. It’s no surprise then that NVidia is in a very strong market position, as they have spent literally decades building the hardware and software technology stack to support their overall solution.
Competition in the Market for AI Chips
It will be difficult to displace NVidia, but just as nature abhors a vacuum, competitors seek to undermine NVidia’s near-monopoly and fat profit margins. Here’s where other chip-making players are in the race for AI chips and supporting hardware for AI.
Google: While not a semiconductor company, one of the biggest users of deep learning decided to make their special-purpose chip. The Google TPU (Tensor Processing Unit) is a specialized processor designed specifically for machine learning workloads. Google’s latest deployed TPU version is TPU v4, which outperforms previous TPU v3 by more than 2X. It has optical circuit switches for better power consumption. It is optimized for matrix multiplication operations and is used for training and inference of machine learning models.
This has been built and used internally by Google for building AI models, but others can get access to the TPUs via Google cloud service and the TPU v4 Pod. In an interesting inception-like manner, Google used TPU-trained ML models to help design its next generation TPU chips.
AMD: AMD is a long-time competitor in the video-graphics space, and their latest generation video cards compete with NVidia’s best consumer video cards. AMD also recently announced their most advanced entry in the AI chip space, the Instinct M1300. The M1300 is massive, with 10,000 GPU cores, 24 Zen CPU cores, and 128GB of memory, in a chiplet design. The 147 billion logic transistors makes it larger than the also massive 80 billion transistor H100 Hopper from NVidia.
Intel: Having only participated in the GPU market for integrated graphics, Intel was for many years out of the running for high-end graphics chips, and that’s put them on the back-foot when it comes to AI chips. However, in 2019, they bought Habana Labs for their AI chips, and currently offer the Gaudi2 AI chip. These chips claim an approximate 2X speed up versus A100, but that also implies these are behind the H100.
Intel also offers the Ponte Vecchio chip, a 100 billion transistor chip designed for high-performance computing that can take on AI training workloads.
Even if these chips from AMD and Intel don’t exactly match the H100 on a chip-per-chip basis, they can offer a solid value proposition based on pricing. One might expect the competitors would grab a slice of market share, but they haven’t gained too much traction compared with NVidia’s near-monopoly. Why is that?
NVidia has over 80 percent GPU market share for AI and is likely to keep most of it for now, because NVidia has a strong AI-supporting ecosystem for model training and serving. It’s not just the chips, but the interconnect, data center infrastructure and deep-learning software tuned to NVidia GPUs with the CUDA library. It’s also the fact that AI algorithms are tuned to the advantages of NVidia GPUs. Part of NVidia’s moat is the symbiosis of AI training algorithms and the NVidia GPU architecture.
If AMD and Intel want to compete, they will need to work on building their technology stacks and ecosystems to fully support AI.
AI changes everything and one of the things AI is disrupting is the chip business. The data center will change and the nature of edge devices will change. Apple is including neural inference chips in their computers and iPhones. Microsoft Azure and AWS are provisioning to enable training and inference of custom AI models.
NVidia is and will likely remain dominant for the time being, but this market for AI chips is growing and will be huge. There will be space for multiple profitable chip providers in the market. In the meantime, hold on to your NVDA shares.