Intel Innovation and the AI Chip Wars
Intel's Core Ultra with NPU and GPU on-board signals the era of AI PCs has arrived
My latest article “Running Your Own AI Models” made some snarky comments about how Apple chips were currently superior to Intel and the Windows PC running AI loads on a laptop. I said: We’ll know the era of edge-run AI has arrived when Intel processors come equipped with AI inference engines on-chip to compete with Apple.
Well, that moment is arriving soon, in mid-December in fact. And that processor is called Meteor Lake, to be renamed Core Ultra as a product.
At Intel Innovation 2023, Intel made a number of AI announcements, and AI figured prominently in their pitch for a number of products and technology advancements.
Pat Gelsinger, Intel CEO, calls it the “Siliconomy”, by which he means the technology, product, and economic stack that comprises the $8 trillion global technology industry, all built on the substrate of silicon technology and computer chips. Our digital economy rests on literal sand.
With the acceleration of Artificial Intelligence, that entire ecosystem is shifting right down to the silicon. What Meteor Lake and other advances represent is an ongoing and large shift towards chips and features on chips to serve AI workloads and use cases.
“AI represents a generational shift, giving rise to a new era of global expansion where computing is even more foundational to a better future for all,” - Pat Gelsinger
Intel Innovation 2023
Meteor Lake - Core Ultra
As mentioned, the most immediate upcoming chip that signals embrace of AI is Meteor Lake, to be marketed as Core Ultra, dubbed the “Intel’s biggest laptop CPU update in years.”
Its Intel’s first consumer process using chiplets instead of a monolithic die, with four chiplet ‘tiles’ manufactured by both Intel and TSMC, combined using a packaging technology called Feveros to connect them.
These advances enable a number of notable features for AI, specifically:
It includes a neural processing unit (NPU) AI engine on board, in their SoC tile, for efficient acceleration of AI inference.
It includes an Arc integrated GPU with “twice as much performance per watt” compared to the prior generation, that loads 128 of Intel's Vector Engines (formerly Execution Units, or EUs)
The payoff from these improvements are that a number AI workloads, will run smoother, faster and at lower power. They mention generative AI, NLP, computer vision, image enhancements, and audio (such as speech-to-text) and gave an example of speeding up Stable Diffusion, where the NPU showed a 7x improvement in efficiency over using the CPU for running Stable Diffusion.
Putting the NPU directly in the silicon only matters if software can take advantage of it. To that end, Intel showed off a number of AI applications running locally, and touted how Microsoft have Co-pilot and other applications accelerated by the NPU.
Time will tell how the Meteor Lake “Core Ultra” stacks up against Apple’s M2 Ultra, but Intel has promised to be “just as powerful and efficient as the worlds best CPUs,” showing they intend to be world-beating for AI on the edge.
OpenVINO
To support edge AI workloads, you need good software support for running AI on the edge. Intel supports OpenVINO as an AI inferencing and deployment toolkit to “optimize and deploy AI inference at the edge.” It automatically maps how AI models are inferred using targeted compute resources. This is important as a part of Intel’s developer ecosystem to support AI application adoption on their devices.
As part of supporting flexible uses of AI, Intel will release a Hybrid AI SDK next year to help drive use cases where AI inference may switch from local to cloud depending on edge resources.
Intel’s AI Supercomputers
Another big Intel announcement was a deal to provide Stability AI a massive AI supercomputer. This AI supercomputer is built on Intel Xeon processors and 4,000 Intel Gaudi2 AI hardware accelerators. It will be the largest AI supercomputer in Europe and in the top 15 in the world.
“StabilityAI is the anchor customer,” Pat Gelsinger said of the system and deal.
NVidia remains the market leader for GPUs for AI by far, but the market is large and NVidia’s H100s are backlogged for many quarters. Gaudi2 is a credible GPU on par with H100 depending on the benchmark:
According to those benchmark results, Gaudi2 outperformed H100 for gaining multimodal transformer BridgeTower model, but Gaudi2, using Optimum Habana, achieved x2.5 times better performance than the A100. Not only did the results validate Gaudi2’s place in the field of AI but also in Vision-Language training.
So this offering may give Intel a foothold in the market. In addition to this deployment, Intel is working with other systems providers like Dell to beef up AI-related offerings, and providing more AI capabilities in their Xeon chip for the data center market.
AI Chips Wars
The shift to AI is disrupting the entire economy, and the base of that disruption is in the semiconductor industry.
As with other tech companies in 2023, Intel leaned in heavily on AI this year, showing they are responding to the shift. Aside from AI-related product announcements and using AI demos to showcase capabilities, Intel also gave the 2023 Intel Lifetime Achievement for 2023 to Dr. Fei-Fei Li, Professor at Stanford University, for her contributions to Artificial Intelligence.
Intel was the world’s semiconductor leader for forty years of CPU dominance, but recently found itself on the backfoot with the rise of NVidia and their GPU eclipsing the CPU in importance. The rise of AI is changing the PC and edge devices in ways as profound as the rise of the internet and mobile. It’s the next iteration for technology.
NVidia is still the king in GPUs, and H100 is still the in-demand market leader. They will keep improving on their products. AMD is not standing still either; their high-end chips with on-board GPU are performance leaders for PC gaming, and can also serve well for AI use cases.
But just as Google is responding to the rise of AI upstart competition, so too Intel seems to be rising to the challenge as well, with edge chips and systems built for AI and improving products to serve AI data center market.
Choice is good. Competition is good. In the end, whether it’s an Acer AI PC based on Intel Core Ultra or a MacBook with M2 Ultra, the consumer wins, because they are enabled to use AI apps and features unimagined not long ago.
The takeaway from Intel’s show this week is this: The era of AI chips, AI devices, AI PCs and AI data centers has arrived.