NVIDIA plans to release a new inference chip integrated with Groq's &#34;Language Processing Unit&#34; (LPU) technology at next month's GTC developer conference, representing NVIDIA's acceleration towards the field of inference computing to meet the urgent demand from customers for efficient and low-cost computing solutions.
According to The Wall Street Journal, this new system, described by NVIDIA CEO Jensen Huang as &#34;something the world has never seen,&#34; is designed specifically to accelerate query responses for AI models. The launch of this product is expected to reshape the current AI computing power market landscape, directly impacting cloud service providers and enterprise investors seeking more cost-effective alternatives.
As an important sign of preliminary market recognition of this technology, ChatGPT developer OpenAI has agreed to become one of the largest customers of this new processor and announced plans to purchase large-scale &#34;dedicated inference capacity&#34; from NVIDIA. This move not only solidifies NVIDIA's core customer base but also sends a clear signal to the market: the underlying infrastructure supporting autonomous AI agents is shifting from large-scale pre-training to efficient inference.
In the face of fierce competition from Google, Amazon, and numerous startups, NVIDIA is breaking away from its traditional reliance on graphics processing units (GPUs). By introducing new technological architectures and exploring pure central processing unit (CPU) deployment models, the company aims to continue consolidating its market dominance in the next phase of AI industry evolution.
<h2>Integrating LPU Design to Address Large Model Inference Bottlenecks</h2>
As the AI industry shifts from model training to actual application deployment, inference computing has become the core focus. AI inference is mainly divided into two stages: pre-fill and decode, with the decoding process of large AI models being particularly slow. To address this technical bottleneck, NVIDIA has chosen to break through physical limits through external technology integration.
According to The Wall Street Journal, NVIDIA spent $20 billion at the end of last year to acquire key technology licenses from the startup Groq and brought in an executive team, including founder Jonathan Ross, in a large-scale &#34;core hiring&#34; deal. The &#34;Language Processing Unit&#34; (LPU) designed by Groq features an architecture that is fundamentally different from traditional GPUs, demonstrating extremely high efficiency in processing inference functions.
Industry analysts believe that the upcoming product may involve a disruptive next-generation Feynman architecture. According to a previous article from Wallstreetcn, the Feynman architecture may adopt a more extensive SRAM integration scheme and even deeply integrate the LPU through 3D stacking technology, specifically optimizing for the two major inference bottlenecks of latency and memory bandwidth, thereby significantly reducing the energy consumption and costs of AI agents' operations
<h2>Expanding Pure CPU Deployment to Provide Diverse Computing Options</h2>
While introducing the LPU architecture, NVIDIA is also flexibly adjusting the way it uses its traditional processors. NVIDIA's previous standard practice was to bundle the Vera CPU with its powerful Rubin GPU in data center servers, but this configuration has proven to be too costly and inefficient in terms of energy when handling certain specific AI agent workloads.
Some large enterprise clients have found that pure CPU environments are more efficient for running specific AI tasks. In line with this trend, NVIDIA announced this month an expansion of its collaboration with Meta Platforms, conducting its first large-scale pure CPU deployment to support Meta's advertising-targeted AI agents. This collaboration is seen by the market as an early window into NVIDIA's strategic adjustment, indicating that the company is moving beyond a single GPU sales model and attempting to lock in different segments of the AI market through a diversified hardware combination.
<h2>Market Demand Shifts Gears, Competitive Landscape Continues to Escalate</h2>
The evolution of this underlying hardware design is directly driven by the explosion of demand for AI agent applications in the tech industry. Many companies building and operating AI agents have found that traditional GPUs are too expensive and not the best choice for running models in practice.
OpenAI's movements highlight this trend. In addition to committing to purchase NVIDIA's new systems to improve its rapidly growing Codex tool, OpenAI also reached a multi-billion dollar computing partnership with startup Cerebras last month. According to Cerebras CEO Andrew Feldman, its inference-focused chips outperform NVIDIA's GPUs in speed. Additionally, OpenAI has signed a significant agreement to use Amazon's Trainium chips.
Not only startups, but major cloud service providers are also accelerating their self-developed chip efforts. Widely regarded as a leader in the automated coding market, Anthropic Claude Code currently relies primarily on chips designed by Amazon AWS and Alphabet's Google Cloud, rather than NVIDIA's products. In the face of competitors' encroachment, Jensen Huang emphasized in an interview with wccftech that NVIDIA is transforming from a pure chip supplier to a builder of a complete AI ecosystem encompassing semiconductors, data centers, cloud, and applications. For investors, next month's GTC conference will be a key moment to test whether NVIDIA can maintain its 90% market share myth in the era of inference

NVDA

OpenAI

NVIDIA's upcoming inference chip system integrates Groq's "Language Processing Unit" (LPU) technology, utilizing an architecture that is fundamentally different from traditional GPUs. It is optimized specifically for latency and memory bandwidth bottlenecks in large model inference through broader SRAM integration and 3D stacking technology. This new product may be based on the next-generation Feynman architecture design, significantly reducing the energy consumption and costs of AI agents. OpenAI has committed to purchasing and investing $30 billion

- NVIDIA is set to unveil a new inference chip integrating Groq's Language Processing Unit (LPU) technology at next month's GTC conference, aiming to accelerate its shift towards inference computing.  
- This product, described by CEO Jensen Huang as unprecedented, is designed to enhance AI model query responses and potentially reshape the AI processing market, attracting cloud service providers seeking cost-effective solutions.  
- Following OpenAI's agreement to become a major customer for this new processor, both NVIDIA's strategy and market competition are evolving as companies explore alternatives to traditional GPUs.

Wallstreetcn

NVIDIA

SPDR S&P Semicon

Invesco Semiconductors ETF

iShares Semiconductor ETF

Direxion Semicon Bull 3X

GraniteShares 2x Long NVDA Daily ETF

T-Rex 2X Long NVIDIA Daily Target ETF

VanEck Semiconductor ETF

Spdr Select Tech

XL2CSOPNVDA

XI2CSOPNVDA

YieldMax NVDA Option Income Strategy ETF

Direxion Daily NVDA Bear 1X ETF

T-Rex 2X Inverse NVIDIA Daily Target ETF

Report: NVIDIA will launch a "new inference chip" incorporating Groq LPU design at next month's GTC conference