OpenAI Unveils Its First Custom AI Chip, Built With Broadcom, in Major Push to Control AI Infrastructure

OpenAI has revealed Jalapeo, the company’s first custom-built AI chip-a pivotal development as OpenAI aims to own more of its technology stack and rely less on third-party semiconductor vendors. The new application-specific integrated circuit (ASIC), designed with Broadcom, is engineered to run AI inference on applications like ChatGPT and Codex, as well as forthcoming AI agents. Unlike graphic processing units (GPUs) typical of the market (Nvidia’s being the leading vendor), Jalapeo is tailored to execute AI tasks, specifically for LLMs in the inference stage – after models have been trained.

Inference-also known as the output phase of AI operations- is among the most costly and rapidly escalating facets of AI development.

In early tests within OpenAI, preliminary figures indicate substantial gains in power and performance on certain AI tasks. According to an insider close to the initiative, who was privy to the development but not authorized to discuss it publicly, Jalapeo demonstrates an equivalent power efficiency to some of Nvidia’s best designs-at least in some AI scenarios-at an estimated fraction of their cost. The hardware chip is an outcome of approximately a nine-month process-from design concept to tangible functioning unit-a speed that far exceeds traditional chip development timelines. The company credits some of its own LLM technologies, in fact, for expediting particular phases of the design work-demonstrating yet another way AI is actively helping construct the computer hardware of tomorrow.

In an interview, Jensen Huang, chief executive officer of Nvidia, revealed last week during its quarterly earnings that while GPUs continue to command significant sway in the artificial intelligence market-holding their ground in the most demanding model training and some of the most compute-intensive parts of large model inference-custom silicon from cloud providers and chip manufacturers is picking up speed for others areas.

“There’s also… custom design ASICs that are in production from TSMC” for cloud providers. The company will also eventually leverage TSMC for mass production. In the latest deal with OpenAi, Broadcom chief executive officer, Hock Tan, in a press statement, said that “OpenAI is on the cutting edge of deploying large language models at scale to a massive user base…Our custom accelerator chip technology is designed to deliver the performance, efficiency, and cost advantages that leading innovators like OpenAI demand to realize their AI ambitions.

It will power some of the world’s most advanced AI inference applications. ”The manufacturing will be provided by Taiwan Semiconductor Manufacturing Company and system integration by Celestica. OpenAI has not been involved with any hardware designers, however they also announced a chip development business in September of previous year.

Other big business like Amazon, Apple, Google have been also increasing there silicon advancements for AI inference and data-center hardware applications.

The announcement has ramifications far beyond a simple procurement.

This isn't merely about cost savings on hardware. OpenAI is doubling down on what it calls a "full-stack" approach, encompassing everything from AI models to data centers, networking, software, and now, the fundamental chip architecture underpinning these operations. The ambition is to gain tight control over the entire technological pipeline, optimizing performance across the entire ecosystem while improving the economic efficiency of operating a massively scaled AI service.

Nvidia, while the dominant supplier of accelerators for AI training, is unlikely to be replaced for this purpose any time soon.

Training large language models still requires unprecedented computational power, an area where GPUs continue to excel. However, shifting a portion of its inference workload onto custom-designed hardware could lead to significant operating cost reductions and diminish its exposure to potential supply constraints impacting third-party hardware. Broadcom, meanwhile, benefits immensely from this collaboration. The semiconductor firm has firmly established itself as a key partner for companies developing custom AI silicon.

OpenAi's choice of Broadcom underscores the burgeoning demand for specialized hardware to power the AI era and solidifies Broadcom's position as a crucial enabler.

As AI adoption accelerates globally, the race for dominance is no longer confined to software and algorithms alone. Those who can efficiently construct and manage the underlying infrastructure may hold the key to future success. With the introduction of Jalapeo, OpenAI is clearly indicating its intent to compete in the foundational realm of hardware.