Jalapeño is OpenAI’s first in-house AI chip, built with Broadcom to serve models rather than train them. Its main idea is simple: in AI inference, moving data costs more energy than doing maths, so the chip is designed to move as little data as possible. OpenAI says the result is 1.5 to 1.9 times more tokens per watt than Nvidia’s Blackwell systems, from a 700-watt part. The first chips go into service at the end of 2026 in very small volumes, with a wider rollout in 2027.
For organisations that buy AI rather than build chips, Jalapeño matters for a different reason. It shows where the cost of AI is heading, how a small team used AI to design hardware in nine months, and which number to watch when you compare AI services: tokens per watt.
What Jalapeño is and where it stands
OpenAI announced Jalapeño on 24 June 2026 as an “Intelligence Processor” for large language model inference. OpenAI designed the architecture, Broadcom handled the silicon implementation and networking, and Celestica builds boards and racks. It is the first product of the 10-gigawatt agreement the two companies signed in October 2025, to be completed by 2029.
At Hot Chips on 25 August, OpenAI shared the specifications, summarised by ServeTheHome and Tom’s Hardware:
- 13.4 PFLOPS of MXFP4 compute and 216 GiB of HBM4 memory at 15.4 TB/s per chip
- 700 W package power, with measured draw at or below 550 W on the tested workloads, according to DatacenterDynamics
- 64 core slices, each paired with its own slice of HBM
- 128 chips in a local domain, up to 2,048 in a pod of 16 racks, linked over Ethernet with Broadcom Tomahawk 6 switches
The chips are not for sale for now. OpenAI uses them internally for ChatGPT, Codex and its API and says its demand for compute is huge, though it has left the door open to a broader rollout. In the racks they run with AMD EPYC Turin processors as hosts rather than Nvidia’s Vera CPU, which hardware chief Richard Ho told Tom’s Hardware was a pragmatic choice.
The design bet is about data movement
On paper, Jalapeño’s peak numbers are not exceptional. Nvidia’s Blackwell Ultra delivers more raw FLOPS. The difference, Ho argues, is how much of that peak a chip can use in real serving. In a 22 October interview with Tae Kim, he put it in one line: “Energy efficiency boils down to data movement and the reduction of data movement.”
Three choices follow from that:
- Each core gets its own memory. Every one of the 64 core slices is wired to its own part of the HBM4 memory and works mostly with that local slice. The chip does not have to keep a shared memory view consistent across all cores, which saves energy. The cost is a harder programming model, because software must decide where data lives. OpenAI bet that AI models would learn to do that placement work, and Ho says they did.
- One chip does every phase. Many inference setups split prefill, where the model reads the prompt, from decode, where it writes the answer, across different groups of chips. The model’s working state, the KV cache, then has to travel between them. Jalapeño runs all phases on the same chip, keeps the KV cache local and powers down blocks that are idle in a given phase. Ho also likes the flexibility: with split hardware, “you have to commit ahead of time to how much of your hardware is going to be on prefill, how much on decode.”
- HBM rather than on-chip SRAM. Some inference chips rely on very fast on-chip SRAM. “You get a lot of bandwidth out of SRAM, but the capacity is so limited”, Ho says. OpenAI’s models are large, so capacity won.
How to read the benchmark numbers
OpenAI tested Jalapeño on InferenceX, SemiAnalysis’ open benchmark that measures how many tokens a system delivers at a given latency. Results were normalised by package power: 700 W for Jalapeño, 1,200 W for an Nvidia GB200 and 1,400 W for a GB300.
- GPT-OSS 120B versus GB200: about 1.9 times more tokens per kilowatt, 1.7 times lower end-to-end latency.
- DeepSeek R1 versus GB300: about 1.7 times more tokens per kilowatt, 3.6 times lower latency.
- Kimi K2.5 versus GB300: about 1.5 times more tokens per kilowatt, 3.4 times lower latency.
Four caveats belong next to those figures. OpenAI ran the tests itself, and none of the coverage, including TechCrunch’s, mentions independent verification. Per watt is not per chip: a Nvidia system can still be faster in absolute terms. Settings differed, since the Nvidia systems used multi-token prediction and Jalapeño did not, and headlines about “up to 104 times” come from a single operating point that favours Jalapeño. Finally, Nvidia’s next generation, Vera Rubin, has been in full production since June and is now shipping to its first customers, and the ranking may shift once it is tested. OpenAI published no cost per token or total cost of ownership.
How a team of about 100 people took a chip to tape-out in nine months
The design went from the first RTL code in February 2025 to tape-out in November 2025, roughly nine months. First silicon came back in May 2026, and Codex ran on it the same month. For a new high-performance ASIC, that is fast.
Ho credits two things. The first is AI in the design loop. More than half of the core was written in XLS, a hardware language close to software that models handle well. AI agents searched for better trade-offs between power, performance and area: “an AI can parallelize tens of thousands of runs.” OpenAI reports a BF16 multiplier 56% better than the human baseline and AI-optimised kernels that run 1.5 to 1.8 times faster than expert versions. The second is iteration speed. “A team that can iterate in an hour will outpace a team that takes a week to iterate”, Ho told Tae Kim.
The team is small. “The team is about 100 people, but those 100 people cover everything”, from chip design and verification to packaging, thermals, racks, supply chain and software. And it does not trust the machine blindly. Ho told Ian Cutress of More Than Moore, as reported by The Next Web, that he still doesn’t “100% trust” the models. In one case they saved more than 13% of die area with changes engineers could not explain, so every change goes through full validation.
That combination is the transferable lesson. AI shortened the search, people kept the sign-off, and a short feedback loop did the rest. It applies to software, product and process work as much as to chips.
Why OpenAI still buys Nvidia and AMD
Jalapeño does not end OpenAI’s dependence on Nvidia. Training runs almost entirely on Nvidia GPUs, and for inference OpenAI keeps buying from several suppliers. “It’s a capacity thing. We have so much demand for compute”, Ho says. He even wants GPUs to improve, because OpenAI still needs them.
The roadmap is taking shape. Gen 2 is in the lab for qualification and targets better performance per watt, Gen 3 targets low-latency serving at low cost, and Ho says work on a fourth generation has started. He won’t promise a fixed rhythm: “I wouldn’t call it annual. It might be pragmatically close to annual.” Every new generation needs a large step up, because rolling out new hardware in data centres is expensive.
The pattern is familiar. Google has its TPU 8t and 8i, Amazon has Trainium and Meta has MTIA. Start-ups such as Tenstorrent and Cerebras sell alternatives. Jalapeño adds one of the largest buyers of AI compute to the list of companies that design their own inference silicon.
What this means for European organisations
You can’t buy Jalapeño, and you’ll only meet it through OpenAI’s services. It still changes a few things for organisations that use or procure AI.
- Power is the binding constraint. A data centre has a fixed grid connection. A chip that produces the same tokens with half the watts can almost double the answers that site delivers. In Europe, where new grid capacity is slow and expensive, that matters more than raw speed. It is also why sovereign AI plans depend as much on energy as on models.
- Energy reporting is coming to AI infrastructure. Under the Energy Efficiency Directive, data centres with at least 500 kW of IT power already report energy and water data to an EU database. On 21 September 2026 the European Commission proposed a common rating scheme for data centres, and a legislative proposal for minimum performance standards is planned for the second quarter of 2027. The EU AI Act also asks providers of general-purpose AI models to document known or estimated energy consumption. Tokens per watt is becoming a number vendors will have to explain.
- Ask your AI vendors about efficiency, not just price. When you compare model providers or private hosting options, ask which hardware serves your traffic, what the energy per million tokens is, and how that will change as new chips arrive. The answers say something about future pricing and about your own sustainability reporting.
- Don’t assume prices will follow automatically. OpenAI has not said whether Jalapeño will make its API cheaper. Lower serving costs give it room, but demand is high and the first volumes are small.
Frequently asked questions
Can companies buy or rent Jalapeño?
Not for now. OpenAI uses the chips for its own services. Customers benefit indirectly, through capacity, latency and possibly price.
Is Jalapeño faster than an Nvidia GPU?
Per watt, according to OpenAI’s own InferenceX results, yes: 1.5 to 1.9 times more tokens per kilowatt than GB200 and GB300. Per chip in absolute terms, not necessarily.
Does OpenAI train models on Jalapeño?
No. The chip is built for inference and computes mainly in low-precision MXFP4. Training happens almost entirely on Nvidia GPUs.
When will ChatGPT run on Jalapeño?
Ho expects deployment at the end of 2026 “in very small volumes”, with more significant deployment in 2027.
The number to watch is tokens per watt
For two years, AI hardware was compared on raw compute. Jalapeño makes a different argument: a modest chip that wastes less energy on moving data can beat a stronger one where it counts, in answers delivered per unit of power. If OpenAI’s numbers hold up against Vera Rubin in 2027, expect every AI supplier to start quoting tokens per watt. European buyers, facing grid limits and new reporting rules, have good reason to ask for that number now.
