OpenAI’s Jalapeño chip cuts inference costs

OpenAI’s first custom chip, Jalapeño, is built to make inference faster and cheaper.

OpenAI’s first custom chip is a sign that AI infrastructure is moving deeper into custom silicon, with early tests showing better performance-per-watt than current alternatives.

1. Jalapeño

OpenAI’s new processor, Jalapeño, is its first custom-built inference chip and was designed with Broadcom for the company’s own workload needs. It is not a general-purpose AI chip. It is aimed at the specific job of running pre-built models after training is done.

That focus matters because inference is where user requests turn into answers, code, and agent actions. OpenAI said early testing points to lower operating cost and better performance-per-watt than current state-of-the-art alternatives.

• Designed for inference, not pre-training
• Built for real-time coding models
• Still in testing
• Targets lower power use per unit of work

2. Broadcom

Broadcom is the manufacturing and design partner behind the chip, which puts one of the biggest networking and semiconductor vendors into a more direct role in OpenAI’s stack. The partnership was first announced in October, but this is the first public look at the result.

For OpenAI, the appeal is control. Working with Broadcom gives the company a path to tailor hardware around its own model behavior instead of adapting models to generic chips. That can matter when small efficiency gains turn into large savings at scale.

• Partnership announced in October
• Chip is custom-built for OpenAI workloads
• Part of a broader move into purpose-built AI silicon

3. Inference-first design

Jalapeño is built for inference, the phase where a trained model responds to prompts. That makes it different from the chips used for pre-training, which usually demand much heavier compute and memory bandwidth.

OpenAI said the chip is especially aimed at low operating cost for real-time coding models. That suggests the company is trying to trim expenses where usage is constant and user-facing, not just where the biggest training runs happen.

4. OpenAI’s full-stack approach

OpenAI says it is designing more than models and products. It is also shaping the infrastructure underneath them, including chip architecture, kernels, memory systems, networking, scheduling, deployment systems, and product experience.

That full-stack approach gives the company more knobs to turn when optimizing for speed, reliability, and cost. It also helps explain why custom silicon is attractive: if one company controls the model, the software, and the hardware, it can tune all three around the same workload.

• Chip architecture
• Memory systems
• Networking and scheduling
• Deployment systems

5. Why this matters for Nvidia

OpenAI has long been seen as dependent on Nvidia GPUs, and Jalapeño is part of the effort to reduce that reliance. The chip will not replace Nvidia across the board, especially for more compute-heavy pre-training jobs, but it could chip away at the cost of serving everyday traffic.

That is the real business story here. Even modest savings on inference can improve margins for products like Codex and other agentic tools, where usage may scale quickly and continuously.

How to decide

If you care about AI product economics, Jalapeño is the most important part of this story because it shows where OpenAI sees the biggest cost pressure: inference. If you care about semiconductor strategy, the Broadcom partnership is the key signal because it shows OpenAI moving from buyer to co-designer.

If you track the AI chip market, the main takeaway is simple: the next fight is no longer only about training giant models. It is also about who can run those models more cheaply, more reliably, and with less power.