OpenAI and Broadcom announced Jalapeño, OpenAI's first custom inference chip, on June 24, 2026. The companies claim it can deliver roughly 50% lower cost per inference token than current-generation NVIDIA GPUs. Small-scale prototype deployments are planned before the end of 2026.
That claim addresses a substantial production expense. Every generated token consumes computing capacity, and serving models to millions of users requires a large, continuously operating hardware fleet. NVIDIA accelerators, including H100s and B200s, dominate that infrastructure. Their cost shows up in cloud bills and in the expense of running AI services.
Jalapeño gives OpenAI a way to reduce its dependence on that hardware for inference, the process of running a trained model to produce an answer. The announced design uses TSMC's 3nm manufacturing process and has reached tape-out, the point when a chip design is finalized for manufacturing. Whether the claimed savings survive production conditions remains the main question.
A chip built around inference
Jalapeño is an application-specific integrated circuit, or ASIC. Unlike a general-purpose GPU, it is designed for a narrower workload: large language model inference.
GPUs can handle many kinds of parallel computing, from graphics to scientific simulation and AI. That flexibility is valuable, but a chip dedicated to transformer inference can make different tradeoffs. Its designers can focus on the operations that dominate serving costs, including memory movement around attention computations, frequently used computational kernels, and communication between chips.
The announced architecture centers on a systolic array, an arrangement of computing elements that passes data through a regular pattern of arithmetic operations. Google's TPUs use a similar approach. Jalapeño also includes eight HBM, or High Bandwidth Memory, stacks, described as packaged directly on the die.
Keeping HBM close to the computing elements, rather than fetching data through system memory, reduces latency and helps keep the chip supplied with data. The design's premise is that memory bandwidth, rather than raw floating-point computing capacity, is the dominant constraint in LLM serving. A purpose-built accelerator can tune its balance of memory bandwidth and computation around that constraint.
The reported TSMC 3nm process places Jalapeño on a leading-edge manufacturing node rather than an older process chosen mainly to reduce fabrication costs. Coverage places it in the same lithography generation as Apple's M4 and NVIDIA's Blackwell Ultra. The proposed savings therefore depend on specialization and system efficiency, not simply cheaper manufacturing.
A short development schedule
OpenAI and Broadcom announced their partnership in October 2025 and unveiled Jalapeño in June 2026, about eight months later. The development schedule is described as nine months from partnership announcement to tape-out. Broadcom calls it one of the fastest ASIC development cycles achieved at the high-performance end of advanced semiconductor design.
A well-understood workload helps explain how such a schedule could be possible. Inference offers a relatively stable computation graph and a known set of dominant kernels. OpenAI has also spent years running GPT-class models at large scale, giving it detailed information about where its serving systems stall and what consumes the most resources.
That knowledge can reduce uncertainty in chip design. Engineers still have to solve the physical implementation problems, but they have a clearer specification for what the hardware needs to do. In the joint announcement, OpenAI president Greg Brockman emphasized the company's deep understanding of its workload as the basis for designing an accelerator that could expand what its systems can do.
A fast tape-out is encouraging, but it doesn't establish production readiness. The year-end prototypes will be proofs of concept rather than replacements for OpenAI's existing accelerator fleet. As of June 26, 2026, a full production ramp is still estimated to be 12 to 18 months away.
More control over the serving stack
Custom silicon extends OpenAI's control over inference beyond model architecture and serving software. It adds the accelerator to a system that also includes networking and workload scheduling. Those parts can be designed together rather than fitted around hardware built for a broader market.
Other large AI operators have taken similar paths. Google has used TPUs since 2016. Amazon built Trainium for training and Inferentia for inference. Meta has its MTIA chips. The economic argument is straightforward: when AI computation accounts for a large share of operating costs, owning more of the hardware design can be worth the expense.
OpenAI differs from those companies because it started as a model developer and became an infrastructure operator out of necessity. It doesn't sell general-purpose compute as AWS does. Yet serving tens of millions of users and real-time API traffic from tens of thousands of enterprise customers gives it a substantial infrastructure bill. A 50% reduction in inference cost, if achieved at that scale, would materially change the economics of its services.
Jalapeño is inference-only and won't run pre-training. The argument for that boundary is that frontier-model training involves more exploratory and irregular computing requirements, which fit less comfortably into a fixed ASIC design. NVIDIA retains that workload for now. Once a model version has been trained, however, it can be used for inference billions of times per day. That repeated production workload is the target for Jalapeño's savings.
Lower internal costs don't set API prices
A cheaper token for OpenAI to generate doesn't automatically mean a cheaper token for an API customer to buy. Even if Jalapeño meets its target, OpenAI could use the savings to fund model development, expand capacity, or compete on capability before reducing prices.
Lower computing costs may still push API prices down over time. They reduce the cost floor for serving models, and competing providers have reasons to pursue similar savings. If Anthropic and Google see comparable reductions through their custom-silicon programs, competition could force some of the benefit through to customers.
API costs have already fallen by an order of magnitude over three years. Custom accelerators could support further declines, though the timing and share passed to customers remain uncertain. A 50% hardware-level saving should not be treated as a forecast of a 50% API price cut.
For organizations with substantial OpenAI API spending, Jalapeño is more useful as a signal about the 2027 to 2028 cost environment than as a reason to renegotiate a current contract. Prototype deployments and a production fleet have very different effects on a provider's total costs.
Longer-term architecture planning can account for that possibility without depending on it. Over a three-year horizon, cost-based throttling, caching, and routing requests between model tiers may need to be reassessed if inference becomes much cheaper. Plans can include a scenario in which costs fall by half while keeping today's systems viable at current prices.
Broadcom's role and the alternatives to NVIDIA
Broadcom reported that its AI revenue doubled to $8.4 billion in Q1 2026, driven by custom ASIC work for OpenAI, Anthropic, Meta, and others. Its role gives model companies a route to custom hardware without requiring them to build every part of a semiconductor design operation themselves.
The division of work is clear: the model company supplies workload specifications, Broadcom handles physical design, and TSMC manufactures the chips. That arrangement is likely to recur as more companies look for hardware tuned to their own serving requirements.
NVIDIA still has substantial advantages. CUDA has a decade of momentum, its software stack is strong, and it remains the only practical option for training frontier models for now. Its position in inference faces more competition. AMD is making progress with ROCm, Google's TPUs are available to external customers, and Amazon's Inferentia is already in wide use. Jalapeño adds another custom design to that field, although its immediate purpose is OpenAI's own infrastructure.
For infrastructure teams, a broader range of accelerators creates both options and work. Comparing hardware means evaluating cost profiles and software support, and some deployments may need serving systems that support more than one accelerator type. Where alternatives are available, teams gain more room to negotiate and assign workloads to the hardware that runs them most economically.
The year-end Jalapeño prototypes should begin to show whether the design's advantages hold under real serving load. Power consumption per token, production failure rates, and memory bandwidth utilization will be more informative than the tape-out schedule alone. Those operational results will determine how much of the claimed cost reduction OpenAI can sustain as deployment grows.
