Skip to content
LATEST
Technology

Cerebras Deploys Disaggregated Inference to Increase AI Throughput Efficiency

Cerebras launched a disaggregated inference architecture using AMD GPUs and wafer-scale systems, achieving a 5x increase in power efficiency.

Cerebras Deploys Disaggregated Inference to Increase AI Throughput Efficiency

Cerebras Systems has transitioned its disaggregated inference architecture into production, reporting a fivefold increase in throughput per megawatt. By offloading prompt processing (prefill) to AMD Instinct GPUs and reserving its wafer-scale CS-3 systems for token generation (decode), the company has optimized power efficiency without compromising response latency. This production deployment follows initial testing conducted in October 2026.

The architecture addresses the distinct computational requirements of large language model inference. Prefill operations are compute-intensive and well-suited for standard GPU architectures, whereas decode operations are memory-bound and benefit from the high-bandwidth, on-chip SRAM capacity of wafer-scale hardware. By separating these stages, operators can maximize the utility of their existing power budgets in constrained data center environments.

To facilitate broader adoption, the company has released a software stack and a decode-only SDK. These tools enable infrastructure partners to integrate their own prefill engines—such as vLLM or SGLang—with Cerebras’ decode engine via an RDMA-based KV Portal. This interface manages the transfer of intermediate model states and key-value caches between the two hardware pools.

Early-access partners include Amazon Web Services, Fireworks AI, Gimlet Labs, and General Compute. The company has set a strategic target to scale this disaggregated inference model across hundreds of megawatts of capacity within its own cloud and partner deployments by the end of 2027.

While the reported performance gains are significant, the scalability of this model remains subject to operational variables. Future performance will depend on the consistency of throughput across diverse partner-selected accelerators, varied software frameworks, and large-scale cluster deployments. The transition from a single shared endpoint to widespread infrastructure integration will be the primary indicator of the architecture's long-term viability in commercial data centers.

More stories

Latest news