Supermicro plans 10,000 SambaNova SN50 chips to run decode beside NVIDIA GPUs

Supermicro will build and sell inference systems that pair NVIDIA GPUs with SambaNova’s SN50 chips, starting with a plan for 10,000 SN50s, SambaNova announced on 3 November. The GPUs handle prefill, where the model reads the prompt. The SN50s handle decode, where it writes the answer one token at a time.

The deal is a new sales channel for an existing chip. What it lacks so far is any published number for the systems Supermicro will actually ship.

What the two companies agreed

Supermicro will integrate, test, deliver and support the racks, including on-site service. SambaNova engineers will tune models for SN50 and help with deployments. Sales start with neoclouds (cloud providers built around AI) and sovereign clouds, then move to companies running AI in their own data centers.

The 10,000 figure is a plan, not a disclosed order. StorageReview reports that neither company has disclosed a purchase commitment, pricing or a ship date. It adds that Supermicro has not said which GPUs or interconnects the systems will use.

What is inside an SN50

SN50 is SambaNova’s fifth generation RDU, or reconfigurable dataflow unit. It chains a model’s operations together instead of going back to memory between steps. Its product page lists 1,600 TFLOPS at BF16 and 3,200 at FP8 on a 5 nm TSMC process. Each chip has 432 MB of on-chip SRAM and 64 GB of HBM2E, the stacked memory that holds active model weights.

That memory layout is the argument. SambaNova calls decode memory-bound, meaning speed depends on how fast weights reach the compute units, so it leaves the compute-heavy prefill to GPUs. Our plain guide to what GPUs and NPUs each do for AI covers the general split.

Rack power, and the arithmetic

A SambaRack SN50 is air-cooled and draws roughly 20 kW, SambaNova says, and its benchmark posts describe a 16-chip rack. At that density, 10,000 SN50s would fill about 625 racks. At that average draw, our rough arithmetic gives around 12.5 MW for the RDU racks alone, before GPUs, networking and cooling.

What has actually been measured

SambaNova has published SN50 speeds, but not for Supermicro systems. At RAISE in July, four NVIDIA H200 GPUs ran prefill and one 16-chip SN50 rack ran decode on the MiniMax M2.7 model. SambaNova says Artificial Analysis measured up to 850 tokens per second on short contexts and over 450 on long ones.

A SambaNova post on a SemiAnalysis benchmark gives about 800 tokens per second with all 16 chips on one request. Splitting the rack into two 8-chip groups to serve more users gave about 400. SambaNova plots these as per-user speeds: how fast one answer streams, not how many users a rack serves.

We saw these figures only in SambaNova’s posts and could not check settings such as batch size. Results from different benchmark boards rarely use the same setup. SambaNova’s larger SN50 claim is still a target. In June it said SN50 with B300 GPUs aims for 500 tokens per second per user on MiniMax M2.7 and ten times the system throughput, without naming the baseline.

What to do this week

If you buy inference from a neocloud, ask whether it will add RDU capacity and what per-user speed it commits to. If you run your own GPUs, ask Supermicro for the GPU model, the GPU to RDU ratio and measured tokens per second at your context lengths.

Sources

• SambaNova: SN50 chips to power Supermicro’s new inference racks (3 November 2026)
• SambaNova: SN50 RDU product page and specifications
• SambaNova: SambaRack SN50 rack page
• SambaNova: SN50 MiniMax M2.7 results at RAISE (8 July 2026)
• SambaNova: SemiAnalysis benchmark of SambaRack SN50 (30 July 2026)
• SambaNova: first disaggregated inference demo (3 June 2026)
• StorageReview: Supermicro plans 10,000 SambaNova SN50 RDUs