SambaNova Cloud

SambaNova Cloud is best understood as the software face of a hardware company. The value proposition is not model access — the models it serves are open weights you can get elsewhere — it is the cost per token and per watt of serving them. The RDU uses a three-tier memory hierarchy and dataflow execution instead of the GPU's SIMT model, and SambaNova's claim is that this yields fast decode at meaningfully lower power. The numbers it publishes are specific and checkable in one case: MiniMax M2.7 decoding at up to 435 output tokens per second on SN50, gpt-oss-120b above 600 tokens/s, DeepSeek-V3.1 up to 200 tokens/s, with an independent SemiAnalysis benchmark covering the MiniMax case. SambaNova also says MiniMax M3 runs fastest on SambaCloud. Strengths. First, decode speed and energy efficiency are the two hardest things to fix in production inference, and those are exactly what the RDU is designed for. Second, the API surface is deliberately unambitious in the best way: OpenAI-compatible endpoints mean a one-line base-URL swap for most existing code, and support for the Anthropic Messages API means Claude-shaped applications port without rewriting message handling. Third, prompt caching for MiniMax M2.7 cuts cost and latency on repeated context — the pattern most agent loops actually exhibit. Fourth, SambaStack lets you switch between frontier-scale models on a single node, so a multi-model agent doesn't need a fleet of separate deployments. Fifth, for regulated buyers, sovereign deployments in Australia, Europe, the UK, and Japan (with TEPCO Systems) plus bring-your-own-checkpoints cover the data-residency and custom-weights requirements that rule out most US-hosted inference APIs. Weaknesses. The catalog is small on purpose. If your product depends on hot-swapping arbitrary open models, or on proprietary models not optimized for the RDU, this is the wrong platform — SambaNova's own framing is that frontier open-weight models are pre-optimized so you don't run a per-model engineering project. Second, throughput claims are vendor-stated for most models and tied to a specific model-hardware pairing; the SemiAnalysis result covers MiniMax M2.7 on SN50, not the whole catalog. Third, documentation of rate limits and context limits is not something this research pass could verify, so treat your own load test as the source of truth. Fourth, the training story is absent — this is an inference platform. Where it fits. Teams running sustained production inference on a named open model where decode latency and power cost drive the unit economics: customer-service agents, coding agents in IDEs, code generation and reasoning pipelines. Neocloud operators evaluating SN50 on the six-month payback claim. Governments and enterprises with residency mandates. Where it doesn't. Early-stage teams that want to try five models this afternoon. Products whose architecture assumes a broad, swappable model catalog. Anyone whose workload is dominated by prefill rather