SGLang vs vLLM: H100 Benchmark Shows a 29% Throughput Edge

A benchmark gap that is hard to ignore

For most teams that self-host large language models, vLLM has been the default serving engine. A new performance comparison complicates that choice. According to a 2026 self-hosting guide, SGLang now delivers about 29% more throughput than vLLM on NVIDIA H100 GPUs, roughly 16,200 tokens per second versus 12,500 on standard workloads. 1 The same guide says the gap grows to as much as 6x on prefix-heavy retrieval-augmented generation (RAG) pipelines. It credits SGLang's RadixAttention, which reuses shared prompt prefixes instead of recomputing them. 1

The vLLM figure in that comparison is not a weak baseline. The guide says vLLM peaks around 12,500 tok/s on H100 with FlashInfer enabled. It also notes that vLLM's v0.6.0 release brought a 2.7x throughput gain and a 5x latency reduction over earlier versions. 1 SGLang is therefore being measured against a heavily optimized engine.

The 29% figure comes from a single guide, and the methodology behind "standard workloads" is not fully described. Treat it as a strong signal rather than a settled verdict.

Why vLLM still holds the default position

Every other account of the inference landscape still places vLLM at the center. One 2026 survey of open-source engines calls it the throughput-oriented default. Its author says that when a team builds a serving layer for a chatbot or RAG system on NVIDIA hardware, vLLM is almost always the pick. 2

The survey points to several reasons for that position:

• Origins and license: vLLM came out of UC Berkeley's Sky Computing Lab and is Apache-2.0 licensed. 2
• Memory management: It popularized PagedAttention for managing key-value cache memory. 2
• Features: It supports continuous batching and speculative decoding. 2
• Quantization: It covers a wide range of formats, including FP8, MXFP4, NVFP4, INT4, GPTQ/AWQ and GGUF. 2
• Compatibility: Its OpenAI-compatible API lets it drop into almost any client library. 2

The self-hosting guide agrees, calling vLLM the reference production engine with the broadest model support and the largest community. 1

The community is also visibly active. The project's own account of its 2026 Korea meetup describes field engineers from companies and research institutions sharing production deployment strategies. It presents vLLM as fast becoming foundational infrastructure across cloud and enterprise environments. 3 One enterprise session covered RAG-based agents with access-control structures that protect sensitive data while relying mostly on open-source tooling. 3

vLLM is also still adding architectural capabilities. An enterprise deployment guide highlights disaggregated prefill and decode, introduced in vLLM's 2025–2026 releases. This feature splits compute-heavy prompt processing from memory-bandwidth-bound token generation, so each phase can get different hardware or scheduling policies and GPU utilization improves. 4

Workload, not brand

The sources agree more than the headline numbers suggest. The self-hosting guide frames the choice as picking an engine for the workload rather than the brand. 1 The engine survey places SGLang alongside llama.cpp, Aphrodite, LMDeploy and LightLLM, each with its own strengths in hardware support, quantization and structured output. 2

The 6x RAG figure is the more consequential claim, but it applies to a specific pattern: many requests sharing long common prefixes, such as system prompts or retrieved document context. Prefix caching pays off heavily in that case. On workloads with little shared context, the advantage likely narrows toward the general-purpose 29% gap, or below it. That is an inference about how prefix reuse behaves, not a measured result.

The economics behind the engine race

Throughput matters because it directly sets cost per token, and that is what decides whether self-hosting makes sense. The self-hosting guide estimates that self-hosting on reserved GPU capacity breaks even with frontier APIs at around 2 million to 5 million tokens per day over 12 months. Below that volume, APIs still win. 1 The guide also argues that open-weight models such as DeepSeek V4, Llama 4, Qwen 3.5 and Gemma 4 now match or beat closed models on most non-reasoning tasks. 1

The enterprise guide adds a second motive beyond price. API costs scale linearly with usage, and every call sends potentially sensitive data through third-party infrastructure. 4 It argues the break-even point has fallen far enough that platform teams should seriously evaluate on-premises deployment. 4

A 29% throughput gain on the same hardware lowers that break-even point further. It does so without any new GPUs.

The takeaway

SGLang has not dethroned vLLM, but it has made the default choice worth testing. vLLM keeps the advantage in ecosystem breadth, quantization coverage, community and enterprise features such as disaggregated serving. SGLang's reported lead, especially on prefix-heavy RAG, makes it a serious candidate wherever throughput drives the budget.

For teams already running at the volumes where self-hosting pays off, the practical step is to benchmark both engines on their own traffic patterns.