Overview
SGLang is an open-source LLM inference engine used for high-throughput serving, including disaggregated prefill/decode deployments. cloudflare Workers AI runs production models on SGLang.
Recent Developments
- 2026-10-26: Coordinated disclosure with Cloudflare on cross-request KV cache exposure when over-length prompts were rejected but decode continued with stale cache; upstream patches and pool-sizing hardening (2026-10-26-cloudflare-glm-kv-cache-exposure)