At Cloudflare’s scale, small improvements are greatly magnified. Engineers optimized the pingora-ketama consistent hashing library used by Pingora Backend Router (PBR), reclaiming more than 100TB of RAM globally on top of 100TB shed by the DNS team the prior month.
The investigation found PBR using significantly more memory than expected in structures associated with pingora-ketama, Cloudflare’s open-source consistent hashing library. In some cases, a single service used 6GB of memory for hash rings.
Key optimizations:
- Compacted Point struct from 8 bytes to 6 bytes by using u16 for server index instead of u32
- Reduced hashes per server by 90% after mathematical analysis showed diminishing returns beyond ~10,000 hashes per server at 2048-server data centers
- Gradual migration using dual-ring deployment to avoid cache invalidation across the global network
The coefficient of variation for workload distribution dropped from ~99% (single hash) to ~8% (160 hashes) to acceptable levels with fewer hashes once collision effects were modeled.
Changes are available in pingora-ketama via an unadvertised v2 cargo feature with backward-compatible v1 rings.