Definition
Running trained language models to generate outputs — encompassing prefill (processing input) and decode (generating tokens) phases with associated compute, memory, and cost tradeoffs.
Key Points
- 2026-09-17: deepseek-v41-flash targets input-heavy agent workloads where kv-cache dominates HBM/SSD costs (2026-09-17-deepseek-v41-flash-huggingface)
- Cache-hit charges are major agent deployment cost driver; compression directly affects self-hosted economics
- Prefill vs decode activation asymmetry (8B/16B in CED) optimizes agent cost shape