Definition
System design that separates transformer inference into prefill (prompt processing) and decode (token generation) stages, sometimes with specialized silicon for each.
Key Points
- Etched markets custom prefill (low-voltage inference) and decode (cluster-scale memory) systems
- Claims flexibility across MoE and Mamba architectures
- Part of broader inference-ASIC challenge to GPU monoculture