This page may contain stale information. Last updated: 2026-07-05
Definition
MirrorCode is a long-horizon coding benchmark co-developed by epoch-ai and metr. Agents must reimplement complete software programs end-to-end without source code access, matching original program output exactly on held-out tests.
Methodology
| Element | Detail |
|---|---|
| Input | Execute-only black-box binary + checkable specification |
| Output | Full program reimplementation |
| Validation | Exact output matching on end-to-end tests including private held-out suite |
| Programs | 25 targets: Unix utilities, bioinformatics, interpreters, static analysis, crypto, compression |
Key Results (June 2026)
- gotree: ~16,900 LoC Go bioinformatics toolkit, 40+ commands — Opus 4.7 solved in 14h ($251); human estimate 2–17 weeks
- Pkl: ~61K LoC Apple configuration language — Opus 4.7 near-perfect (99%+ tests)
- choose, cal: Solved in early results
- 17/25 programs had at least one perfect-scoring run; 4 more scored >99%
- Inference cost: Up to $2,600 over 19 days for single large-task attempt
Early April 2026 results showed Pkl unsolved at 1B token budget; full June release documented near-perfect solve — present capability bounds honestly when citing preliminary vs final results.