Definition
Orca is a general world foundation model from baai that learns a unified world latent space via Next-State-Prediction rather than isolated next-token, next-frame, or next-action objectives. Downstream readouts expose text generation, image prediction, and embodied robot control from a frozen backbone.
Key Points
- Architecture: Frozen qwen3.5 VLM encoder; unconscious learning (continuous video) + conscious learning (language-described events, VQA)
- Pre-training scale: 125K hours video, 160M event annotations
- Readout heads: Qwen3.5 language head (text); Stable Diffusion 3.5 adapters (image); Action Expert module (robotics, trained from scratch)
- Robotics claim: Orca-4B matches Physical Intelligence π0.5 across five manipulation tasks despite zero action labels during base pre-training; only 200 real-world recordings per task for Action Expert fine-tuning
- Error recovery: Paper examples show Orca retrying failed grasps where π0.5 repeats failures
- Project: orca-wm.github.io; arXiv 2606.30534
Related
- world-models
- baai
- physical-ai
- robotics
- sim-to-real
- chinese-ai
- vision-language-models
- reinforcement-learning