The Beijing Academy of Artificial Intelligence (BAAI) has released Orca, a “world foundation model” that breaks from the dominant approach in today’s AI. Instead of predicting the next token, the next video frame, or the next robot action, Orca models the next state of the world in an abstract internal representation.
According to the technical report, intelligence shouldn’t be defined by specialized prediction models like language models, video generators, or robot controllers. What’s needed, the team argues, are models that build a general grasp of how the world changes and can use that as a base for many different tasks.
Orca builds an internal picture of the world from image and language signals. Separate, swappable add-on modules then turn that picture into text, images, or robot movements.
Two training methods that work together
Orca combines two learning modes. “Unconscious learning” uses raw videos without any captions. The model sees an image and predicts what the next one will look like, not at the pixel level, but in an abstract space, picking up motion patterns, occlusions, and typical scene dynamics along the way.
“Conscious learning” adds verbal instructions. Videos are split into segments, each labeled with a description of the state change, so the model learns how a state shifts when a specific action happens. It also trains on classic video question-answering tasks so it can still respond to natural language queries.
A frozen core with swappable output heads
The pre-trained language-image model Qwen3.5 serves as the base, and after training, this core stays frozen. For each output type, the researchers attach a separate, smaller module that converts the internal state into the desired form. Text runs through Qwen3.5’s existing language head. For images, Stable Diffusion 3.5 stays unchanged too, with only small upstream adapters trained to pass Orca’s internal state to the image generator. Robot actions come from a control module trained from scratch called “Action Expert.”
Scaling behavior and benchmark results
Orca was trained at two sizes, with 0.8 and 4 billion parameters. On text benchmarks, Orca-4B posts the best overall average of 51.8 percent across MVBench, TemporalBench, 3DSRBench, and SWITCH among the compared small VLMs and larger world models. It beats several VLM baselines like Qwen3.5-4B, Gemma 4-4B, and DeepSeek-VL2-3B on average. Orca-4B also tops the much larger world models Emu3 (8B) and Emu3.5 (34B) on average.
For image prediction, the researchers built their own benchmark called PRICE-V0.1, which uses real-world robot and first-person scenes and asks the model to generate an image showing the result of a command like “close the microwave.” Orca-4B hits 59.8 percent on average, beating specialized image generators like FLUX.2 small (56.1 percent), FLUX.1-context (40.9 percent), and OmniGen2 (39.6 percent).
Robot control without action data during pre-training
In five manipulation tasks using a two-armed humanoid robot on wheels, including shelving books, stacking bowls, and scooping sugar, Orca matches π0.5, a system built specifically on robot data. Orca’s base model never saw during pre-training which movement goes with which image. For actual robot control, a separate module was trained afterward on 200 real-world recordings per task, pairing camera images with the movements performed.
V-JEPA 2.1 and Qwen3.5, paired with the same control module as baselines, fall well behind. Orca also shows an edge in error recovery, retrying after a failed grasp while π0.5 stays stuck in repeated failures according to the examples in the paper. The authors see this as a potential way to ease robotics’ chronic shortage of labeled action data.
For training, the team uses its in-house FlagScale library with several memory and communication tweaks, reaching 2.91 training samples per second per GPU on H100 cards. That’s about 4.4 times faster than StarVLA, a codebase widely used in robotics.
Orca still only learns from images and text, with sound, force, and touch missing entirely. Visual prediction runs in the space of a pre-trained image encoder rather than learning its own world space from scratch, and at 0.8 and 4 billion parameters, the models are too small for full world modeling.
Training draws on 125,000 hours of video footage, 160 million event descriptions, and 11.5 million question-answer pairs.