Strata runs Qwen3.8-Flash-Next on a normal PC. This is a large, smart AI model that usually needs a server. It chats, writes code, reads pictures and works with your apps and coding agents. Nothing leaves your PC.
How fast is it?
Measured on two ordinary gaming PCs. A token is about ¾ of a word.
NVIDIA: RTX 5070 (12 GB), Ryzen 5 7600, 64 GB RAM
| Size | Writes answers | Reads your prompt |
|---|---|---|
| Q2_0 | 94 tokens/s | 2,650 tokens/s |
| IQ2_XS | 79 tokens/s | 2,090 tokens/s |
| IQ3_XXS | 62 tokens/s | 1,750 tokens/s |
| IQ3_S | 53 tokens/s | 1,620 tokens/s |
| Coder | 55 tokens/s | 2,180 tokens/s |
AMD: RX 9070 XT (16 GB), Ryzen 9 3900X, 47 GB RAM
| Size | Writes answers | Reads your prompt |
|---|---|---|
| Q2_0 | 60 tokens/s | 1,160 tokens/s |
| IQ2_XS | 52 tokens/s | 1,110 tokens/s |
| Coder | 44 tokens/s | 1,420 tokens/s |
An RTX 3090 (24 GB) should write about 100-140 tokens per second.
What you need
| Graphics card | NVIDIA GeForce RTX 20, 30, 40 or 50 series, or AMD Radeon RX 7900 XT / XTX, RX 7800 XT / 7700 XT, RX 9060 XT, RX 9070 / 9070 XT, Radeon AI PRO R9700 or RX 6800 / 6900 series. 12 GB VRAM or more. |
| RAM | 32 GB or more. 64 GB runs every size. |
| Disk | About 80 GB free. SSD recommended. |
| System | Windows 10 / 11 or Linux, current graphics driver. |
Install
Download Strata and unzip (or git clone). Windows: double-click START-HERE.bat. Linux: run ./setup.sh. The installer finds your card, asks which model/size, context length, and image support, then downloads the model (~70 GB) and starts at http://127.0.0.1:8080.
AI coding assistants can set up Strata via docs/AI_SETUP.md. Strata also exposes an MCP server for agent automation.
Using it
- Browser:
http://127.0.0.1:8080— Chat, Monitor, About. - Apps and coding agents: OpenAI-compatible provider at
http://127.0.0.1:8080/v1. Anthropic API at/v1/messages. OpenAI Responses API at/v1/responses. - Thinking: off, low, medium or high reasoning effort.
- Pictures: optional image input (AMD Linux reads via CPU; Windows AMD not yet).
How does it work?
Models like this usually run on servers with hundreds of gigabytes of graphics memory. Strata makes the model fit by sharing work across the whole PC:
- The model has 24,576 small specialists (“experts”); each word needs only 10.
- GPU keeps frequently used experts; RAM holds all; CPU works on the rest; SSD holds lookup tables.
- A small helper guesses next words; the big model checks them all at once (1.6-1.8x faster).
- Long texts read in pieces up to 8,192 tokens at a time, over 1,000 tokens/second.
The model is Qwen3.8-Flash-Next by the Qwen team, compressed by ISTA-DASLab, UkisAI and Unsloth. Strata uses llama.cpp/ggml. MIT License.