The Collective
A private AI team that runs on one GPU: a Queen model, specialist models, local memory, retrieval, and construction-specific tools.
Off the Rails — 2026The Collective is our answer to a practical constraint: a construction company needs an AI that knows the job, the contract, the drawings, and the law, but it cannot depend on a metered cloud API or send private job data to one.
It runs on a single Windows PC with one GPU. A 30B Queen handles the hard reasoning and synthesis. Smaller specialists do the narrow work: Scout routes the request, Librarian retrieves and reranks, Mechanic runs math and deterministic tools, Archivist keeps memory, Scribe compresses context, Eyes reads images, Ears transcribes audio, and Painter generates images. The models are not a debating swarm; the coordinator is deterministic, and the system only loads what a request needs.
The construction build includes a local knowledge shelf with Oregon construction statutes and trade guidance, page-aware context, exact read-only tools for jobs, contracts, payments, tools, weather, and prices, and a public assistant that answers only from approved company facts. The portal assistant is scoped to the signed-in person; the public assistant never sees the database.
It is not frontier-capable, and it is not pretending to be. It is a local system that gets close enough to be useful without a cloud bill or a data-export conversation.
The brief
Cloud AI is expensive per token, dependent on connectivity, and difficult to justify when the context is a client's contract, a job cost, or a set of drawings. The goal was a useful assistant that could run locally on one GPU without making the desktop unusable.
The approach
Use a compound system instead of one giant model. A 30B mixture-of-experts Queen is loaded on the GPU for the difficult work. Small CPU models handle routing, compression, vision, and speech. Deterministic tools answer exact questions and can propose actions, but writes need human confirmation. Memory and retrieval are local. The context is assembled per person and per page, then capped so the model gets the useful part, not the whole database.
The outcome
A private AI system that runs on one machine, answers from the company's own records, and keeps the expensive model out of the way until it is needed. The wider product goal is a single-GPU AI that gets close to frontier usefulness without a cloud bill.
9
specialist roles
313k
local retrieval chunks
1
GPU, no cloud
Next project
ALTER
Have a brief like this?
Bring us the messy version. We are good with messy.