Work / Local-first AI

Stem: A Local-First AI System With Zero Confirmed Leaks Across 19,124 Production Events

A daily AI practice that mixed sensitive documents with ordinary questions defaulted every request to the same cloud model, or slowed down to sort them by hand. A fail-to-local router and a deterministic redaction gate now sit as the one code path a request has to clear before it can reach a cloud model, checked against a 96/96 self-test gate.

The product page shows Stem running.

Anonymized. Client, employer, project, manufacturer, part, and pricing identifiers are removed; the mechanism and the numbers are real.

The user and their week

The user is one person, running sensitive, document-heavy work daily: notes, transcripts, documents, reference libraries, and a growing trail of prior decisions, scattered, and some of it not safe to hand to a cloud model. Every day mixed plain general-knowledge questions with proprietary and confidential material of his own.

Before a router existed, every request faced the same expensive choice: run it through the easy default, a frontier cloud model, and accept the risk that one careless prompt leaks sensitive material to a third-party API, or stop and manually decide, case by case, what was safe to send off the machine. That manual sorting cost real hours on its own: re-explaining context every session, re-deciding questions that were already settled. Across the life of the system, one demanding daily user has put 30,230 stored turns through it so far.

Why it mattered

Treating every request the same way had two costly failure modes: overpay for cloud inference on questions a cheap local model handles fine, or build a system where one careless prompt leaks sensitive data to a third party. The system had to solve five connected problems at once: keep sensitive information local by default, search and reason across a large fragmented knowledge base, preserve context across sessions without refilling every prompt with history, execute multi-step tasks reliably across different models and machines, and improve over time so the same failures stop repeating. The requirement was an operating environment around the model, memory, control, evidence, and learning, with private data confined to controlled infrastructure.

The bet, and what was rejected

The product bet, in its plainest form: sensitive work should remain private without giving up access to frontier AI. Two easier paths were rejected on purpose. A single cloud subscription would have been simpler to run and would have given every task access to the strongest available reasoning by default; that convenience was given up. A policy document telling the system not to send sensitive data to the cloud was rejected too, on the grounds that a policy gets read once and violated under deadline pressure.

What replaced both: a tiered architecture, sensitive inference kept on controlled local infrastructure, cloud models used only for tasks carrying no private content or under tightly controlled read-only review, with the boundary enforced through code paths, exercised by dedicated privacy and sovereignty tests.

Acceptance criteria, set before build

What the sources do show is the standard the system was actually held to. On routing: any low classifier confidence, any classifier error, any attachment present, or any ambiguity at all defaults to local, never to cloud, and confidence has to clear a deliberately high bar before a request is ever allowed out. On shipping: nothing ships unless the full self-test suite passes; the gate stands at 96/96 today, and the suite only grows stricter, never looser. On the newest safeguard, a detector for degenerating output: it had to catch every real repetition loop without flagging a single healthy turn, checked against the full stored history, 30,230 turns, three loops caught, zero healthy turns flagged.

What shipped

A router that classifies every incoming request, before it goes anywhere, as general-knowledge or sensitive/proprietary; the classifier runs locally, on the request itself, so nothing crosses the boundary just to decide whether it should.
A deterministic redaction gate as the single code path in the system allowed to emit anything externally: every field in every outbound payload passes an allowlist, anything unrecognized gets tagged and stripped, and the gate runs a self-test on its own output before emitting, refusing to emit at all if any substantive source content survived.
A human approval gate, named plainly: the system drafts, builds, and reconciles; a person approves anything that ships, spends, or leaves the machine. Nothing that touches money, external delivery, or an outbound request passes without that approval.

Every request in this system passes through one gate before it can leave the machine. If the classifier is unsure, the request stays local. Nothing skips the line.

What failed, and what changed

Two production incidents traced to the same root cause: infrastructure edges the system had never been tested against under real conditions. First, a nightly evaluation process kept generating reports that a silent downstream failure left unread; review freshness was added after the fact, once the gap surfaced. Second, a shell-specific argument-handling issue stripped quotation marks during model handoff; the fix moved prompt delivery from command arguments to standard input.

Each incident became a permanent regression test before its fix shipped, per the standing rule that a failure has to convert into a test before it counts as closed.

Result

The audited number is the router's: zero confirmed leaks, validated against the full production transcript history, 1,821 transcripts and 19,124 individual events. Escalation runs through exactly one code path with no parallel route around it, so there is nothing else to audit; the gate either holds for every request or it does not run at all.

The self-test gate that has to pass before anything ships stands at 96/96. The degeneration detector, checked against the same discipline, ran against 30,230 stored turns and caught three real loops while flagging zero healthy turns.

The honest cost of this result: erring conservative on every borderline case means real cloud-tier speed and quality get left on the table any time the classifier is not confident enough to escalate. That is the accepted price of a boundary with no bypass to audit.

Adoption, and what carries forward

The adoption evidence on this page is one person: the builder, running the system daily against work he could not risk handing to a cloud model outright, across the same 30,230 stored turns cited above.

What carries to the next engagement is the pattern: classify before routing, using a model cheap and local enough that classification itself never becomes a new leak surface; default to the safe side on any doubt; enforce the boundary in exactly one code path, with a fail-closed allowlist and a self-check, so the guarantee never depends on a future contributor remembering a rule. Engineering and construction organizations carry the same fragmented-knowledge, unclear-boundary problem this system was built to solve; the transferable version connects fragmented records with provenance intact, defines the data and approval boundary in code, and turns incidents into regression tests and standards, cutting repeat costs, while the people with technical and commercial authority keep the approval.