Anonymized portfolio piece. This describes the personal AI operating platform I designed, built, and run daily. Two of its subsystems already have their own case studies on this site: the hybrid local/cloud router with a code-enforced data-sovereignty gate and the concurrent multi-agent orchestrator. This case covers the operating system around them. All identifying detail about what the system is used for is removed; the architecture, mechanism, and validation figures are real.
The problem
Most AI tools are optimized for isolated conversations. They do not reliably preserve operational knowledge, distinguish sensitive information from general questions, complete work that outlives one context window, or convert failures into permanent safeguards. That is a poor fit for sensitive, document-heavy work, where the useful knowledge lives scattered across notes, transcripts, documents, reference libraries, and prior decisions.
The system needed to solve five connected problems:
- Keep sensitive information local by default.
- Search and reason across a large, fragmented knowledge base.
- Preserve useful context across sessions without filling every prompt with history.
- Execute multi-step tasks reliably across different models and machines.
- Improve over time instead of repeating the same failures.
The requirement was an operating environment around the model: memory, control, evidence, and learning, with private data confined to controlled infrastructure.
What I built
The architecture separates work across three tiers. A primary workstation handles sensitive inference, document processing, speech services, scheduled tasks, and agent execution. A second local machine provides embeddings, intent classification, and low-latency voice inference. Selected cloud models are used only for tasks that carry no private content, or for tightly controlled planning and review workflows with restricted, read-only file access. This separation lets the system use stronger external reasoning when appropriate without treating every model or task as equally trusted.
On top of that foundation the system operates as one connected environment:
- Private routing. Every request is classified before execution. Sensitive or recalled personal information routes to local models; external knowledge questions may use cloud models when the content is safe to send. Classification uncertainty and routing errors fail toward local execution.
- Large-scale retrieval. More than 67,000 indexed knowledge chunks across 19 sources, with scoped search, deduplication, incremental reindexing, offline-source tolerance, stale-document annotation, and provenance-aware results.
- Persistent memory. Completed sessions are distilled into structured memory. Explicit and automatic recall inject only relevant prior context, and a recalled private item forces the entire request back onto local infrastructure.
- Task execution. Durable queues, retry and blocked states, multi-agent decomposition, parallel research, compact-and-continue for work that outlives one context window, and structured delegation and synthesis.
- Local voice. A complete local speech-to-speech path reaching approximately 115 milliseconds to first content when warm, plus a separate desktop dictation application, with audio processing and inference kept on local hardware.
Privacy enforced in code
Privacy is enforced through code paths rather than written policy alone. Personal content cannot cross the external-model boundary unless it has been explicitly approved and transformed into an allowed format: synthetic fixtures, redacted structural examples, or aggregate telemetry. Cloud models may review process and architecture by default; reviewing sensitive content requires explicit approval. The boundary is exercised by 41 dedicated privacy and sovereignty tests, so the rules exist as verified code paths instead of documentation. The enforcement mechanism itself is covered in the data-sovereignty case study.
A second frontier model serves as an independent planning and review layer, operating through a read-only interface with schema-controlled output and file-level citations. Its recommendations are adjudicated by the primary orchestrator before anything is acted on. In initial production exercises, one review achieved 7 of 7 precision with zero false positives, and a planning exercise then surfaced a real schema incompatibility in the review interface itself, which was corrected and re-tested before the interface was expanded. The reviewer found a flaw in the review system, which is exactly the failure class it exists to catch.
Failures converted into safeguards
A central design requirement was that failures become permanent system improvements. Three production incidents show the pattern:
- Retrieval-path failure. A platform-specific path-format inconsistency caused deep scoped searches to silently return nothing. The indexer was corrected at the source, all 67,394 existing chunks were migrated in place, and a permanent regression test now prevents the failure class from returning unnoticed.
- Review-consumption failure. A nightly evaluation process was generating reports that a silent downstream failure left unread. The pipeline gained same-night retry, freshness monitoring, and stale-path checks, and report freshness became part of the system self-test.
- Command-interface corruption. A shell-specific argument-handling issue stripped quotation marks during model handoff. Prompt delivery moved from command arguments to standard input and was re-verified through the full review path.
Each meaningful failure or lesson also becomes a regression case, so the test suite only tightens: 93 of 93 system self-tests, 104 of 104 retrieval cases, and 54 of 54 memory and recall cases currently pass. Settled architecture and operating decisions are recorded and injected into future sessions; the decision log holds more than 210 numbered decisions, so previous reasoning is preserved instead of re-argued.
The result
The current system runs more than 67,000 indexed knowledge chunks across 19 sources, 41 dedicated privacy tests, 251 passing evaluation cases across self-test, retrieval, and recall, 43 historical sessions backfilled into structured memory, more than 210 recorded operating decisions, and a warm local voice path at approximately 115 milliseconds to first content. Task queues, retrieval, memory, voice, and multi-model review workflows are all production-tested, and every one of the incidents above now exists as a permanent safeguard.
Why it transfers
Engineering and construction organizations face the same underlying problems this system was built to solve. Project knowledge is fragmented across files, email, software, and individual employees. Proposals and reports require repeated reconstruction of information the organization already has. Senior judgment is concentrated in a small number of people. AI experimentation happens without clear data or approval boundaries, automations accumulate as isolated scripts, and the same lessons are repeatedly rediscovered.
The transferable opportunity is an internal AI layer with the same properties demonstrated here:
- Connect fragmented knowledge across proposals, project records, correspondence, and prior decisions, with provenance intact.
- Define the data and approval boundaries in code, so confidential content stays on controlled infrastructure while external reasoning is still available where it is safe.
- Make judgment reusable and failures permanent, so settled decisions carry forward and incidents become regression tests, approval gates, and operating standards instead of repeat costs.
Qualified professionals keep technical and commercial approval; the system does the connecting, checking, and remembering that no one has time to do by hand.
Public disclosure note. This case study describes architecture, capabilities, generalized failure modes, and verified system metrics. It intentionally excludes personal content, private source documents, credentials and network details, identifying project information, sensitive prompts, internal file contents, and security-sensitive implementation specifics.
Josiah Bujanda • AI & Construction Technology Consultant
Construction AI • AI Systems Architecture • Workflow Automation • Business Systems Integration