Work / AI Systems

Pantheon

A local-first AI harness in one installable desktop app. Local or cloud models read and edit files, run commands, call connectors and dispatch agents, and every risky action passes a permission gate first.

Status
Working desktop app (v1.7.10), not released
Stack
FastAPI · React · TypeScript · Tailwind · Tauri · Ollama / LM Studio
Started
July 2026
Source
Private repo
Pantheon's Agent Constellation view: the agent crew orbiting the Oracle at the centre, with usage, skills, agents and hooks panels around it
The Agent Constellation, shot on a fresh starter vault.

What it is

Pantheon is an AI harness: the program that sits between a model and a machine. A local model through Ollama or LM Studio, or a cloud model with a key, can read and edit files, run shell commands, drive a browser, call MCP connectors and hand work to a crew of agents. Every action with side effects goes through a permission gate first.

It started on 5 July 2026 as something much broader: a personal operating system with a project pipeline, business and personal-life pages and a home dashboard. Over the following seven weeks those surfaces were cut one by one, and on 24 August Pantheon became the harness and nothing else. The capture pipeline stayed, because turning raw notes into tracked projects is part of the harness's own job.

Everything it holds stays in open formats: notes as plain markdown in an Obsidian-compatible vault, which has to be git-backed before the app will write to it, and telemetry as append-only JSONL. Every vault write goes through one tested adapter, and anything an AI writes is tagged as AI-written.

Today it is a working Windows desktop app at version 1.7.10. It is built as if it will be released, but it has not been. A pre-release audit closed in early September with all 82 findings resolved, and the most recent commit is from 7 September 2026.

What it does

  • Local models firstOllama and LM Studio are first-class. Cloud providers sit behind the same provider layer as optional extras, with fallback chains, and the app works offline against a local model.
  • A chat that actsFile read, write, edit, grep and glob, a shared terminal, background tasks and an agent-only browser, all available as tools in one conversation.
  • A permission gateEach tool call is allowed, asked about or denied, and a central approval prompt handles the "ask" cases. A built-in set of destructive commands stays denied in every mode.
  • Skills, agents, hooks, pluginsThe whole harness can be created and edited from inside the app. Agents carry deity names and are laid out by department on the constellation.
  • Dumps in, projects outRaw text is triaged by AI into ideas, tasks and projects. Projects then move across an explicit board: Inbox, Triage, Backlog, Active, Blocked, Done.
  • Git-aware sessionsA chat can open in its own worktree, and the app can branch, commit, push and open pull requests without leaving it.

Build log

From personal OS to harness

A dated record of how Pantheon was built, oldest first, from the first commit on 5 July 2026 to the latest on 7 September 2026. It covers every phase, rewrite debate, audit and cut along the way.

As of 4 Oct 2026, counted from the repository.

  1. Era I · 5 – 9 JulA personal OS in five days
  2. Era II · 10 – 12 JulThe chat becomes the centre
  3. Era III · 16 Jul – 12 AugHarness parity
  4. Era IV · 15 – 20 AugThe design handoff and the installed app
  5. Era V · 24 Aug – 7 SepThe harness and nothing else

Era I · 5 – 9 Jul 2026

A personal OS in five days

Ten build phases went into five days: the foundation, a working chat and vault, agents and connectors, and a wide spread of personal and business pages. Two full audits ran in the same week, along with a reversal of the "private forever" stance and a native desktop wrap.

  1. A fresh build, not a fork

    Pantheon started as a new codebase instead of a fork of Odysseus, an open-source local-AI app it studied. Odysseus kept user data in a database and had no project pipeline, which clashed with both of Pantheon's core needs. Five of its strongest parts were ported behind Pantheon's own interfaces: the provider layer, the encrypted key store, the skills format, the MCP client and its security patterns. Each port came from an MIT-licensed snapshot and kept its notices.

    Method Nine invariants were written before any feature code, among them local-first, plain-markdown storage, a single vault-write adapter and git-backed writes. Every plan since has been checked against them first.

  2. A React PWA on FastAPI

    The stack became React, Vite, TypeScript and Tailwind in front, shipped as an installable web app, with FastAPI in Python behind it. The reasons were practical: AI models write this stack most reliably, Python owns the local-AI ecosystem, and a single web codebase covers desktop now and phone later. Svelte, a no-build frontend and an Electron-first app were all considered and rejected.

  3. The walking skeleton

    By the end of the first day the app could be installed, could stream chat from a small local model and logged usage as JSONL. The core loop also worked: paste a raw dump, let a model triage it, and the results land in the vault as notes tagged with where they came from. All of it ran against a test fixture vault, so a bug could never damage real notes during development.

  4. Phases two to four in a day and a half

    Four more pieces landed before the second day ended. A kanban pipeline ran over note frontmatter, and skills and slash commands arrived. Chat history was saved as readable vault markdown, so every conversation survives outside the app. Then came an MCP client, agents stored as vault files, a tool-using chat loop, an automations scheduler and local voice dictation.

    Method The agent crew and the MCP client were built by three Codex workstreams running in parallel, with Claude reviewing every diff. Producer and verifier were always different models.

  5. A full audit, fixes deferred

    After phases zero to four, an adversarial audit across 23 dimensions turned up 74 findings, and 37 of them were confirmed real. Every confirmed finding came from the deliberate "single user, one machine, no auth yet" posture rather than from a regression. The rule that came out of it still holds: review each diff as it lands, and run a full sweep only at milestone gates.

    Brian's call He deferred every fix to the end of the build rather than churn code he was still shaping.

  6. The Pantheon style is locked

    The visual identity was fixed: obsidian-dark editorial, Greco-Roman, and a "living OS" view where you can watch agents work. Agents got deity names as their public face while their functional names stayed canonical in files and APIs. One rule stood out from the rest: nothing on screen may glow or report activity without real data behind it.

    Brian's call He locked the identity the same day an earlier note had said to keep it generic.

  7. The scope keeps widening

    Phases five and six added a business launchpad, a drawing canvas built on Excalidraw, personal life-tracking pages and a home dashboard. They worked, and over the next seven weeks every one of them was taken out of scope.

  8. An orchestrator and a bench

    A request for agents in every role became a catalog instead of a few hundred live agents. A bench of specialists drawn from the MIT-licensed agency-agents collection is written into house format only when it's needed, and an orchestrator agent routes work to the bench. Dispatch shipped with a depth cap of one, a budget per run, and a flat refusal to hand any task from a local model to a cloud one.

    Method Depth one came free: the orchestrator runs inside the chat loop and a dispatched specialist runs one-shot, so a second level can't exist at all.

  9. Phase seven in one Codex-first run

    Seven lanes of work shipped in a single run: a command deck, voice-dictionary learning, a live registry of running agents, and in-app authentication. Login is a password exchanged for an HttpOnly session cookie, and requests from the machine itself skip the check so local use stays friction-free. The test suite grew from 1,000 to 1,064.

    Method Codex wrote each lane from a written brief and Claude acted as manager, reviewing every diff and running the gates per lane.

  10. The extraction rule

    Reference clones of other repositories had been piling up beside the project. A standing rule replaced them: take what's useful into Pantheon's own structure, record the upstream source, pinned commit and licence, then delete the clone. The full-sweep audit for the end of the build ran the same day and returned 215 raw findings.

  11. The hardening wave

    Every confirmed finding from both audits was fixed. That included the three that blocked wider exposure, one of which was a path that could have sent private vault content to a cloud model. Authentication also gained an absolute session ceiling, a remote sign-out-everywhere switch and an audit log. A Codex review of the fixes found more, and those were closed the same day.

  12. Redundancy becomes a reason to remove

    The knowledge-graph view was deleted because Obsidian draws a better graph over the same markdown. From then on, being redundant with a better app has been an explicit reason to remove a feature. The constellation view stayed, since it is the design identity and not a knowledge graph.

    Brian's call Pantheon does not compete with apps that already do a job better; it connects to them.

  13. Built as if it will be released

    The founding stance of "private forever" was reversed. Pantheon may never ship, but its code and docs stopped assuming privacy. Licence notices moved into one central lineage document, and the rule against porting anything from the later, copyleft-licensed version of Odysseus went from dormant to binding.

    Brian's call The pivot was his, decided in one session.

  14. A desktop app and a project system

    A Tauri shell wrapped the same backend-served frontend as a native Windows app. The rule was to wrap it and never rewrite it. The same day the project system shipped: a project stays a note in the pipeline until it becomes real, at which point a folder is created for it, and deleting a folder takes a typed confirmation of the project's name.

    Method Spec, then plan, then eight staged implementation dispatches, each reviewed before the next began.

Era II · 10 – 12 Jul 2026

The chat becomes the centre

An overnight overhaul rebuilt the shell around a single conversation, and the harness gained a shared terminal, an agent browser and a real permission system. It ended in a debate over whether to rewrite the whole thing in another language.

  1. The overnight overhaul

    Pantheon now opened on a chat stage. A black-hole Oracle sat at the centre, drawn in WebGL, with a pan-and-zoom constellation of agents around it. Management panels showed only while a conversation was empty and disappeared once it began. Agents could no longer be prompted directly; dispatch became the main model's job.

    Method Brian dictated the brief overnight and a ledger of Codex and Claude lanes carried it out, each reviewed by a manager agent. A review pass fixed twelve issues across both stacks.

    Brian's call Answering the ledger's open questions, he ruled that decorative motion and real agent activity get separate visual languages that must never be confused, and that hooks would be Pantheon's own system rather than a mirror of another tool's.

  2. A terminal and a browser

    A real terminal shipped, built on a pseudo-terminal behind xterm.js, along with a persistent shell the AI can use and a browser the agent drives, with its outbound traffic guarded. The aim was the robustness of a coding harness, in the app itself.

  3. The permission gate

    A policy engine began sorting every tool call into allow, ask or deny. Ahead of all configuration sits a set of destructive commands that no rule, mode or bypass can override. One app-wide approval centre took over from the per-surface prompts. An independent review then found five holes, including a way to slip a delete past the deny-set inside an interpreter call; three were closed the same day, one was deferred and one was handed off as its own task.

  4. A stale backend, traced

    A report that chat "can't run shell commands" turned out to be an old backend still serving old code after an app restart. The app had started a backend if none was running and adopted whatever it found on the port. The first answer, recorded that day, was to make the backend a supervised service with a version handshake. Separately, two migrated projects were removed from the repository because their very long file paths broke git worktrees.

  5. A rewrite decided, not started

    After studying OpenCode, a four-voice council voted unanimously to keep the stack. Later the same day the opposite was recorded: rewrite everything in TypeScript on Electron with a Node backend, migrating piece by piece. No code was written for it.

    Brian's call Electron as the shell was his choice in the rewrite decision.

Era III · 16 Jul – 12 Aug 2026

Harness parity

The rewrite was called off in favour of closing the real gap, which was in the tools and not the shell. An evaluation suite came first, then five stages of capability in a single day, then a rethink of how the installed app owns its backend.

  1. No rewrite; the gap was the tools

    A second council, with Codex in the critic's seat, again voted unanimously to keep the stack. The premise had been wrong: Electron is the shell of chat clients, not the source of any harness's capability. An audit found the real gaps were in the backend. Claude models couldn't use any tool at all, there were no file tools, the loop stopped after five steps and tool calls ran one at a time.

    Brian's call He had promised no rewrite once a model could create a file from inside Pantheon, and the promise was kept.

  2. An eval suite first, then five stages

    A versioned suite of end-to-end scenarios, run against both a local and a cloud model, became the exit gate for every stage. In one day the harness gained tool use for Claude models, file tools behind the permission gate (search runs on a bundled ripgrep), higher loop caps, background tasks the model can start, and tested skill auto-firing. A local 14-billion-parameter Qwen model finished at 12 of 12 scenarios.

    Method Codex had argued that both paths fail if judged by demos, and the eval-first order was its dissent, adopted.

  3. Hooks that watch, never block

    Pantheon's own hooks shipped as an observe-only layer. They fire before and after tool calls and automations, and can only journal or run a literal, operator-written command, which itself has to pass the permission gate. Hooks able to veto a call were rejected, because blocking is already the permission policy's job and a second place to block would be less audited.

  4. The eval suite catches silent truncation

    The scoreboard dropped to 9 of 12 after the final stage, yet the code wasn't at fault. Ollama's default 4,096-token window was cutting off long tool-loop transcripts mid-task, and the model was losing its instructions without any error. Only an end-to-end scenario could have caught it. A month later the harness learned each model's context window and began refusing an overflow up front rather than letting it truncate.

  5. Open it and everything starts; close it and everything stops

    The supervised-service design was reversed. The installed app now starts its own frozen backend inside a Windows job object, so closing the window, a crash or a forced kill takes the backend down with it. It never adopts a backend it didn't start, and it refuses one whose version doesn't match.

    Brian's call He re-anchored the product on how Claude Desktop and Codex behave, and parked the requirement that the backend outlive the window.

  6. An overnight readiness pass, and two things deleted

    This pass fixed 30 UI defects, including a phone-width layout, and closed several security holes in the authentication boundary and the file tools. The first round of those fixes came back broken under adversarial checking, and the corrected versions shipped instead. An in-house web search was written and then deleted on review, since a working search connector already existed. Recording completion failures in the provider circuit breaker was built and reverted the same day after it locked out the local model.

  7. Transcripts that show the work

    The chat's density lenses became true views of one transcript. Verbose shows real line diffs, file windows and terminal output, Normal shows collapsible chips, and Summary shows one-line chips with real details. Tool activity began to be saved in the session note, so the verbose view survives a reload; before this it had been lost.

Era IV · 15 – 20 Aug 2026

The design handoff and the installed app

A full UI handoff was built in a day. The following days were mostly spent finding out what only the installed app could show: fixes Brian never received, a terminal that never started, empty vaults and invisible menus. The constellation also got a spacing standard, and the app adopted a component library.

  1. A UI handoff, built in seven slices

    A complete design handoff became the UI contract and was built the same day. It brought a department constellation of six galaxies, draggable stage panels, a reworked shell and composer, and a git harness with worktrees, branches, commits, pushes and pull requests. Fast mode was made to depend on the provider, so a small local model counts as a legitimate fast tier.

    Method Seven parallel Codex slices in separate git worktrees, each reviewed and merged by Claude.

    Brian's call Sessions get the full git harness, and fast mode is never tied to one vendor.

  2. A setting must control something

    A sweep of the eleven interface settings found five that changed nothing. Four of those named features that had never been built, though the design mock had them behind the same switch names. Rather than delete the switches, all four features were built. A setting the user is offered that nothing reads breaks the honesty rule in exactly the place they're told they're in charge.

    Brian's call Build the features, not delete the switches.

  3. Panels dock instead of taking over

    The terminal, browser, artifacts and git views became docked panels that shrink the stage instead of replacing it. Hovering previews a panel and clicking pins it. Worktree and pull-request controls moved above the composer because they belong to the session being typed into.

    Brian's call The layout, and the terminal as dock-only, were his corrections; you type into a terminal, so a preview that vanishes on mouse-out is worse than none.

  4. Agents that couldn't be clicked

    No agent on the constellation could be selected. The pan gesture captured the pointer on every press, which redirected each click to the canvas, and the large invisible glow around each orb caught clicks meant for its neighbours. Capture now begins only once a drag starts, glows ignore the pointer, and every agent on the stage became reachable.

  5. Ship it or Brian cannot see it

    Brian reported the click bug for a third time while the fix had already been committed twice. Both were true: he uses the installed app, and its build predated the first fix by twenty minutes. Since then, any session whose work shows up only in the app runs the update pipeline before it ends, without being asked.

    Brian's call A standing rule, along with its companion: stop every server you start.

  6. The installed app's empty vaults

    The installed app had been opening empty vaults inside its own program folder, a folder the next upgrade replaces wholesale. A wrong path became data loss on a timer. Vault paths moved out of the install tree, a first-run setup screen began creating and git-initialising the vault, and the starter roster of 27 agents ships with release-safe text that names no real person or project.

  7. The harness becomes writable

    Agents, skills and hooks can now be created, edited, renamed and deleted from inside the app, and all of it goes through the single write adapter. An edit merges into what's on disk instead of replacing it, so fields no form collects survive. A rename writes the new file before removing the old one, and a delete removes a folder only if it is empty.

  8. Memory, plugins and chips

    Every model the harness runs now reads a global PANTHEON.md on every turn, plus the project's own file when a chat is tied to a project. Plugins arrived as bundles of skills that are read in place and never copied. Task chips arrived too: a model can post a one-line suggestion for side-work that opens its own session on a click, modelled closely on how Claude Code does it.

  9. A terminal that had never worked

    In the installed app the terminal showed only a blinking cursor. The packager had collected the terminal library's DLLs but not the two helper programs it launches at runtime, because static analysis can't see a dependency that is launched rather than imported. Errors had also been swallowed silently. The build now bundles the helpers and checks they're present, and a terminal that fails to start now says so on screen.

  10. The build states its own identity

    Stale interface bundles had reached the screen three different ways, including a service worker that kept re-registering itself in a build documented as having none. The build now stamps its own id into the bundle, and the backend reports which build it is serving, so freshness is checked by comparing two identities instead of asking a cache. The installed app had been carrying 1,795 leftover files from earlier builds, and they are no longer served.

  11. One spacing standard for the constellation

    The galaxy had been re-tuned three times in three days, and each pass just moved the tightest gap somewhere else. A single slot template now derives every distance, so adding an agent never needs a re-tune. Space reserved for planets that only department leads can host was given back, and the lead orbs grew from 13.7 to 30.1 pixels on Brian's screen without any gap getting tighter.

    Brian's call The ring size, and the ruling that only leads host planets, were his.

  12. shadcn/ui, with the token sheet in charge

    The interface moved onto shadcn/ui, which is Radix primitives copied into the repository as source. A styled library was rejected because the app already has a visual identity. A test page checked the colour mapping before any screen was converted and caught four mapping bugs that couldn't be seen by eye. The old context menu had declared menu roles for assistive technology with no keyboard support behind them, and it is now fully navigable by keyboard.

  13. Two limits measured before buying

    Two leftovers from the migration were measured before anything was changed. The tooltip swap turned out to be a consistency refactor with a narrow accessibility gain, and all 85 call sites were moved anyway. Native form controls were kept for the moment. The real defect was elsewhere: 44 controls had no accessible name, and all 44 were fixed, with a scan added so the problem can't come back.

    Brian's call The full tooltip swap, and keeping native controls (overridden a week later).

  14. Every tooltip shipped at 16 pixels

    The class-merging helper was silently deleting the app's custom text sizes. It read them as colours and resolved the supposed clash by dropping the size. The same flaw stripped the ink from small buttons. The fix went in once, at the helper itself, and a test now reads the custom sizes straight from the config so a new one can't be missed.

Era V · 24 Aug – 7 Sep 2026

The harness and nothing else

Everything that wasn't the harness came out, about 26,000 lines in all. A pre-release audit followed, and its fixes kept failing until a different agent attacked them. The era closed with a full read-through of the backend.

  1. The harness split

    The personal and work-hub sections were removed outright: about 26,000 lines, twelve route modules and fifteen services. The state before the split was tagged in git, and nothing was hidden behind feature flags. The capture pipeline stayed as part of the harness's core, and the navigation came down to three surfaces: Pipeline, Pantheon and Projects.

    Brian's call Ruled by voice: retire everything that is not the harness.

  2. Out of scope, permanently

    Brian reported that the app still had a dashboard. It didn't: the shortcut he had clicked belonged to one of the two sibling apps that had taken over those jobs. The ruling that followed was that a dashboard, a business surface or a life-tracking feature is out of scope for good, and the sibling projects left this repository's docs entirely. The documentation pass that followed still found three real defects, one of them a prompt telling every model about a vault the app no longer had.

    Brian's call A proposal to add any of those is answered "out of scope" without re-arguing it.

  3. Every control that can be Radix is Radix

    The ruling to keep native controls was overridden, and twelve selects and ten checkboxes were converted. Only a real browser could show the problems that followed. A dropdown opened from a dialog rendered beneath that dialog's own overlay, and a small radius on a 15-pixel box drew every checkbox as a gold circle. A legal design token in the wrong place is exactly what a token check can't catch.

    Brian's call The counter-case was put once with its evidence, and he chose full conversion as standing policy.

  4. A label must come from what resolved it

    In the installed app, and only there, the git panel showed "Not a git repository" under an option that named the app's own repository. That option pointed at the install folder, where no repository exists. The backend now reports what it actually resolved, and the option appears only when there is a repository behind it. The same day, app updates went silent: the new build is verified headless, and a window reopens only if one was already open.

    Brian's call Updates must not open a window he didn't ask for.

  5. The pre-release audit

    A full audit across 24 dimensions ran with two models verifying, plus a hands-on walkthrough of a fresh install, and returned 82 ledger entries. The critical ones clustered on a single assumption: that any request from the machine itself was Brian. A web page open in any browser can also reach a local port, so allowed-origin checks were added to sit beside that trust. All five critical findings were fixed the same day.

    Method Each critical fix was wrong or incomplete until a different agent attacked it, which surfaced three working bypasses of code that had already passed its own tests. Adversarial verification was budgeted as a first-class cost from then on.

  6. Verifiers keep breaking fixes

    The whole backend was cleared in the second session. A verifier refuted three of one lane's six fixes with reproductions, and one of the three re-opened the very boundary it was meant to close. Two gates the docs had claimed for weeks finally existed: a Python linter, which found 31 of its own suppressions were dead, and a doc-size check.

    Method Each lane wrote failing tests before production code, so when lanes died on rate limits the specification survived and only the implementation was lost.

  7. The campaign closes, 82 of 82

    All 82 entries were resolved: 81 fixed and one refuted with evidence. An accessibility test had been passing while it skipped exactly the fields it most needed to check. Once again, a regression test passed against the very bug it was named for, and it was rewritten. A project attribute that contradicted the locked scope was deleted rather than explained away in the docs.

    Brian's call Removing the attribute followed his standing scope ruling.

  8. A full backend read-through

    A pass over every backend module found three defects. The shell reported success when a PowerShell command failed, so a model would treat a failure as done. A dead dependency still shipped after the feature it served had gone. And approval prompts didn't show which folder a command would run in. A sweep across the backend and frontend then stripped audit ids and incident history from comments, leaving only invariants and reasons. Version 1.7.10 was cut that day, and the latest commit followed on 7 September.