Scaling context: Building Miter’s internal brain

At Miter, we’re building the operating system for the construction industry. Our customers work in high-stakes environments where software glitches and long support loops are expensive liabilities. To keep up, our on-call engineers embed directly within our support operations, tackling high-priority escalations alongside our frontline team. Every issue used to follow the same ritual: read the ticket, query the DB, squint at logs, trace the codepath, and slowly piece together what actually happened. None of this is particularly hard; it’s just tedious, repetitive, and cuts into valuable time I can otherwise spend perfecting my lunchtime Cava order.

We wanted to collapse that loop. We started by building a Claude skill that pulls ticket context, queries the database, and navigates the codebase to explain the situation in plain English. The skill is incredibly effective for a simple reason: context switching is expensive. Adoption spread through our engineering team like wildfire. When thirty-minute investigations were replaced by thirty-second summaries, the manual sleuthing through the database, codebase, and logs became obsolete. Now, I can crack open a Spindrift and browse X while Claude flibbertigibbets and discombobulates away.

The success of this skill highlighted a growing divide. While engineers were moving at 10x speed, our non-engineering teams were being left behind because they lacked the environment required to run complex agents. This led us to evolve the tool into a hosted agentic workspace accessible to any Miter employee with a web browser.

The goal was to remove every technical barrier. You should not need to install a CLI, manage tokens, or configure a dozen MCP connections just to get an answer. By shifting to a containerized cloud environment, we replaced a fragmented mess of personal configurations with a unified infrastructure. It turns a high-barrier engineering tool into something as simple as a Google search. This “Ask Miter” interface allows the agent to perform deep forensics within its own private sandbox while maintaining a clear read-only boundary to our production systems.

For our support team, the experience is now seamless: they ask a question and get an answer that used to require a three-engineer search party to dig up the skeletons. Sales reps started answering niche, high-stakes questions mid-demo, and our launch team even started building their own skills to automate data pulls. Once that context became accessible to the whole company, a huge chunk of “engineering work” simply… stopped being engineering work.

The Miter brain: Architectural deep dive

We didn’t want to over-engineer a solution from scratch. Instead, we took inspiration from the background agent patterns pioneered by teams like Ramp (Inspect) and Stripe (Minions). Our goal was to build a scrappy, high-impact version of those systems in less than a week. Below is a high-level overview of our architecture:

Sandbox compute: The isolated runtime

Every agent session lives inside an isolated Modal sandbox, an ephemeral container with dedicated CPU, memory, and an encrypted network tunnel. Inside, we run a custom runtime built on opencode that serves a local API. The runtime is wired to a set of MCP servers, giving the agent structured access to internal tools and documentation. 

Crucially, the sandbox maintains a full checkout of our monorepo, updated via a scheduled cronjob. By allowing the agent to traverse the codebase, trace codepaths, and read config files in real-time, we ensure its investigations are grounded in the actual state of the product, not just stale documentation.

Bridging latency: The warm pool and prompt caching

To bridge the gap between infrastructure cold starts and a responsive UI, our orchestrator maintains a warm pool of pre-booted sandboxes. This eliminates the 9-second delay typically required to pull images and initialize the environment.

We optimize this further through a deliberate trade-off in prompt caching. We prime each warm sandbox by sending a throwaway prompt that triggers the model to cache our system instructions and tool definitions. While we pay for these unseen inference cycles, the investment yields a massive reduction in latency for the user’s first message. A scheduled reconciliation loop runs to refresh the warm pool and evict sandboxes nearing their 30-minute limit. This also serves as our deployment engine; if a new container image is detected, the system automatically drains and rebuilds the pool to ensure every session runs on the latest code and documentation.

Declarative context: The skill system

The Miter Agent uses a declarative skill system that allows anyone to build new capabilities in their browser without writing a line of code. Skills are markdown files in GitHub that declare their own dependencies, such as API docs or data models. When a user calls a skill, the orchestrator fetches these references in parallel and bundles them into the prompt. 

To bridge the gap between engineering and the rest of the company, we built a reference browser directly into the UI. Team members can create, edit, and preview skills in real-time without local dev environments. Updates are handled as atomic multi-file commits through the GitHub API, ensuring that versioning and metadata stay perfectly synchronized.

State management: Suspend and resume

To reconcile 30-minute ephemeral sandboxes with conversations that span days, we built a suspend and resume flow. When a session is idle for 10 minutes, the orchestrator snapshots the entire filesystem and stores the image ID in MongoDB before terminating the container.

When a user returns, we spawn a fresh sandbox from that specific snapshot. While the filesystem is restored, keeping bash history and tool artifacts intact, the LLM itself is stateless. To fix this, we re-inject the conversation history from MongoDB as a structured context block on the very first message of the resumed session. 

Learnings

The vision for this project was to solve a gap most companies ignore: democratizing AI instead of just letting the engineering org run up a cloud bill rivaling a small nation’s GDP. The reality is that a company’s foundational truths are usually locked away inside its codebase. We think that’s a bug, not a feature. However, democratization is impossible when there is friction. It is incredibly hard for non-engineers to keep up with a fast-moving engineering stack. When the solution requires someone to manually install CLIs, manage MCP connections, or rotate secret keys just to get an answer, people naturally fall behind. By moving those capabilities into a zero-config cloud environment, we are doing more than just saving time, we’re equalizing the playing field. Everyone across the company now has the same high-leverage toolkit as the engineering team. We’ve effectively cured the “Quick Question Disease” where a simple product question requires engineering context switching. Now, information stays free, while our product DNA stays focused.

Owning the harness

We know tools like Claude Desktop exist, and we’re careful not to fall into the trap of just rebuilding it. But the best AI tools aren’t the ones with the most features; they’re the ones you don’t have to think twice about using. With the Miter Agent living right in the browser, everything is already set up for our users. We don’t have to pick a single winning horse in the AI race. Instead, we own the harness, which gives us the sovereign environment to run experiments that the “off-the-shelf” market can’t yet offer. And if the “off-the-shelf” world ever gets so good that our custom agent becomes a relic? We’ll happily retire it. We aren’t here to build a monument, we’re here to get out of everyone’s way.

The Big Swing Philosophy

A wise friend once told me that projects like these are “how you avoid becoming a fossil”. There is a specific kind of battlefield wisdom that only comes from building at the edge. Taking big swings produces tribal knowledge that you simply wouldn’t otherwise have access to. As we bake these agentic capabilities into our actual product, those lessons become our greatest competitive advantage. This project reflects a broader commitment to a culture of high velocity. We believe that the future belongs to the teams that refuse to be bottlenecks. So, if you’re interested in taking big swings and building things that actually move the needle, come work with us. The fence is down, let’s see how far we can run.