Your AI is brilliant. And blind.

I keep a fair amount of production running. A GCP environment with 100+ VMs, a couple of bare-metal racks, pfSense firewalls, Arista switches, a 30TB Cassandra cluster, PostgreSQL and Redis here and there, Nomad scheduling the workloads, TrueNAS Enterprise with MinIO, a few Pure FlashArrays, and plenty of other moving parts. VictoriaMetrics, VictoriaLogs, and Grafana watch all of it. It’s the kind of setup where when a disk fills up or a service starts flapping at 2am, there’s no platform team to page. It’s me, an SSH session, and journalctl.

And since I’m the only person on call 24/7 and generally responsible for keeping it all running, I’ve been keen to hand some of that off. Over the last year the coding agents got good - good enough, at least, that I let them write real code as long as I review it. But there’s a wall they hit every single time, the moment the work leaves my editor and touches the actual infrastructure.

Brilliant at code, blind on the host

Ask an agent why a Nomad job keeps restarting and you’ll get a beautiful answer. Five plausible theories, ranked, with the right vocabulary and a confident tone. And every one of them is a guess, because it has never seen the allocation, never read the logs, never noticed the host is out of inodes. It’s diagnosing a patient it has only heard described over the phone.

That’s the thing people miss about why agents are bad at ops. It isn’t intelligence - the model can reason about your stack better than most humans can. It’s that it’s missing the two things that make an engineer useful on infrastructure: context and access.

Context is the live state: what’s actually running, what the logs say, the fact that a disk filled up ten minutes ago. Access is being allowed to go look, and then to do something about what you find.

In your editor the agent has both - the code is right there and it can change it. On your infrastructure it has neither, so it does the only thing left to it: it pattern-matches from training data and whatever you’ve pasted into the chat, and writes you a very articulate guess.

For a while you just bridge the gap by hand. It asks what nomad alloc status says, you paste the output in. It asks for the logs, you paste those too. You’re its eyes and hands while it does the thinking - and it works, right up until you realize you’ve become a copy-paste relay between a chatbot and a terminal. Nobody stays there long.

So you give it a shell

There’s an obvious fix, and most people reach for it within about five minutes: give it a shell.

Drop it onto the box over SSH. Tell it about tailscale ssh once and it’ll shell into production on its own, including the times you’d rather it didn’t. Wire up an MCP server that runs bash. Point it at gcloud ssh or your kubeconfig. Pick your poison - the effect is the same, and the blindness just evaporates.

Now it can look. It runs the status command, tails the logs, greps the config, spots the full disk in thirty seconds flat, and proposes the fix. The first time you watch it track down the problem and apply the fix on its own, it lands like a trick - and you catch yourself thinking you might actually sleep through the next incident.

And then, if you’ve run infrastructure for any length of time, you start to feel a little uneasy. Because you think about what you just did and what can happen.

You just handed root to a guessing machine

When was the last time somebody on X posted about an AI agent deleting something it shouldn’t have? Yeah, today. You gave a process that hallucinates - confidently, in fluent complete sentences - a root shell on production and permission to run anything in it, even when you’re not watching.

And the guardrails you’re counting on don’t work with SSH - when the agent asks you to approve the next command, all it shows is ssh iamnaive@mycorpprod.com '<50 lines of bash truncated to 50 chars>' (or, even worse, cat script.sh | ssh ...). There is no easy way to review that, and right underneath there’s an option to approve every command with “this prefix”. You click it once out of fatigue, and now the agent has more freedom on your production than it has on your dev machine.

None of this is a bug in the permission system. It just can’t see past the first quote.

Walk through what that actually means:

• It can run anything you can do in a shell.
• It guesses on a live system. The most innocent mistake it can make is a syntax error. But it can also run a nomad job stop on the wrong job, or an rm -rf where a variable quietly expanded to /. A perfectly valid command pointed at the wrong thing does damage too.
• It leaks. No matter how hard you try, your agent will eventually read your kubeconfig, your tokens, or your access keys. Now your production credentials are in your scrollback and in some model provider’s logs.
• There’s no record. When it goes sideways, the entire forensic story is “scroll up and hope you can reconstruct what happened.”
• An SSH key can be reused. One SSH key gives the agent lateral movement into every other VM that key opens.

And you can’t fence it at the shell level either - no amount of patterns, allowlists, or even supervising agents will stop a free-form text model from finding a phrasing that works. The shell access itself is the problem, not the agent holding it.

Locally those problems can be solved by sandboxing the agent so a bad command can’t escape the repo. But you can’t sandbox production - production is the thing you’re trying to maintain and protect.

And that’s the whole reason I want this to work. I’d happily hand the infra babysitting to an LLM - it buys back the hours I’d rather spend on code I actually like writing. What I won’t do is buy those hours by leaving the laptop caffeinated with a model still loose on production, hoping I come back to find everything where I left it.

I just don’t enjoy that mix of anxiety and adrenaline that hits me every time I step away.

What I built instead

Everything new is just a new way to do the same old thing. How did we give somebody restricted access before? We gave them buttons and knobs that performed specific actions, and we let them push those buttons. We didn’t give them full access to do whatever they wanted.

So I stopped trying to fence the shell and simply took it away. That’s emisar - the reason I let an agent near my infrastructure at all. The idea is simple: give it eyes, not a shell.

The agent gets a catalog of declared actions - a fixed allowlist, each entry a specific operation with typed, validated arguments rather than free-form text. “Tail this service’s logs.” “List the failed allocations.” “Stop this job.”

It’s built as an MCP server and a CLI tool, so any AI agent (Claude, ChatGPT, Cursor, and pretty much any other) can drive it directly, or you can call it from a script. The runner that does the work sits on your host and dials out over TLS, so there’s no new port to open.

Every action carries a risk tier. When the agent wants to read something - tail a log, list allocations - it just runs, so investigation stays fast and autonomous. When it reaches for something destructive - stop a job, restart a database - the action freezes and waits for approval: I see who asked, the exact arguments, the target host, and the policy rule that tripped the gate, and nothing executes until I say yes.

Policy is default-deny, so anything not explicitly allowed is simply refused. And every call - allowed, approved, or denied - lands in a hash-chained journal on the host and a searchable audit in the cloud, with known secret patterns scrubbed from the output before it leaves the box.

How does it feel?

Yes, I’m biased, but I have to say it - using this thing feels like magic. You ask an agent to build packs of actions covering your entire infrastructure, review them once (they’re just YAML), deploy them, and from then on you can run it in YOLO mode all day long without the downsides we just discussed.

A careful reader might ask: “what if the agent needs something that isn’t in the catalog?” And it’s a fair question. The catalog is only as good as what you thought to put in it. The agent can draft the packs, but deciding what belongs in them is on you - and when something lands outside of what’s declared, policy is default-deny, so the agent stops and you have to go open a terminal yourself. That’s the trade, and one I’d take again: spending a little more time up front to buy the ability to walk away.

For example, a few months ago a storage driver bug wiped a chunk of my telemetry - a CSI driver raced the kernel and ran mkfs over a live volume during a node drain. The agent worked the incident through declared actions across half a dozen packs, traced it to a multipath path-group race, and when it wanted to stop the affected allocation - a high-risk action - it stopped and asked me first. Then it shipped proper Terraform changes to prevent this from happening again. The whole thing took a few hours, and it only took that long because I was slow to respond.

And yes, emisar wouldn’t have prevented that wipe - that was a driver fighting the kernel - and no amount of gating will fix the model’s judgment either. What it changes is everything around the incident.

Just try it. It’s at emisar.dev - source-available, and free for a few runners, which is plenty for a home lab or a staging setup. It’s early, so go break it, and tell me what broke.