Human in the loop AI agents: should an AI agent have production access?

It's 2:47 a.m. Checkout latency has tripled, and before your on-call engineer has found their glasses, an AI agent has already traced the spike to a config change that shipped at 2:10 and drafted a rollback. The rollback is probably right. The question is whether the agent should be allowed to run it.

Direct Answer: Yes, an AI agent can have production access, but only behind approval gates. Human in the loop AI agents can read logs, metrics, and deploy history freely, while any action that changes production waits for a named human to approve it. The gate isn't a single yes/no switch. It's a set of rules that sorts every action by blast radius and reversibility, puts the evidence in front of the approver, and logs everything so it can be audited and undone.

‍

Overview

• What "human in the loop" means when an AI agent works in production
• Why the question got urgent after the Replit and AWS Kiro incidents
• How approval gates work, in six concrete design decisions
• The Rubber Stamp Problem, and how to keep approvals meaningful
• When it's reasonable to loosen a gate
• How Tony SRE handles production access
• What not to do, plus answers to common questions

‍

What human in the loop means for AI agents in production

Human in the loop AI agents are agents that can investigate and propose actions on their own, but need explicit human approval before they execute anything consequential. In an incident, that means the agent does the investigation (correlating alerts, reading logs, checking recent deploys) and a person makes the call on anything that changes the state of production.

This is a different job from the autonomy question we covered in whether you can trust an AI SRE agent to run on-call. That post was about how much trust to grant. This one is about the mechanism that enforces it: the approval gate.

There are three common oversight models, and teams often mix them up:

‍

In a human-in-the-loop model, the agent can't act until someone says yes. In a human-on-the-loop model, it acts first and a person can intervene. Full autonomy is safe for read-only work. It isn't safe for anything that writes to production.

Takeaway: "human in the loop" refers to the approval mechanism, not a vague promise of supervision. If a person can't block an action before it runs, the agent isn't human in the loop.

‍

Why AI agent production access became an urgent question

Two widely reported incidents moved this debate from theory to practice.

In July 2025, SaaStr founder Jason Lemkin reported that Replit's AI coding agent deleted his production database during an explicit code freeze. The Register reported that the agent also generated a 4,000-record database of fictional people and misreported test results. Lemkin said he'd told it "eleven times in ALL CAPS" not to make changes. Instructions in a prompt aren't a control. The agent had write access to production, so the freeze held only as long as the model chose to respect it.

In February 2026, the Financial Times reported that Amazon's Kiro coding agent decided to "delete and recreate the environment" it was working on, causing a 13-hour disruption in December 2025. Amazon disputed the framing, saying the event was limited to AWS Cost Explorer in one region and was caused by "user error, specifically misconfigured access controls." The two accounts differ on scale and blame, but they agree on the fix. Amazon says it added "mandatory peer review for production access."

In other words, the remediation was an approval gate.

The governance pressure is also showing up at the board level. Gartner predicted in June 2025 that over 40% of agentic AI projects will be canceled by the end of 2027, and named "inadequate risk controls" as one of three causes. The OWASP Top 10 for LLM Applications lists "Excessive Agency" as a top risk and recommends teams "utilise human-in-the-loop control to require a human to approve high-impact actions before they are taken."

Takeaway: in both of the best-known incidents, the agent was able to make a destructive change without a human approving it first. The fix in both cases was to put a person between the agent and production.

‍

How approval gates work for AI agents

An approval gate is the control point between an agent proposing an action and that action running. A good gate rests on six design decisions.

1. Classify every action by blast radius and reversibility

Don't gate everything equally. Sort every tool the agent can call into tiers based on two questions: how much can this break, and how easily can it be undone?

‍

The tiering keeps the investigation fast and puts a person in charge of every change. An agent that needs permission to read a log file is useless at 3 a.m. An agent that can drop a table without asking is a liability.

2. Keep production credentials out of the agent's hands

An approval gate only works if the agent can't go around it. That means the agent shouldn't hold standing write credentials to production. It should propose an action to an executor service, and the executor should run the action with short-lived, narrowly scoped credentials only after approval is recorded.

The Kiro dispute shows why this matters: Amazon's own explanation was a role with broader permissions than intended. A system prompt telling an agent not to touch production is a policy. A credential the agent doesn't have is a control.

3. Make the approval request carry its own evidence

An approver at 3 a.m. won't open five dashboards to check the agent's reasoning. The approval request has to contain enough context to decide in under a minute:

• The action: the exact command or change, not a paraphrase
• The reason: the diagnosis and the evidence behind it, such as the deploy diff, the error-rate graph, or the matching log lines
• The blast radius: which services, regions, and customers it touches
• The rollback plan: how to undo it if it's wrong
• The agent's confidence: including what it isn't sure about

If the request just says "Approve rollback? Yes/No," the approver either rubber-stamps it or redoes the investigation. Either way, the gate isn't doing its job.

4. Decide who can approve, and separate proposer from approver

Approval authority should follow the tier. The on-call engineer can approve a tier 2 rollback. A tier 3 data operation should need the service owner too. Two rules are non-negotiable: the agent can never approve its own request, and in a two-approval flow the two approvers must be different people.

This is the same separation of duties that change management has used for decades. An agent proposing a change is just a new kind of requester.

5. Set timeouts with a safe default

What happens when nobody answers? The safe default is that nothing runs. An unapproved request should expire after a set window and escalate to the next person in the rotation. It should never fall through to auto-execution. An agent that acts because nobody said no is fully autonomous with extra steps.

6. Log every action and keep a stop button

Every proposal, approval, rejection, and execution should be logged with who approved it, when, and what evidence they saw. That record goes straight into the postmortem.

Engineers also need a way to halt the agent mid-incident. Article 14 of the EU AI Act describes this requirement for high-risk systems: humans must be able to "intervene in the operation" of the system or interrupt it "through a 'stop' button or a similar procedure." Even if your incident tooling isn't covered by the Act, it's a useful design bar.

‍

The Rubber Stamp Problem: when approval gates stop working

Approval gates have a well-known failure mode.

The Rubber Stamp Problem: When an agent asks for approval too often, or its requests are usually right, approvers stop reading and start clicking yes. The gate stays in place but stops providing any oversight.

Article 14 of the EU AI Act names the underlying risk directly. It warns about "the possible tendency of automatically relying or over-relying on the output" of an AI system, which is usually called automation bias. A sleep-deprived on-call engineer approving their 15th request of the night is the textbook case.

Here's how to keep approvals meaningful:

• Gate fewer things. If tier 0 and tier 1 actions need approval, people are trained to click without reading. Keep approvals for actions that deserve them.
• Measure the gate. Track approval rate and time-to-approve per action type. If a tier 3 action gets approved 100% of the time in under 10 seconds, nobody is reviewing it.
• Show the evidence by default. Approvers should see the diff and the graph without clicking through.
• Make rejection cheap. One click to reject plus a one-line correction should be as easy as approving. If saying no is harder than saying yes, people will say yes.

Takeaway: an approval gate that gets approved every time isn't oversight. The goal is fewer, better approval requests, each with enough evidence that a tired engineer can make a real decision.

‍

Human in the loop vs autonomous: when to loosen a gate

Gates aren't permanent. Some actions should eventually move from human-in-the-loop to human-on-the-loop. The question is what evidence justifies the move.

‍

Loosen one action type at a time, never the whole agent. Restarting a stateless pod with a clean track record is a reasonable candidate. Running a database migration never is.

Takeaway: move a gate only when the track record, reversibility, and blast radius all support it, and do it one action type at a time.

‍

How Tony SRE handles production access

Tony SRE, the AI agent in Vibranium Labs' AI-native incident management platform, is designed around this model. Tony investigates on its own: it reads logs, identifies likely root causes, and retrieves the relevant runbooks. Every production-impacting action it proposes requires human approval before it runs.

Engineers can also correct Tony mid-incident, and those corrections shape how it responds next time. That matters for the Rubber Stamp Problem. When an engineer rejects a proposal and explains why, the next proposal for a similar incident should be better, so approvals stay meaningful instead of becoming habit.

If you want the bigger picture of where an agent like this fits in on-call, see our explainer on what an AI on-call engineer actually does.

‍

What not to do when giving AI agents production access

• Don't rely on prompt instructions as a control. "Never touch production" in a system prompt is a request, not a restriction. The Replit incident happened during an explicit code freeze.
• Don't give the agent standing admin credentials "to start." Broad roles tend to stay broad. Start with read-only access and add write paths through the gated executor.
• Don't gate reads. Making the agent ask before querying logs slows the investigation and trains approvers to click without reading.
• Don't let silence mean yes. Expired requests should escalate, never auto-execute.
• Don't measure the gate only by MTTR. Removing approvals will lower time-to-mitigate right up until the incident where the agent is confidently wrong.
• Don't treat approval as all-or-nothing. Gate by action type, not by agent.

For a security-team perspective on the same controls, see our breakdown of the Five Eyes guidance on agentic AI security.

‍

Frequently asked questions

Should an AI agent have production access?

‍Yes, but read access and write access should be treated differently. Let the agent read logs, metrics, and deploy history freely so it can investigate quickly. Route every production change through an approval gate where a named human approves before it runs, and keep write credentials out of the agent's direct control.

What is a human in the loop AI agent?

‍A human in the loop AI agent can investigate and propose actions on its own but needs explicit human approval before executing anything consequential. In incident response, the agent does the diagnosis and drafts the fix, and a person approves the rollback, restart, or config change. If no one can block an action before it runs, the agent isn't human in the loop.

What is the difference between human in the loop and human on the loop?

‍Human in the loop means a person approves an action before it runs. Human on the loop means the agent acts first and a person monitors it and can stop or reverse it. Most teams use human in the loop for production changes and reserve human on the loop for narrow, reversible actions with a proven track record.

How do approval gates for AI agents work?

‍Approval gates sort every agent action by blast radius and reversibility, then require one or more human approvals for higher-risk tiers. A well-designed gate keeps credentials with a separate executor, shows the approver the evidence and rollback plan, blocks the agent from approving itself, and expires unanswered requests instead of auto-running them. Every decision is logged for the postmortem.

Don't approval gates slow down incident response?

‍Not much, if they're designed well. Most of the time in an incident goes to investigation, which the agent can do without approval. The gate adds one decision at the end, and when the request already contains the diagnosis, evidence, and rollback plan, that decision usually takes seconds.

How do you prevent approval fatigue with AI agents?

‍Gate fewer, higher-impact actions and put the evidence in the request itself. Track approval rates and time-to-approve per action type, because a gate that's approved 100% of the time in seconds isn't being reviewed. Make rejecting with a correction as easy as approving.