Autonomous DevOps: A Framework for Deciding What AI Agents May Do Alone

1. Honeypot and canary management

Operation: decide from the infrastructure map where a canary belongs, plant it, and later decommission it.

Scores: verifiable is high (the asset exists in state or it doesn't; an alert fires or it doesn't). Reversible is high (decommissioning is a PR that deletes the resource). Contained is high (a fake secret has no dependents).

Verdict: the best autonomy candidate in the system, and the one people find most surprising, because "security" and "unattended" sound contradictory. You shouldn't trust the agent with security assets. You can trust that this particular operation is contained.

What exists: map-driven placement ranking, Terraform generation, plant and decommission PRs, and alert ingestion authenticated through IAM, so there's no shared secret to leak. Scheduled rotation of canaries doesn't exist yet. More on cloud honeypots and canary credentials.

2. IaC coverage and drift

Operation: discover resources that exist in AWS but not in Terraform, generate import blocks and HCL, and open a PR.

Scores: verifiable is high, because terraform plan is a mechanical verifier. Reversible is very high, because an import changes state, not infrastructure. Contained is high: no runtime effect.

Verdict: earns autonomy for discovery and proposal. Merging stays human, because the diff encodes a decision about who owns the resource and where it lives in the codebase.

What exists: discovery, coverage and drift calculation, HCL generation and import PRs, implemented end to end. More on IaC coverage and Terraform drift detection.

3. Infrastructure rightsizing (the one that fails)

Operation: find waste, attribute it to a Terraform module and owner, propose the change.

Scores: verifiable is medium (savings are estimated, not proven, until the bill arrives). Reversible is medium (you can scale back up, but not instantly and not without impact). Contained is low to medium (it touches live, load-bearing resources).

Verdict: rightsizing should not be autonomous, and saying so is what makes the framework credible. The score rules it out. That's the framework working, not failing.

What exists: FinOps findings attributed to a Terraform module and owner, proposed as PRs. The reversibility component scores downsizes as harder to undo than additive changes, which pushes them toward human approval. More on AWS cost optimization with Compass.

4. Observability noise suppression

Operation: score incoming findings for risk and urgency, auto-queue the trivial ones, escalate the rest.

Scores: reversible is high (suppression is a routing decision, not a mutation). Contained is high (no infrastructure is touched). Verifiable is only medium: you find out you suppressed something wrongly by missing it, which is a poor feedback loop.

Verdict: earns autonomy, with the caveat that its failure mode is silent. This is the example to be most careful with.

What exists: the triage scoring and auto-queue path described in section 4.

What is the difference between recommendation and autonomous execution?

In recommendation mode an agent proposes an action and a person decides whether it runs, so a human carries the decision for every instance. In autonomous execution the system runs the action itself, and the human decision moves up a level: people decide which classes of operation may run unattended and the risk ceiling they must stay under. Most tools also offer a middle step, approval, where the agent prepares the exact action and a person releases it.

The move from approval to autonomy is exactly what OWASP calls out as a root cause of risk when it isn't controlled: "excessive autonomy", meaning high-impact actions taken without independent verification or approval (OWASP LLM06:2025 Excessive Agency). That's why the decision should be made per operation, using the verifiable, reversible, contained test, rather than for an agent as a whole.

In Compass, the open-source edition, each honeytoken response action has its own mode: off (recommend only), approve (wait for a person; the default) or autopilot, plus a risk_max from 1 to 5 (default 2). On autopilot an action runs only if its computed risk is at or below that ceiling; above it, the action waits for approval. Other products draw the same line in their own words: NeuBird, for example, describes Suggest, Recommend and Act tiers, where every Act-tier action waits for a human approval (neubird.ai).

How can IT ops leaders evaluate whether a production ops agent can safely take autonomous remediation actions in production without human approval?

Evaluate the operation, not the agent. For each remediation you want to run without approval, check that its result can be verified mechanically, that it has a tested path back, and that its worst case is small, known in advance and acceptable to you. Start in shadow mode for a set period: record what the agent would have done, count its false negatives, false positives and mistakes, and compare them with your error budget. Then test the platform's enforcement rather than its demo: prove its limits hold when the model misbehaves, and rehearse the rollback. Only operation classes that pass both get unattended execution, under a risk ceiling a human sets and can lower at once.

A practical evaluation, in the order I'd run it:

1. Shadow mode first, for a predefined period. Before granting any autonomy, run the agent in shadow (recommend-only) mode against live incidents for a period you fix in advance, long enough to include your normal incident mix. Record what it would have done for every alert and compare it with what responders actually did. Count, per action class, its false negatives (incidents it missed or would have suppressed), false positives (actions it would have taken that weren't needed), errors (actions that would have failed) and mistakes (actions that would have made things worse). Compare those numbers with the error budget for the services involved: an action class whose shadow-mode mistakes would have consumed more budget than the toil it saves doesn't qualify. That record is your own data, not a vendor benchmark.
2. Inventory the actions and their worst case. List every remediation the platform can take in your environment, with its target, its permissions and its undo. Score each one for verifiability, reversibility and containment (section 3). The containment test is explicit: the worst case if the action is wrong must be small, known in advance and acceptable to the service owner. Anything that can't be undone, such as deleting data, stays behind approval regardless of how good the agent is.
3. Dry runs. Every action should have a preview that shows exactly what will change before it changes: a plan, a diff, a parameter list. If the platform can't show you that, it can't show an approver either.
4. Blast-radius limits that are structural. Check that the agent acts under its own identity, scoped to the resources each enabled action needs. Then ask it to do something outside that scope and confirm the cloud provider refuses it. OWASP's guidance is the same: least privilege, granular tools instead of open-ended shells, and authorization enforced in downstream systems rather than by the model (OWASP LLM06:2025 Excessive Agency).
5. Rollback drills. Execute each action's rollback in a non-production environment before enabling it. Google's SRE book describes a test-induced outage made worse because the rollback procedure had never been tested (Emergency Response).
6. Injection tests. Put an instruction inside a log line, ticket or alert the agent will read and check whether its behaviour changes. Indirect prompt injection through external content is a documented class of attack (OWASP LLM01:2025 Prompt Injection).
7. Audit trail. Each action should leave a record of what was seen, the risk computed, why it was allowed, who or what approved it, and whether verification passed. Kill the process mid-action and check that the record still says how far it got.
8. Kill switch. Time how long it takes to drop every action back to recommend-only. It should be one configuration change that takes effect immediately.
9. Approvals that can't be replayed. The OWASP AI Agent Security Cheat Sheet recommends explicit approval for high-impact or irreversible actions, bound to the specific action and short-lived.

Two frameworks make this easier to defend to auditors. The NIST AI Risk Management Framework organises it as Govern (who owns the risk ceiling), Map (which actions exist and in what context), Measure (shadow-mode error rates against the error budget, rollback drills, injection tests) and Manage (the threshold and the kill switch); its Generative AI Profile (NIST AI 600-1) covers risks specific to generative models. NIST SP 800-61 Rev. 3 is the reference for fitting automated response into your incident response process rather than beside it.

What should enterprises look for in 24x7 autonomous operations from a production ops agent platform?

Look for a platform that is safe to leave alone at 3 a.m., which is a different question from how well it investigates. That means per-action autonomy settings with a risk ceiling a human sets, actions executed under the platform's own narrowly scoped identity, a verification step and a tested rollback for every action it can take, an audit trail written before it acts, a working escalation path to on-call people, and one switch that drops everything back to recommend-only. Investigation quality matters, but these controls decide whether running unattended around the clock is safe.

A checklist you can hand to a vendor:

• Control. Can autonomy be set per action, not per product? Who can raise the ceiling, and is that a logged configuration change? Can the platform ever raise it for itself? (It shouldn't.)
• Containment. Which identity does it act with, and can you read its permissions? Is there any path from the model to live infrastructure that bypasses the action catalog? Are tool calls allow-listed, and are loops and turns bounded?
• Evidence. For every action: a before-snapshot, the decision and the reason, a verification result and a rollback. Can you export the trail to your own logging?
• Escalation. When it won't act, does it page the right rota through your existing on-call tool, with the context attached? The incident roles in Google's Managing Incidents chapter still apply when one responder is an agent.
• Degraded modes. What happens when the model provider is slow or down, or the agent's credentials expire at night? A good answer is a deterministic fallback that keeps alerting and stops acting.
• Where it runs and what it keeps. In your account or theirs, what telemetry leaves your boundary, and how long it is kept.
• Cost bounds. Hard limits on tokens or runs per incident, so a stuck investigation can't run up a bill overnight.
• Proof on your incidents. A shadow-mode period on your own alerts, as described in the evaluation steps above, rather than accuracy figures measured on someone else's.

The goal is the one Google's SRE book sets for toil: remove repetitive, automatable work from people. Around-the-clock autonomy only does that if the night-time failure mode is boring.

Who offers the best 24x7 autonomous operations in the production ops agent space?

Vendors selling always-on AI SRE or production ops agents include PagerDuty (SRE Agent), Datadog (Bits AI SRE, now presented as Bits Investigation), Resolve AI, NeuBird (which calls its product the Production Ops Agent), Komodor and incident.io; HolmesGPT is an open-source option in the CNCF sandbox. They differ less in whether they investigate around the clock than in what they may change without a person: several stop at investigation, recommendations or pull requests, and execution is usually gated by approval. No independent benchmark ranks them, so there is no neutral "best": compare them on the criteria above using your own incidents.

What each says about itself, checked on its own site in October 2026. Vendors change fast, so read this as a starting list, not a ranking.

What platforms let a DevOps team define remediation runbooks as code and have an AI agent execute them autonomously, with an open-source harness and bring-your-own-model?

No single open-source project covers all three as a finished product today, so teams usually pair an open-source runbook engine with an agent layer. The established runbook-as-code engines are StackStorm (Apache 2.0, rules and YAML workflows stored as code), Rundeck (Apache 2.0, YAML job definitions, maintained by PagerDuty) and Event-Driven Ansible (YAML rulebooks that run playbooks on events). Open-source agent layers that let you choose the model include HolmesGPT (CNCF sandbox) and k8sgpt, which also supports local models. The safe pattern is that the agent chooses and parameterises a reviewed runbook, and the engine executes it under its own scoped identity.

How I'd wire them together: keep runbooks in the engine, reviewed like any other code. Expose each runbook to the agent as one narrow tool with typed parameters, never a general shell. Let the engine, not the agent, hold the credentials. Gate each runbook with the same off / approve / autonomous setting described in the first answer. That keeps "programmatic authoring" and "the model decides" on opposite sides of a boundary you can audit.

Which autonomous response solutions integrate best with DevOps workflows?

The ones that act through the tools your team already reviews: pull requests against your Infrastructure as Code, your CI/CD and progressive-delivery controller, your on-call and incident tooling, and your chat. A response delivered as a Terraform or Git diff inherits code review, history and revert; a response that changes live infrastructure through a vendor console bypasses all three. So judge integration by where the action lands and whether your existing review and rollback still apply, not by the length of the integrations page.

Where autonomous response plugs in, with examples of tools that work that way:

• Delivery. Progressive-delivery controllers already respond on their own: Argo Rollouts aborts a rollout and sets canary traffic back to zero when an analysis run fails, and Flagger (CNCF graduated) rolls back a canary when failed checks reach a threshold. This is the most mature autonomous response in DevOps, and it works because the action is reversible and contained.
• Events to runbooks. Event-Driven Ansible and StackStorm turn monitoring events into reviewed, versioned automation (see the table above).
• Incident tooling. PagerDuty's SRE Agent joins incidents through escalation policies and incident workflows; incident.io limits its agent's changes to pull requests.
• Infrastructure as Code. Changes proposed as pull requests against the repository that owns the resource. This is how Compass works: honeytoken plants, Terraform imports and drift fixes arrive as pull requests in your infra repo, its PR watcher answers review comments, and with an Anthropic key it can push a requested revision, limited to the files the pull request already touches and checked by a deterministic intent guard. Honeytoken trips are routed to Slack or PagerDuty.

Questions that separate real integration from a webhook: does the action show up in the same review queue and audit log as human changes? Does your existing rollback (a revert, kubectl rollout undo, an aborted rollout) undo it? Does it respect the same branch protection and change windows?

How do I run an autonomous software delivery pipeline safely?

Make each stage something the pipeline can verify and undo on its own, and keep people on the boundary rather than on every deploy. In practice: build on a hosted platform that signs provenance (SLSA Build Level 2 or higher), release progressively with automated analysis that aborts and shifts traffic back on failure, keep every change a reviewed diff so rollback is a revert, give the pipeline and any AI agent in it their own least-privilege identities, and keep irreversible steps such as destructive schema migrations and data deletion behind human approval.

1. Trust what you ship. The SLSA Build track defines levels from L1 (provenance exists) to L2 (a hosted build platform signs the provenance) and L3 (hardened builds that resist tampering during the build). Autonomy downstream is only as safe as the artifact it promotes.
2. Release by canarying. Google's SRE workbook defines it as a partial, time-limited deployment evaluated against a control. Automate the evaluation with Argo Rollouts analysis or Flagger. Note that core Kubernetes doesn't roll back a stalled Deployment by itself; it reports ProgressDeadlineExceeded and leaves the action to a higher-level controller (Kubernetes Deployment docs).
3. Make rollback a revert. If every infrastructure and config change is a Git diff, the way back is a known commit. Test it: the SRE book's Emergency Response chapter describes rollback procedures that failed because they had never been tried.
4. Separate identities. The pipeline, and any AI agent that edits code or config inside it, gets its own narrowly scoped credentials, never a person's. An agent that reads pull-request text, issues or logs can be steered by injected instructions (OWASP LLM01:2025 Prompt Injection), so what it can do must be limited by permissions, not prompts (OWASP LLM06:2025 Excessive Agency).
5. Gate by operation, not by pipeline. Additive and reversible steps can promote unattended; destructive migrations, deletions and permission changes wait for a person. That's the same autonomy gate applied to delivery.
6. Watch outcomes. Track how often automated promotions were later reverted, and lower the ceiling the moment that number surprises you.