A practical deployment checklist for self-hosted AI agents

An AI agent is not ready for production because it answered ten prompts correctly on a laptop. A deployed agent is a software service with credentials, state, external dependencies, and permission to affect other systems. Self-hosting gives you more control over those pieces. It also makes them your responsibility.

This checklist is for agents that run on infrastructure you control, whether that means one virtual machine, Docker Compose, Kubernetes, or a private cloud. It applies whether the model is local or reached through an API.

One boundary matters from the start: self-hosted runtime does not necessarily mean self-hosted inference. If the agent calls a hosted model, the prompts and any tool results included in them still leave your environment under that provider's data terms. Self-hosting also does not make a system HIPAA-, SOC 2-, or GDPR-compliant by itself. It changes where controls can be applied; it does not supply the controls.

1. Write down the production boundary

If this cannot be described clearly, the deployment is not ready. Infrastructure will not repair an undefined authority model.

2. Give the runtime its own identity

Do not run the agent with a founder's API token, an administrator's database account, or a shared SSH key. Give the service a dedicated machine identity and grant only the permissions needed for its current job.

Keep read, create, update, send, and delete operations separate where the upstream system allows it. A research agent that reads documents should not inherit permission to remove them. OWASP describes unnecessary tools, permissions, and autonomy as forms of excessive agency. That framing is useful because the safest permission is the one the agent never receives.

For MCP-connected tools, enforce authorization at the server, not only in the prompt or client UI. The official MCP authorization guidance recommends short-lived tokens, audience validation, least-privilege scopes, encrypted token storage, and HTTPS in production. It also warns against logging credentials. Those are server controls; instructions such as "only access the current customer's records" are not an authorization layer.

3. Isolate the process

Assume an agent process may eventually receive a hostile prompt, compromised document, malformed tool response, or vulnerable dependency. Limit what that process can reach.

Run it as a non-root user. Mount only required directories, and make them read-only unless the workload genuinely writes there. Drop unused Linux capabilities. Put temporary work in a dedicated volume. Restrict outbound network access to required model, tool, identity, and telemetry endpoints. Keep the agent away from the Docker socket, cloud instance metadata, host credentials, and unrelated internal services.

Docker rootless mode runs both the daemon and containers without root privileges, reducing the impact of some daemon or runtime vulnerabilities. It is a useful layer, not a complete sandbox. The MCP security guidance similarly recommends sandboxing local servers with restricted filesystem, network, and system access because local MCP servers run with the client's privileges.

4. Keep secrets out of prompts, images, and logs

Inject secrets at runtime from a secret manager or a platform secret mechanism. Do not bake them into an image, commit them in configuration, pass them through model-visible context, or print them during startup diagnostics. Docker Compose has a dedicated secrets mechanism for granting selected services access to sensitive material.

Use separate credentials per environment. Rotate them on a schedule and after an incident. Test rotation before launch, including what happens to an in-flight run. If an agent serializes pending work for later resumption, inspect that stored state as carefully as a database record: it may contain tool arguments, user data, approval decisions, and previous model output.

5. Separate durable state from the worker

An agent worker should be replaceable. Store job status, approval state, idempotency keys, and required conversation state in a durable system rather than only in process memory or a container filesystem.

Every side-effecting tool needs a stable operation ID. If the worker crashes after sending an email but before recording success, a retry must not send the same email again. The same applies to payments, CRM updates, tickets, calendar invitations, and file changes. Design tools so repeated calls with the same operation ID are safe, or check the destination's state before retrying.

Set explicit timeouts and bounded retries. Retry transient network failures with backoff; do not retry authorization failures, invalid input, or a human rejection as though they were network glitches. Send exhausted work to a visible failure queue with enough context for an operator to recover it.

6. Put hard limits around the loop

These limits contain accidental loops, retry storms, and unexpectedly broad requests. A limit should fail closed with a clear status, not silently turn a partial run into a reported success.

7. Build health checks that measure the right thing

A process can be alive while unable to accept work. Expose separate checks for liveness and readiness. Liveness should answer whether the process is stuck and requires a restart. Readiness should answer whether this instance can safely receive new work, including required configuration and durable-state access.

The Kubernetes probe documentation distinguishes these behaviors: failed readiness removes a pod from service traffic, while failed liveness can restart it. It also warns that badly designed liveness checks can cause cascading failures. Do not restart every agent worker because a model provider has a short outage. Mark it unready, queue work, or degrade the affected capability.

8. Observe decisions and side effects without creating a data leak

For each run, record a correlation ID, tenant or account ID, agent and configuration version, model identifier, tool names, latency, result status, approval events, and the external operation IDs created. Track tool errors, approval rejection rates, queue age, token use, retries, and runs stopped by budget limits.

Full prompts and tool results can make debugging easier, but they may also contain personal data, customer records, credentials, or confidential documents. Default to structured metadata and redacted payloads. Grant trace access narrowly and set retention deliberately. OpenAI's Agents SDK, for example, documents that traces may include sensitive model and tool inputs and outputs and provides a configuration to control that behavior. Check the equivalent setting in the runtime you use.

Alerts should describe an operator action. "Error rate is high" is less useful than "CRM writes have failed for 10 minutes; new write jobs are paused; read-only search remains available."

9. Make approval durable and specific

Approval is part of the system, not a chat message that says "looks good." Pause before a sensitive tool executes. Show the reviewer the exact action, destination, affected records, and meaningful diff. Bind the decision to those arguments, expire it, and record who approved it.

If a process can wait hours for a reviewer, persist the paused state and make resumption safe after a deploy or restart. Rejecting an action should end or reroute that action cleanly; it should not prompt the agent to find a different tool that achieves the same blocked result.

For a deeper design, see how to scope AI agent permissions and human approval. If you need the runtime and tool layer built together, our AI agent systems service covers the agent architecture, while systems integration covers reliable connections to the software it operates.

10. Plan updates, rollback, and recovery before launch

Pin the agent code, prompts, tool schemas, dependencies, and model configuration as versioned release inputs. A model alias or third-party API can change behavior even when your repository does not. Record what ran so an incident can be reproduced.

Deploy to a staging environment first. Use a canary or small traffic slice for production changes. Keep the previous known-good release available, and make database migrations backward-compatible across the rollback window. A rollback that restores code but cannot read the new state format is not a rollback.

Back up durable state and configuration. Then perform a restore drill. A green backup job only proves that files were written somewhere; a restore drill proves that an operator can rebuild the service and resume or safely close pending work.

11. Prepare for incidents

The operator needs a kill switch that stops new runs and disables side-effecting tools without taking away access to logs or pending approvals. Document how to revoke service credentials, drain the queue, identify affected external records, and notify the right owners.

Define incident classes before one happens: data exposure, unauthorized action, duplicated action, degraded model quality, unavailable provider, runaway cost, and compromised dependency. Each needs a different response. Restarting the container is not an incident plan.

NIST organizes AI risk work around Govern, Map, Measure, and Manage in its voluntary AI Risk Management Framework. The practical point is that deployment is a managed lifecycle, not a one-time server setup. Ownership, measurement, response, and review continue after the first successful run.

The launch gate

If any answer is no, the agent may be ready for a controlled pilot, but it is not ready for unattended production work. Build the missing control, test it, and make the failure visible before widening access.