Most AI governance programs were designed for software that changes only when someone ships a release. Agentic AI does not behave that way. An agent’s effective behavior is assembled at runtime from a model, a system prompt, a tool set, and retrieved context — and any of those can shift while the compliance team is asleep. The emerging answer is governance-as-code: controls written as versioned, testable artifacts that execute inside the agent’s request path instead of in a quarterly review meeting. Here is what that looks like in practice, and where a team should start.

Why Bolted-On Governance Fails Agentic Systems

Classic compliance targets a stable artifact. You review a model card, approve a deployment, and assume the system that runs next month is the system you approved. Agents break that assumption three ways.

First, they act rather than answer. An agent with tool access can move money, send email, or modify production records, so the interesting risk lives in individual tool calls that no release process ever sees. Second, their behavior is contextual. The same agent can be harmless with one retrieved document and reckless with another, which means a one-time evaluation cannot characterize it. Third, they compose. Agents call other agents and services, so a control applied at your front door says nothing about what happens two hops downstream.

Governance that runs at deployment time is always looking backward. Agentic systems need a control plane that runs at execution time.

What Governance-as-Code Actually Means

Governance-as-code applies the discipline of infrastructure-as-code to policy. Rules stop being PDFs and become files in a repository: versioned, reviewed through pull requests, tested in CI, and deployed like any other artifact.

The pattern is well established. Open Policy Agent, the open-source policy engine, decouples policy from application code and evaluates rules written in its Rego language against structured input — the same request the agent is about to make. Instead of scattering permission checks across every service, you write the policy once, in one place, and every component calls the same engine.

Applied to agents, a policy file states things like: finance agents may not call payment tools above a threshold without human approval; tool calls that touch EU personal data must be logged with full input and output; models outside the approved procurement list are denied at the gateway. When a regulation changes, you open a pull request, not a project plan.

The Four Control Points That Matter

An agentic architecture has four natural places to enforce policy, and a mature deployment uses all of them.

The tool boundary. Every consequential action an agent takes is a tool call, which makes the tool layer the single best enforcement point. A policy engine sitting between the agent and its tools can approve, modify, or deny each call in milliseconds, with deny-by-default as the posture.

Non-human identity. Agents need their own credentials, scoped to exactly the tools they are authorized for — never a shared service account, and never a human’s token. Short-lived credentials make every action attributable to a specific agent and owner.

The evidence trail. Each decision — input, policy version, outcome — is written to an append-only log. When a regulator asks what your agent did in March, the answer is a query, not a forensic reconstruction.

The change gate. Prompts, tool sets, and model versions are artifacts in version control. CI runs policy tests and evaluation suites against every change, so a prompt edit that unlocks a prohibited behavior fails the build before it ships.

Mapping Controls to the AI Act and the NIST Framework

You do not have to invent the control catalog. The EU AI Act imposes transparency duties under Article 50, logging and record-keeping obligations for high-risk systems, and a compliance timeline phasing in through 2026 and 2027 — the AI Act portal maintained by the Future of Life Institute tracks obligations and implementation dates in detail. In the United States, the NIST AI Risk Management Framework, including its Generative AI Profile, structures the problem through its GOVERN function: accountability, policy, and monitoring defined as an operating process rather than a document.

Both regimes reward the same architectural property — the ability to demonstrate, continuously and with evidence, what your systems did and why. That is precisely what a policy-as-code layer produces as a byproduct of running.

Starting Without Slowing Agents Down

The objection is always latency and rigidity. Both are manageable. Keep hot-path policies small and fast — allow and deny decisions, not language-model judgment — and reserve heavyweight review for escalation. Write policy tests alongside the rules so failures surface in CI rather than in production. Review policy metrics the way you review error rates, because a rule that quietly blocks a third of legitimate calls is its own incident. And treat policy as a product with an owner, because unowned rules decay into exactly the kind of friction teams route around.

Governance-as-code is not a heavier compliance program. It is the version of governance that survives contact with systems that act on their own.

References

• Open Policy Agent — open-source policy engine
• EU Artificial Intelligence Act — obligations, timeline, and compliance resources
• NIST AI Risk Management Framework (AI RMF)

Research and written by Peter Jonathan Wilcheck