Your agent didn’t hallucinate; it exceeded its authority
Our take
The recent wave of AI agent adoption has rightfully focused on safety—preventing harmful outputs and protecting sensitive data. However, as Patel’s article so clearly articulates, this focus has largely overshadowed a critical, and arguably more pressing, concern: business authority. Content filters are a necessary first step, but they don't determine whether an agent is *allowed* to issue a refund, modify production systems, or commit a company to external agreements. Building a Streamlit UI for My LangGraph AI Agent highlights the practical challenges of building and deploying these agents, and it’s clear that simply ensuring they function correctly isn’t enough; we need robust governance frameworks to define their operational boundaries. The increasing prevalence of AI-agent-related incidents, as reported in the Cloud Security Alliance survey, underscores the urgency of addressing this gap—enterprises are rapidly deploying agents without fully understanding or controlling their potential impact. This isn't about AI failing to reason; it's about failing to define the scope of its operational authority.
The concept of an “Agent Authority Contract” – a machine-enforceable record of delegated permissions – is a powerful one. It moves beyond reactive safeguards and establishes a proactive framework for defining what agents *can* and *cannot* do. Patel’s proposed seven questions provide a solid foundation for this contract, covering ownership, permissible actions, data access, materiality limits, escalation triggers, reversibility, and expiration. This approach aligns with the broader trend in AI governance towards specifying clear boundaries and accountability. It's worth noting that even Pinterest, a leader in AI-powered personalization, is grappling with securing their infrastructure, as evidenced by their Resource Provisioner Pipeline, How Pinterest Secures AWS Infrastructure at Scale with a Centralized Terraform Pipeline. Their focus on infrastructure security mirrors the need for a similar level of rigor when it comes to defining agent authority. The distinction Patel draws between guardrails (behavioral constraints) and decision rights (legitimate authority) is crucial; the former prevents *how* an agent acts, while the latter defines *what* it is authorized to do.
The shift towards a proportional authorization model—allowing low-risk actions autonomously, requiring approval for high-risk actions, and escalating unexpected behavior—is a pragmatic response to the scalability challenges of constant human oversight. As OpenAI has demonstrated, agentic workflows can dramatically increase the volume of code changes, making it increasingly difficult to manually review every decision. Presentation: Keeping ChatGPT Fast as AI Development Accelerates explores the technical challenges of maintaining performance in this environment, highlighting the need for efficient decision-making processes. This underscores the importance of automating authority decisions at runtime, based on dynamic factors like transaction context and potential impact. The emphasis on measuring authority calibration—tracking override rates, escalation precision, and unauthorized action attempts—is a welcome addition, transforming authority from a theoretical concept into a measurable operating variable.
Ultimately, Patel’s piece serves as a stark reminder that the pursuit of AI autonomy should not come at the expense of responsible governance. The focus is shifting; it’s no longer enough to ensure an agent produces safe and technically valid outputs. We must now prioritize defining the scope of its authority and establishing clear mechanisms for oversight, accountability, and reversal. As AI agents become increasingly integrated into critical business processes, the question isn't just *can* they do something, but *should* they, and who is ultimately responsible when things go wrong? The development and adoption of Agent Authority Contracts, and the frameworks supporting them, will be a defining factor in determining the long-term success and trustworthiness of AI-powered automation.
Content filters can block unsafe output. They cannot tell you whether an agent was authorized to issue that refund, touch that production system, or commit the company to an external action. Those are different problems, and most enterprises are only solving the first one.
An AI agent can follow its instructions perfectly and still take an action the business never sanctioned.
In commerce environments, I have seen this pattern emerge in practical ways. A service workflow calculates the correct refund amount but lacks a boundary preventing credits above what the business approved for autonomous action. An order agent correctly applies a requested change but overlooks a financing or fulfillment condition. A procurement agent identifies the lowest-cost supplier, but nobody has defined whether it can accept contractual terms or only recommend the option.
The agent keeps working. The problem may not surface until something downstream breaks.
These are not necessarily AI reasoning failures. They are failures to separate technical capability from business authority.
As enterprises move from copilots that recommend to agents that call tools and trigger workflows, every production agent needs explicit decision rights: What it may execute, what requires approval, what it may only recommend, and what it must never touch.
Guardrails remain necessary. But a guardrail is not an authority model.
Safety controls and decision rights solve different problems
Early gen AI controls screen harmful content, protect sensitive information, validate responses, and constrain tool behavior. That work matters.
Decision rights answer a different question: Even when an action is safe and technically valid, is this agent authorized to take it on behalf of the enterprise?
That governance gap is becoming harder to ignore. In April 2026, a Cloud Security Alliance survey found that 65% of respondents had experienced an AI-agent-related incident in the prior year, while 82% had discovered previously unknown agents operating in their environments. The survey involved 418 IT and security professionals and was sponsored by Token Security.
The findings illustrate how quickly agent activity can outpace the visibility and ownership structures built for conventional software.
The World Economic Forum’s May 2026 playbook reflects this shift. It introduces an Agent Capability and Authorization Profile designed to make delegated actions auditable, enforceable and accountable.
Guardrails constrain behavior. Decision rights define legitimate authority.
Give every production agent an authority contract
Before an agent receives access to enterprise tools, it needs a machine-enforceable record of exactly what authority the business has chosen to delegate. Call it an Agent Authority Contract.
At minimum, that contract should answer seven questions:
Who owns the outcome? Name a human or business role, not another system.
What may the agent do? Read, recommend, write, or commit?
Which systems and data may it reach?
What materiality limits apply? Define dollar thresholds, record counts, customer scope, and operational impact.
What triggers escalation? Uncertainty, anomaly, sensitive data, or potential impact?
Can the action be reversed, and who can reverse it?
When does the authority expire, and how is it withdrawn?
Access control determines whether an agent can reach a system. The authority contract determines whether it may take a specific action in the current context.
Those are not the same check.
Singapore’s updated Model AI Governance Framework for Agentic AI draws a similar distinction. It treats access controls, behavioral guardrails, and human approvals as separate controls and ties oversight requirements to action scope, reversibility and potential impact.
Resolve every consequential action into four outcomes
A working decision-rights model should map every consequential agent action to one of four results.
Allow
Low-risk, bounded, and reversible actions run autonomously.
Examples include retrieving approved information, classifying an inbound request, or updating a non-material field. The agent acts without prior review because the potential impact is limited and the action can be reversed.
Approve
The agent prepares or initiates the action, but execution waits for authorization from a human or deterministic policy service.
This category covers payments, production changes, and actions that materially affect a customer, employee, or third party.
Recommend
The agent analyzes, ranks, drafts, or proposes. A named human makes the final decision.
Use this outcome when contextual judgment matters or when the legal, financial, or individual impact makes automated execution unacceptable.
Deny
The action remains outside the agent’s authority regardless of its confidence.
Deleting critical production data, making a final employment decision or overriding a mandatory compliance control should remain in the Deny category even when the agent’s underlying reasoning appears correct.
One point gets missed consistently: Deny must be enforced outside the system prompt.
A natural-language instruction telling an agent not to do something is not a technical boundary. It is a suggestion.
Make authority decisions at runtime
Static configuration cannot cover every situation.
A small service credit might be allowed under normal conditions but require approval when the amount crosses a threshold, the account is under investigation, or the request involves a regulated customer.
A practical runtime sequence looks like this:
The agent proposes an action.
A policy layer evaluates the agent’s identity, delegated principal, requested tool, data involved, transaction context, and potential impact.
The policy returns Allow, Approve, Recommend, or Deny.
The system records the authority decision, resulting action and outcome.
Operational telemetry expands, narrows, or revokes the agent’s authority over time.
In enterprise commerce, the most dangerous AI mistake is not always a false answer. It can be a technically correct action the agent had no business taking.
A refund may be accurate but exceed an approval limit. An order change may match the customer’s request but invalidate a financing condition. A delivery promise may reflect available inventory while overlooking a carrier constraint applied an hour earlier.
The agent may not have failed to reason. The enterprise failed to define where its authority stopped.
Human oversight should target exceptions, not everything
Requiring human approval for every agent action looks conservative. At scale, it can quickly degrade into rubber-stamping.
When reviewers approve thousands of routine actions, attention declines and genuine exceptions become harder to identify. Singapore’s framework acknowledges that continuous human oversight of every agent workflow becomes impractical at scale and recommends meaningful checkpoints for higher-risk or irreversible actions.
Proportional authorization is the more workable model.
Low-risk actions run within narrow boundaries. High-risk or irreversible actions require approval. Unexpected behavior triggers escalation. Any consequential action without a defined authorization policy is denied by default.
The objective is not maximum autonomy. It is the highest level of autonomy the enterprise can observe, govern and reverse responsibly.
Measure whether authority is calibrated
Once agents are in production, response accuracy becomes too narrow a success metric.
Enterprises should also track:
Override rate: How often do humans reject or materially change what the agent decided?
Escalation precision: Does the agent surface genuinely risky cases, or does it return routine work to people?
Unauthorized-action attempts: How often does the agent try to exceed its system, data, or action scope?
Business-impacting error rate: How often do authorized actions produce financial, compliance, operational, or customer harm?
Decision latency: Are approval requirements managing risk, or slowing down automation that was already safe?
These measures turn authority into a governed operating variable.
Consistently reliable performance may justify expanding bounded authority. Frequent overrides, escalation failures, or policy violations should narrow it.
The governance gap is not in the model
Model safety, output controls, and secure tool use all matter. Enterprises should continue investing in them.
But none of those controls can answer who delegated authority, how much was transferred, under what conditions it applies, or who owns the result when something goes wrong.
An Agent Authority Contract can.
Before asking how autonomous an AI agent can become, the more useful question is: What is the enterprise actually prepared to delegate, and how will that delegation be enforced, observed, and withdrawn?
The agent demo works. That is not the hard part anymore.
Nixal Patel is a product leader. The views expressed are his own
Read on the original site
Open the publisher's page for the full experience