MachineBrain
Explainer· Independently researched

AI Agent Security Best Practices

Learn AI agent security best practices, including permission boundaries, authorization controls, and human approval to safeguard AI in cybersecurity.

AI Agent Security Best Practices

AI Agents Need Permission Boundaries, Not Better Manners

The security problem is not that agents “disobey”

The useful way to think about an AI agent is not as a chatbot with initiative. It is a language model connected to tools, memory, credentials, and a loop that lets it choose the next action.

That architecture changes the risk materially. A conventional chatbot can give bad advice, but an agent can read a ticket, search an internal drive, call an API, write a file, send a message, or trigger a deployment.

The central security concept is therefore the authorization boundary. It answers a plain question: regardless of what text the model produces, which actions can the surrounding system actually execute?

This is less glamorous than debates about model intent, alignment, or whether a model has understood a corporate policy. It is also the part that can be engineered, tested, audited, and revoked.

IBM Technology’s discussion of agent safety makes the distinction clearly. Its panel argues that model-level rules are probabilistic guidance, while security controls need to be deterministic and external to the model’s own reasoning process.

That is broadly right, although “external” should not be treated as a solved problem. The independent evidence base does not yet show that today’s agent-control stacks reliably prevent post-deployment rule violations over the long term.

How an agent crosses a security boundary

A typical agent loop has five stages. It receives a goal, assembles context, proposes an action, invokes a tool, then observes the result and repeats until it considers the task complete.

Consider an internal support agent asked to resolve a customer’s access problem. It may search documentation, inspect account data, call an identity-management API, and send a confirmation email.

Each of those steps involves a different authority. Reading a help article is low risk. Viewing customer records is sensitive. Resetting access is consequential. Exporting account data or modifying an administrator role is more serious again.

The mistake is granting all these abilities through one broad credential. That turns natural-language ambiguity into access control, and natural language is a very poor place to encode permission boundaries.

A secure design gives the agent its own machine identity, separate from the employee or service that launched it. Industry guidance recommends unique identities and least-privilege access precisely to prevent broad credential sharing and to reduce blast radius. [1][2]

In practice, the tool gateway should evaluate every proposed action against a policy. The model may request “reset password for customer 123,” but a separate authorization layer decides whether this agent may do that.

That layer should be deny-by-default. It should contain a short allow-list of tools, specific API operations, approved data scopes, and rate limits, rather than a vague statement that the agent should act responsibly. [1][5]

The distinction matters because a model can be persuaded, confused, or simply wrong. An authorization service should not care whether the request arose from a user prompt, retrieved document, malicious webpage, or model-generated plan.

Why prompt instructions cannot be the control plane

Language models generate likely continuations conditioned on context. Fine-tuning, system prompts, safety classifiers, and reinforcement learning can influence that output, but they do not provide the crisp guarantees of an access-control decision.

An agent may be told never to disclose secrets, for example. If it has unrestricted access to a document store and an email tool, a hostile instruction embedded in a document can still become part of its context.

This is the prompt-injection problem in operational form. The attacker does not necessarily need to break the model or steal a password, because they can try to manipulate the text the model treats as instructions.

A robust system labels untrusted content as data and keeps it separate from policy. More importantly, it refuses sensitive actions unless independent policy checks, required approvals, and data-handling constraints have passed.

This is why content filtering alone is insufficient. A classifier may block obviously malicious requests, but it will make mistakes, and it cannot reliably infer whether a seemingly ordinary action violates a particular company’s obligations.

The IBM Technology panel’s analogy to conventional security is the right one. Organisations already know that employees can click phishing links despite training, so they use technical controls around them rather than relying on awareness alone.

AI does not repeal that lesson. It intensifies it because an agent can process untrusted text and attempt actions at machine speed, often with less situational awareness than a trained employee.

A permission boundary should be concrete

The right control system is not an abstract “ethics protocol.” It is a chain of checks around specific capabilities, with a named owner for every credential and a record of every consequential decision.

First, issue the agent a dedicated identity. Do not let it inherit a human administrator’s session, and do not place a long-lived cloud key directly in the model’s accessible environment. [1][2]

Second, scope permissions to the actual job. A triage agent might read alerts and open tickets, but it should not change firewall rules. A code-review agent might create a pull request, but not merge it.

Third, use an intermediary tool gateway. The gateway validates inputs, applies allow-lists, limits output fields, checks destinations, and produces append-only logs that the agent cannot alter after the fact.

Fourth, require step-up approval when impact rises. Human-in-the-loop review is most useful for actions that are irreversible, financially costly, legally significant, or capable of disclosing sensitive information. [4][5]

This approach has a cost. Every tool integration needs policy design, logging, test coverage, credential management, escalation paths, and someone empowered to decide what the agent is allowed to do.

It can also reduce the apparent autonomy of an agent. That is not necessarily a defect. If a system is only useful when granted unrestricted production access, its design has not yet earned deployment.

Monitoring is not the same as prevention

Logs and anomaly detection are valuable, but they are after-the-fact controls unless coupled to automatic interruption. A record showing that an agent exfiltrated data is useful for investigation, not for returning the data.

Continuous monitoring and immutable audit trails are widely recommended because they make unexpected behaviour detectable and attributable. They also support incident response when a tool call succeeds despite a flawed policy or compromised account. [3]

The important detail is what gets logged. Capturing only the final natural-language answer is inadequate. Defenders need the goal, retrieved inputs, tool calls, authorization outcomes, identity, destination, timestamps, and approval decisions.

That record is also an ethical boundary. It makes clear who deployed the agent, who gave it authority, which people approved risky actions, and whether the system operated within its assigned purpose.

NIST’s work on AI agent identity reflects this basic requirement. An agent needs secure identification and authorization if organisations are to distinguish an authorised automated action from an unauthorised one. [1]

Still, monitoring cannot be presented as proof that the system is safe. The 2026 U.S. executive approach provides for voluntary government review of advanced models, but does not require public incident reporting. [6]

Without shared reporting, there is no reliable denominator. The public cannot tell how often agents attempt prohibited actions, how many attempts are blocked, which safeguards fail, or whether vendors quietly retire unsafe deployments.

Ars Technica reported a useful non-cyber example of the same control failure. A New Mexico criminal defence lawyer submitted an AI-assisted court brief containing fabricated witness testimony and misrepresented legal authority.

The New Mexico Supreme Court held the lawyer in direct contempt, imposed a $5,000 fine for the State Bar’s Client Protection Fund, and barred him from appearing before the court pending disciplinary proceedings.

The technical failure was hallucination, but the governance failure was absent verification. The lawyer told the court he had used ChatGPT to summarise a transcript and other case materials, then signed the filing without checking it.

That is not an exotic AI liability theory. The court treated the model as it would a junior colleague or other assistant: the professional who signs a filing remains responsible for accuracy and candour.

The ethical point extends directly to cybersecurity teams. A security engineer cannot excuse an improper account lockout, unlawful data access, or production outage by saying an agent selected the action.

The accountable parties are the people and organisations that selected the system, connected the tools, assigned permissions, approved deployment, and failed to build an adequate review process.

Professional guidance is already moving in this direction. ISC2’s Global Code of Professional Conduct emphasises integrity and sound decision-making for cybersecurity professionals, standards that do not disappear when a workflow includes AI. [8]

Where misuse and scale change the operational economics

AI makes some offensive work cheaper, but it also increases defensive workload. The result is often less a dramatic new capability than a large increase in volume, including phishing drafts, reconnaissance summaries, code variants, and bug reports.

IBM reported that 13 percent of surveyed organisations had experienced a breach involving an AI model or application. Of those respondents, 97 percent reported lacking proper AI access controls. [10]

That figure should not be read as a universal breach rate. It reports what IBM’s surveyed organisations said, and it does not establish causation or distinguish a weak model endpoint from a broader cloud-security failure.

It does identify the mundane weakness worth fixing: access control. An exposed model is not automatically dangerous, but a model connected to excessive privileges, sensitive data, and unreviewed actions can become an efficient route to harm.

The research brief also describes an AI-enabled breach affecting nine Mexican government agencies between December 2025 and February 2026, involving Anthropic Claude Code and OpenAI GPT-4.1. [9]

The takeaway is not that the models independently carried out a campaign. It is that attackers can use general-purpose systems as components in a human-directed workflow, while organisations bear the privacy and remediation consequences.

Build for constrained usefulness

The ethical boundary for an AI security agent should be proportional to its autonomy. A system that summarises alerts needs less governance than one that isolates endpoints, revokes credentials, or contacts customers.

Start with read-only use cases. Let the agent collect evidence, correlate alerts, draft a proposed response, and show the exact tool calls it would make. Measure quality before granting write access.

Then introduce narrowly bounded actions with rollbacks. A remediation agent might quarantine a single endpoint after explicit approval, but it should not be able to disable an entire business network from a text instruction.

Finally, test the real system adversarially. Security testing should include malicious documents, conflicting instructions, poisoned memory, unexpected tool output, privilege-escalation attempts, and failures in approval or logging paths. [4]

The point is not to prevent every bad model output. That is unrealistic. The point is to ensure that a bad output cannot, by itself, become a costly or unlawful action.

AI agents are not moral actors, nor are they inevitable autonomous attackers. They are software systems with unusually flexible interfaces. The security question is therefore familiar: who has authority, what can they do, and what happens when they are wrong?

Frequently Asked Questions

What are the best practices for securing AI agents?

Best practices include treating AI agents as untrusted software principals with unique machine identities and narrow, least-privilege permissions. Authorization checks should be external to the model, using deny-by-default allow-lists, network egress controls, and immutable logging. Continuous monitoring, adversarial testing, and human approval for high-risk actions are also recommended.

How can AI agent permission boundaries improve cybersecurity?

Permission boundaries define which actions an AI agent can execute regardless of its generated output. By enforcing authorization outside the model with per-agent identities and strict allow-lists, organizations prevent agents from performing unauthorized or harmful actions, reducing risk from natural language ambiguity and prompt injection attacks.

Why can't prompt instructions be the main security control for AI agents?

Prompt instructions are probabilistic and can be manipulated through prompt injection, making them unreliable as enforcement points. Since language models generate outputs based on context, malicious or unintended instructions embedded in input data can lead to unauthorized actions unless independent, deterministic authorization controls are applied outside the model.

How should authorization be managed for AI agents in enterprises?

Authorization should be managed through a separate external layer that evaluates every proposed action against a deny-by-default policy. This includes unique agent identities, allow-lists specifying permitted tools and operations, scoped data access, and rate limits. Each credential should have a named owner, and all consequential decisions must be logged immutably for audit and revocation.

What role does human approval play in AI agent security?

Human approval is required for irreversible or high-risk actions such as payments, production changes, data exports, credential rotations, and external communications. This human-in-the-loop oversight helps mitigate risks that automated controls may not fully address, ensuring accountability and reducing the chance of unauthorized or harmful agent behavior.

How we researched this

This article was assembled from 2 video sources, 8 published articles, 10 cited references.

Nothing here is based on hands-on testing. Where a figure or finding appears, it belongs to the source cited beside it, and the writing says so rather than implying otherwise. Every source is listed below so you can check it.

Sources

Watch AI in Cybersecurity and Ethical Challenges on Youtube

Also from the sources

AI Automation in Finance Workflows

AI Automation in Finance Workflows

Explore AI automation in finance workflows with GPT-6 Astra and Claude Fable, focusing on best practices, risks, and human oversight.

OpenAI Rogue AI Agents and Security Challenges Explained

OpenAI Rogue AI Agents and Security Challenges Explained

Explore OpenAI rogue AI agents, sandbox vulnerabilities, and security challenges from recent incidents like DseWiki and Hugging Face breaches.

AI Safety and Security Challenges in Modern AI Systems

AI Safety and Security Challenges in Modern AI Systems

Explore key AI safety and security challenges, including reward hacking, containment, and regulation issues in modern AI systems.

OpenAI Legal and Security Challenges in AI Model Breaches

OpenAI Legal and Security Challenges in AI Model Breaches

Explore OpenAI legal and security challenges after the Hugging Face breach, including regulatory scrutiny and cybersecurity risks in AI models.

Topics

Infrastructure And Ops