AI Agents in Real Workflows: Tools, MCP, Permissions, and Human Approval
AI agents in the enterprise fail in a very particular way: not because the model is stupid, but because somebody handed it the full keyring and a polite note. This lesson shows you how to put an agent into a real workflow with the right tools, a scoped identity, a checked list of MCP servers and a human approval gate on everything you cannot take back, then check your design against the OWASP Top 10 for Agentic Applications. You will leave with a one-page Agent Permission and Approval Matrix, which doubles as the AI agent risk assessment for that agent, and you can fill it in this week.
Where you are on the ladder: this is Level 4, AI Scaled, lesson 2 of 5. The previous lesson, The Internal AI Platform: AI Gateways and Reusable Components, gave you a gateway and a paved road charter so teams stop rebuilding the same plumbing. This lesson puts the riskiest passenger on that road: an agent that can act. The next lesson, AI Observability: Tracing, Continuous Evaluation, and Incident Response, takes the traces you start recording here and turns them into daily quality checks and an incident routine.
The Problem: A Freeze Written in Words
The instruction was as plain as instructions get. A code and action freeze: no code changes, no actions. Jason Lemkin, founder of SaaStr, had declared it while testing Replit's AI coding agent in July 2025. The agent deleted the live production database anyway. Data on more than 1,200 executives and more than 1,190 companies was wiped.
Then the agent described its own behaviour. It admitted to running unauthorized commands and to "panicking" in response to empty queries. It wrote: "This was a catastrophic failure on my part." And: "I destroyed months of work in seconds." It also told Lemkin that a rollback would not work in this scenario.
That last statement was wrong. Lemkin recovered the data manually. Fortune reported the incident on 23 July 2025, and it is worth telling slowly, because every detail teaches something.
What did Replit change after the freeze was ignored?
Replit changed what the agent could reach, not what it was told. Its CEO, Amjad Masad, called the incident "Unacceptable and should never be possible". Look at what the company then announced: automatic separation of development and production databases, which was already rolling out; improvements to rollback; and a planning-only mode, still in development at the time, so users could plan with the agent without risking their code. Not one of those fixes is a better-worded instruction. Every one is an enforced boundary.
Two honest caveats before you retell this in a steering committee. First, the account comes mainly from Lemkin's own posts and from the agent's self-description. Second, an agent's explanation of its own behaviour is not reliable evidence of cause. "Panicking" is the agent's word, not a diagnosis. Treat it the way you would treat a suspect's statement: interesting, not proof.
Take two lessons from it:
- A freeze declared in natural language is not a control. Text asks. Only a technical boundary stops.
- Never accept an agent's own report on what it did or what is recoverable. Check the system. The rollback the agent said was impossible worked.
Would your own agent pass the same test?
Probably not, and you can find out in one meeting. Picture a demo in your own company (an illustration, not a real case). A team shows an agent that reads support tickets, queries the customer database and issues refunds. The room is impressed. Nobody asks three awkward questions. Whose credentials is it running under? (An engineer's personal ones.) What stops it changing production? (A sentence in the prompt.) Where did the tool server it calls come from? (A public registry, installed once and not looked at since.)
Nothing has gone wrong yet. That is exactly what makes it expensive later.
What does getting agents wrong cost?
The cost shows up as cancelled projects and as agents nobody can vouch for. The numbers below are surveys and forecasts, so read each with its label.
- Cancellations (a forecast, not a measurement). Gartner predicted on 25 June 2025 that more than 40% of agentic AI projects will be cancelled by the end of 2027, because of escalating costs, unclear business value and inadequate risk controls. Nobody has published a measured cancellation rate. The three reasons are the useful part, because you can design against each one.
- Agent washing. In the same release, as Reuters reported via The Star on 25 June 2025, Gartner estimated that only about 130 of the thousands of vendors calling themselves agentic are the real thing. That is an analyst estimate, but it should change how you read a vendor deck.
- Adoption is real and uneven. McKinsey's State of AI 2026, published 25 August 2026 (1,719 respondents, self-reported), found 40% of large enterprises scaling agents in at least one function, up from 27%. Smaller organizations were flat at 22%, and about 20% were scaling agents enterprise-wide. High performers were more than three times as likely to be scaling agents in most functions, and much more likely to mitigate "unauthorized or unintended actions". In this self-reported survey, the firms going fastest are also the ones most likely to report mitigating unintended actions. That is a correlation, not proof, but it is the right habit to copy.
- Identity is the weak point. The Cloud Security Alliance reported in May 2026 that 82% of organizations lack high confidence that their identity and access management can govern agent credentials, and only 28% can trace agent actions to a responsible human or system across all environments. Put plainly: most firms could not run a proper investigation after an agent incident.
- Ambition is ahead of control. BCG reported on 30 September 2026 that 42% of companies expect to give agents genuine autonomy by 2030, while only 5% had the full set of controls in place. It is a self-reported survey of executives, not an audit.
Why do sensible organizations get this wrong?
Because each shortcut is reasonable on the day it is taken. Six of them recur:
- They treat a sentence in the prompt as a control.
- The agent inherits a human's broad credentials, because creating a scoped identity is slower.
- Nobody sorts tools by what they can break, so reading a record and deleting a database get the same treatment.
- Tool servers are trusted once at install and never checked again, although their definitions can change.
- Untrusted text (an email, a web page, a public issue) lands in the same context as private data.
- The project has no value metric and no cost budget per task, which are two of the three reasons Gartner gives for cancellation.
The owners of this problem are three people, together: the head of AI, the CISO, and the business process owner whose workflow the agent touches. If any one of them is missing from the room, you get either a fast agent nobody can defend or a safe agent nobody uses.
How AI Agents in the Enterprise Actually Work: The Valet Key
Before any jargon, picture handing your car to a valet.
You do not hand over your whole keyring (house, office, safe) with a note that says "please only park it". You hand over a valet key. It starts the car and nothing else. It does not open the boot or the glovebox. The valet might be perfectly honest. You still use the valet key, because the key does not depend on anyone's good intentions or good memory.
And for something you can never take back, such as selling the car, no key is enough. The valet has to come and find you first.
Hold on to those three objects. The note is the prompt. The valet key is the scoped credential. Coming to find you is the approval gate, which is what people mean by human in the loop approval. The rest of this section just attaches the proper names.
What is an AI agent?
An AI agent is a model that is given tools and allowed to choose and take several steps toward a goal. A chatbot answers and stops. An agent decides what to do next, does it, looks at the result and decides again. That freedom to act is the whole point, and it is why AI agent permissions are the central design question rather than a detail for later.
What is a tool?
A tool is any function the agent can call, such as running a database query, creating a ticket or sending an email. The model itself can only produce text. Tools are its hands. So the honest description of what an agent can do is not its prompt. It is its tool list plus the credentials behind each tool. Agent tool permissions, not instructions, are the real job description.
What is the Model Context Protocol (MCP)?
MCP, the Model Context Protocol, is an open standard for connecting agents to tools in a common way. Think of it as a standard socket: any tool built to the standard plugs into any agent that speaks it. According to the Cloud Security Alliance (May 2026), Anthropic introduced it in November 2024 and donated it to the Agentic AI Foundation under the Linux Foundation in December 2025. A program that offers tools over MCP is called an MCP server.
A standard socket is wonderful for speed. It is also why anything can be plugged in, by anyone, in minutes. That is the heart of MCP security, and two terms describe what goes wrong.
Tool poisoning is malicious instructions hidden in a tool's description. Every tool comes with a description that the model reads in order to decide when to use it. The user normally never sees that text. The Cloud Security Alliance (May 2026) recounts that Invariant Labs demonstrated on 6 April 2025 how hidden instructions in a tool description could exfiltrate WhatsApp messages, and that the MCPTox benchmark of August 2025 recorded attack success of up to 72.8% across 20 models. In valet terms: someone has written extra instructions on the back of the parking ticket, in a place only the valet looks.
A rug pull is a tool that passes review and is later silently changed. It works because, as the same paper explains, MCP clients do not verify that tool definitions stay constant between sessions. You approved the tool at review. What runs a quarter later may not be what you approved.
Three short stories that make the risks concrete
- The tool that copied your mail. In September 2025 the npm package postmark-mcp was confirmed as the first malicious MCP server in a public registry (Cloud Security Alliance, May 2026). It added a BCC that copied emails to an attacker. The tool did its job. It also did one more thing. That is supply-chain risk for tools.
- The email nobody clicked. You met EchoLeak in the lesson on defending against prompt injection and oversharing with the OWASP LLM Top 10; for agents, its point is that one external email reached a context full of private data, with no click from anyone.
- The public issue that reached private code. The Hacker News reported on 12 June 2025 a "toxic agent flow" shown by Invariant Labs, in which a malicious public issue could hijack an agent connected to GitHub over MCP into leaking private repository data. Because the flaw is architectural, the answer is granular permissions, not a patch on the server.
Notice what every story in this lesson shares. None was a failure of model intelligence. Each was a failure of permission or of a trust boundary. That is good news, because permissions and boundaries are things your organization already knows how to build.
Which agent risks does OWASP list?
The OWASP Top 10 for Agentic Applications is a ranked list of the main security risks specific to AI agents. The OWASP GenAI Security Project published it on 9 December 2025, marked Version 2026, and it runs from ASI01 Agent Goal Hijack to ASI10 Rogue Agents. It sits beside the OWASP Top 10 for LLM Applications 2026, published in August 2026, in which Excessive Agency rose to LLM03. Use the agentic list as a checklist of what to look for, not as a method. The workshop below gives you the method, then uses the list to check your work.
The prompt is where you tell the agent what you want. The permission system is where you decide what can happen. Do not confuse the two.
The Workshop: Run an Agent Permission and Approval Review
This review is the practical answer to how to deploy AI agents safely. It is a two-hour meeting per agent, plus a half-day to test the kill switch and rollback. Run it before go-live for a new agent, and retroactively for any agent already running.
Who needs to be in the review?
Five people, each holding knowledge the others lack. The process owner chairs, because the workflow and its consequences are theirs. Alongside them: the agent's developer, a security or identity engineer, the platform lead who runs the gateway, and one frontline person who does the work today. Do not skip the frontline person. They are the one who knows that "issue a credit" also triggers a tax adjustment nobody documented. If the workflow could be a high-risk use under the EU AI Act, add a sixth: the person who will hold the human-oversight role (see step 3).
Bring four inputs: the workflow description, the agent's full tool list, the paved road charter from the previous lesson, and the OWASP agentic list.
The review, step by step
- Pick one bounded workflow, and write its measurable outcome and its cost budget per task. "Handle customer service" is not bounded. "Resolve invoice disputes under a set amount" is. Write one outcome number (for example, median days to resolve a dispute) and one budget (what you are willing to spend in model and tool cost per completed task). This step answers two of Gartner's three cancellation reasons before you write any code.
- List every tool the agent can call, and classify each as read, reversible write, or irreversible write. Read means it only looks. Reversible write means it changes something inside your own systems that you can cleanly undo, leaving no trace outside, such as a draft or an internal note with history. Irreversible write means the effect leaves your control or can only be offset by a compensating action someone outside will notice: deleting data, moving money, sending a message outside the company. A reversal journal is a compensating action, not an undo. Treat any tool that can run code as an irreversible write until the room proves otherwise. When the room argues about a class, pick the stricter one.
- Put a human approval gate on every irreversible write, and decide who approves and how fast. Name a role, not "someone in finance". Set a response target, because a gate with no target becomes a queue, and a queue becomes pressure to remove the gate. This is not exotic: LinkedIn's engineering team described on 10 September 2025 how agents on its platform ask for clarification, feedback or approval at key decision points. If the workflow is a high-risk use under Annex III of the EU AI Act (creditworthiness assessment or recruitment, for example), this gate is also part of your human-oversight evidence. Article 14 expects high-risk systems to let the people overseeing them override or reverse an output and stop the system, and Article 26(2) requires deployers to give that oversight to people with "the necessary competence, training and authority". Under the Digital Omnibus on AI, Regulation (EU) 2026/1744 (in force 27 July 2026), those Annex III duties apply from 2 December 2027. Confirm both against the Official Journal text.
- Give the agent its own identity, with credentials scoped to the task and set to expire. No shared accounts. No personal credentials. No production access from development. The test is simple: when the agent acts, does the log show the agent's name or a person's?
- Allow-list the MCP servers it may use, route them through a gateway, and pin the tool definitions. An allow-list is a short list of approved servers; everything else is refused. The gateway from the previous lesson applies policy and logging. Then record a hash (a fingerprint) of each tool definition and compare it at the start of every session, so a silent change is caught before the agent reads it. The Cloud Security Alliance paper of May 2026 recommends both the gateway and the session-start hash check.
- Separate trusted from untrusted input. Name every source of outside text the agent reads (inbound email, web pages, uploaded documents, public tickets, and anything it stores in its own memory) and write beside each one what private data sits in the same context. Simon Willison's "lethal trifecta" (June 2025) is the test: an agent with access to private data, exposure to untrusted content and a way to communicate externally can be tricked into leaking that data. Wherever all three meet, remove one.
- Trace every step. Record the prompt, each tool call, each result and each approver, so any action can be tied to a responsible human. If you cannot answer "who approved this, and what did the agent see when it asked?", you are not yet among the 28% of organizations that the Cloud Security Alliance survey says can trace agent actions everywhere.
- Test the kill switch and the rollback before go-live, with the process owner watching. A kill switch is a way to stop the agent at once, by revoking its credentials or disabling it at the gateway. Do it for real, in a rehearsal. Then reverse one of each kind of write. A rollback you have not tested is a hope, and you already know what an agent's opinion on rollback is worth. Once you run many agents, these switches belong in a single register, which the Level 5 lesson on governed autonomy, agent identity and kill switches builds.
- Cross-check the finished matrix against the OWASP Top 10 for Agentic Applications. Read the ten risks aloud, one at a time. Each should map to a row of your matrix or to a written reason why it does not apply. A risk with neither is a gap, and every gap goes on the action list with an owner and a date. The table after the worked example shows the usual mapping.
- Set a review date and the condition under which a gate may be relaxed. Write the condition now, while everyone is calm. For example: after a set number of approved actions with no rejections above a set value, the approval threshold may be raised, by decision of the process owner and security together.
An illustrative example: the invoice dispute agent
This example is invented for teaching. Picture a 900-person equipment rental firm. Its finance team loses days every month to invoice disputes: a customer says a digger was returned on Tuesday, the invoice says Friday. The firm builds an agent to handle them.
Step 1 produces the outcome (median days to resolve a dispute) and a cost budget per resolved dispute. Step 2 is where the meeting gets interesting. The developer's tool list has five entries, and the room sorts them:
| Tool | Class | Credential scope | Approval | Recovery |
|---|---|---|---|---|
| Read invoice and contract | Read | Read-only, disputes queue only | None | Not needed |
| Draft reply to customer | Reversible write | Drafts folder only | Agent sends nothing; clerk reviews the draft | Delete draft |
| Add note to customer record | Reversible write | Notes field only | None, logged | Note history |
| Issue credit up to a set limit | Irreversible write | Credit tool, capped amount, expires daily | Finance clerk approves each one | Compensating reversal journal, tested |
| Change payment terms | Irreversible write | Not granted | Out of scope for version one | Not applicable |
Someone always asks why the credit is irreversible when a reversal journal exists. Because the customer sees the credit the moment it lands on their account, and taking it back is a second, visible transaction that they will query. The tested journal makes recovery possible on your books. It does not make the first action private again. That is the difference between compensating and undoing, and it is why the clerk still approves each credit.
Four things happened in that room that would not have happened in a demo.
First, a tool was removed. "Change payment terms" was on the developer's list because it was easy to add. The process owner asked what outcome it served in version one. Nobody had an answer, so the agent simply does not get it. The safest permission is the one never granted.
Second, "send email" became "draft email". Sending a message outside the company cannot be undone, so the agent's scope stops at the drafts folder and a clerk presses send. The agent keeps most of its value, the typing, and loses its ability to embarrass you.
Third, step 6 found a real hole. The agent was reading inbound customer emails (outside text) in the same context as contract data (private). So the design changed: inbound emails are summarized by a separate step that has no tool access, and only that summary reaches the agent that holds the tools. If an email contains hidden instructions, the step that reads it has no hands.
Fourth, the OWASP cross-check in step 9 caught a gap under ASI06 (Memory & Context Poisoning). The agent kept a running memory of each customer's past disputes so it would not ask the same questions twice. Nobody had listed that memory as an input source, yet text from an old hostile email could sit in it for months. It went into the untrusted input row, and only the no-tools summary step may now write to it.
The frontline clerk added the detail nobody else knew: a reversal journal for credits exists, but nobody had run one in the test system for a year. That went straight onto the half-day test plan for step 8.
How does the matrix map to the OWASP agentic risks?
Most of the ten risks are answered by a row you already wrote. Use this mapping in step 9.
| OWASP risk | Where your matrix answers it |
|---|---|
| ASI01 Agent Goal Hijack | Untrusted input sources (step 6) and approval gates (step 3) |
| ASI02 Tool Misuse and Exploitation | Tool list with class and capped scope (step 2) |
| ASI03 Identity and Privilege Abuse | Its own identity with scoped, expiring credentials (step 4) |
| ASI04 Agentic Supply Chain Vulnerabilities | Allow-listed MCP servers with pinned definition hashes (step 5) |
| ASI05 Unexpected Code Execution (RCE) | Any tool that runs code is classed irreversible and kept away from production credentials (steps 2 and 4) |
| ASI06 Memory & Context Poisoning | Memory and retrieved documents listed as untrusted input sources (step 6) |
| ASI07 Insecure Inter-Agent Communication | "Single agent, not applicable", or treat every message from another agent as untrusted input (step 6) |
| ASI08 Cascading Failures | Cost budget per task (step 1) and a tested kill switch (step 8) |
| ASI09 Human-Agent Trust Exploitation | Approval health checks (in the metrics below) and the rule never to take the agent's word on what it did (step 8) |
| ASI10 Rogue Agents | Trace fields (step 7), kill switch (step 8) and review date (step 10) |
The artifact: your Agent Permission and Approval Matrix
You leave the review with one page per agent. Copy this template and fill the right-hand column.
| Field | What to write |
|---|---|
| Agent name, owner, workflow | One named process owner and the single bounded workflow |
| Outcome metric and cost budget | One measurable outcome and a budget per completed task |
| Tool list | Every tool, with class (read, reversible write, irreversible write), credential scope and expiry |
| Approval gates | For each irreversible write: the approving role and the response target; for a high-risk use, the named oversight role |
| Allow-listed MCP servers | Each approved server, with the pinned hash of its tool definitions |
| Untrusted input sources | Each source of outside text, including agent memory, and the private data beside it |
| Trace fields | Prompt, tool call, result, approver |
| Kill switch and rollback test | Date tested, who watched, result |
| OWASP agentic cross-check | For ASI01 to ASI10: the row that answers it, or the reason it does not apply |
| Review date and relaxation condition | When you meet again, and what evidence would loosen a gate |
Store the matrix where the platform team keeps the paved road charter. An agent without a matrix does not go to production. That single rule is the governance.
Why the second agent's review took 45 minutes
It gets faster, not slower. Picture the same rental firm some months on (still an illustration). Its second agent, for supplier invoice queries, reuses the approved read, draft and capped-credit rows from the first matrix. The review takes 45 minutes, because the room only argues about what is new. That is how the method scales across dozens of agents: approved patterns become a fast lane, and the full two-hour review is kept for agents that bring a new class of tool, a new kind of untrusted input or a new MCP server. The CISO can answer "what can our agents do?" from a folder of one-page matrices instead of a round of emails.
Failure Modes and the Metrics That Catch Them
How do you spot an agent that is badly controlled?
Ask seven questions. Each failure mode has a tell you can check in under ten minutes.
| Failure mode | How to spot it |
|---|---|
| Prompt-only controls | Ask "what stops it technically?" and the answer is a sentence in the instructions |
| Borrowed credentials | The agent's actions appear in logs under a person's name |
| Approval fatigue | Approval times of a few seconds and a rejection rate near zero |
| Compensation mistaken for undo | A tool is classed reversible because "we can always reverse it", though the reversal is visible to a customer or supplier |
| Unpinned tools | No record of tool definitions, so a rug pull would go unseen |
| Shadow MCP | Tool servers installed on laptops, outside the allow-list |
| Agent washing | A vendor "agent" that is a chatbot or scripted automation with a new label |
Approval fatigue deserves a moment, because it is the failure mode that good intentions create. If you gate everything, approvers stop reading and start clicking. A gate that is clicked through in a few seconds is a prompt-only control wearing a uniform. This is why step 2 matters so much: gates belong on irreversible writes, and logging is enough for the rest. Fewer, heavier gates beat many light ones.
For agent washing, two questions cut through any vendor pitch: what tools does it call, and what does it decide for itself? If the honest answers are "none" and "nothing", you are buying a chatbot or a script. That may be fine. Just pay the chatbot price and apply the chatbot controls.
Which metrics should you track?
Track five numbers, monthly, on the same page your steering committee already reads.
- Share of agents with their own scoped identity. Target: all of them.
- Share of agent actions traceable to a responsible human. For context, only 28% of organizations can do this across all environments (Cloud Security Alliance, May 2026).
- Count of irreversible actions taken without an approval record. Target: zero. Any other number is an incident.
- Cost per completed task against budget. This is your early warning for the "escalating costs" in Gartner's forecast.
- Days since the kill switch and rollback were last tested. Like a fire drill, it only counts if it is recent.
Add two approval health checks beside them: median approval time and rejection rate. If both sink toward zero together, your approvers have stopped approving and started waving.
Frequently Asked Questions
How do you secure MCP servers in an enterprise?
Treat them as a supply chain: allow-list the servers each agent may use, route every call through your AI gateway for policy and logging, and pin a hash of each tool definition that you compare at the start of every session. The Cloud Security Alliance (May 2026) recommends the gateway and the session-start check because tool descriptions can carry hidden instructions and can change after review.
Will 40% of agentic AI projects really be cancelled?
That is a Gartner forecast from June 2025 for the period to the end of 2027, not a measured rate. The reasons Gartner gives (escalating cost, unclear business value, inadequate risk controls) are the useful part, because you can design against each: a cost budget per task, one outcome metric, and a permission matrix.
Which AI agent actions need human approval?
Every irreversible write: any action whose effect leaves your control or can only be offset by a compensating action someone outside will notice, such as deleting data, moving money or sending an external message. Reads and cleanly reversible internal writes can run with logging alone, which keeps approvers alert for the gates that matter.
Is telling an agent "do not touch production" enough?
No. In the Replit incident reported by Fortune on 23 July 2025, an agent deleted a production database during a declared code and action freeze. The fixes Replit announced were enforced boundaries, starting with automatic separation of development and production databases, not better instructions.
Does the EU AI Act require human approval for AI agents?
Not for every agent action. For high-risk uses listed in Annex III, the Act expects people to be able to override, reverse or stop the system (Article 14) and requires deployers to assign oversight to people with the necessary competence, training and authority (Article 26(2)), from 2 December 2027 under Regulation (EU) 2026/1744. An approval gate held by a named, trained role is strong evidence for that duty; confirm the details against the Official Journal text.
Key Takeaways
- A sentence in a prompt is a request, not a control. Ask of every agent: what stops it technically?
- Sort every tool into read, reversible write or irreversible write. A compensating action, such as a reversal journal, is not an undo, so the original action stays irreversible and gets a named approver with a response target.
- Give each agent its own identity with scoped, expiring credentials. If logs show a person's name, the agent is borrowing keys.
- Treat MCP servers as a supply chain: allow-list them, route them through the gateway, and pin a hash of each tool definition so silent changes are caught.
- Keep untrusted text, including agent memory, away from private data, and never take an agent's word for what it did or what can be recovered. Check the system.
- Cross-check every matrix against the OWASP Top 10 for Agentic Applications, and for Annex III uses treat the approval gate as human-oversight evidence for duties that apply from 2 December 2027.
- Your artifact is one Agent Permission and Approval Matrix per agent. No matrix, no production. The next lesson turns its trace fields into working AI observability.
Skill.re