TL;DR

In this blog post, Angel Alonso explores how shadow AI (the use of AI tools without the knowledge, approval or governance of the organisation's IT and security functions) affects security and how an AI gateway can help organisations gain control over employees’ AI use.

By providing a control door between employees and the models they use, an AI gateway can help organisations consistently manage access, data policies, model selection and logging. Alonso also looks at where an AI gateway’s protection ends, and why it needs to be supported by broader visibility, clear ownership and controls.

How The Matrix can make us understand shadow AI

In my earlier post, I used The Matrix to talk about the limits of AI in detection and response. Following the same analogy, the film has something to say about a different problem too.

Agent Smith starts the trilogy programmed to be an enforcer with a fixed job: hunt down anything that threatens the system. After his fight with Neo at the end of the first film, something in him changes. He gains the ability to copy himself onto anyone plugged into the Matrix, and he uses it without restraint. Leading to hundreds of copies of Smith, each one acting on its own. By the third film, there are entire crowds of Smiths. What makes Smith a useful analogy here is the proliferation: copies appearing everywhere, beyond anyone’s control.

This is similar to what we see today in our customers. We see AI use spreading beyond centrally planned projects, as employees and teams adopt tools for individual tasks. Someone tried an assistant to summarise a report. A developer wired up an agent to call a model API. A vendor added an AI feature to software you already use. Often, people were simply trying to get their work done faster.

So the question that arises is how can we enable our employees to use AI securely, without losing control?

What is shadow AI?

Right now, every app, script and assistant in your company probably talks straight to a model provider. Each one of these holds its own key and makes its own choices about which model to use and what to send it.

Back in January 2020, I sat down with Robby Peralta for an episode of his security podcast named “Cloud Security with an Angel”. We talked a lot about shadow IT; people reaching for the tool that got the job done, procurement finding out weeks later, security finding out even later. Six years on, I could record that same episode and only change one word. Shadow AI follows the same pattern: employees adopt the tools before the organisation has agreed how to manage them.

Shadow AI is the use of AI tools without the knowledge, approval or governance of the organisation's IT and security functions. It can include personal chatbot accounts, unapproved coding assistants and direct model API connections.

One lesson from shadow IT is that restrictions are harder to sustain when the approved tools do not meet people’s needs.

The employee's intention may be entirely reasonable: finish the task faster. But the organisation may not know which information was submitted, under which account or with which retention settings. It cannot assume that a personal subscription has the same controls or contractual terms as its approved enterprise service.

Consider the difference between summarising a public advisory and uploading a customer’s incident timeline. The interface can look identical, while the information being disclosed is very different. With an agent, access can extend further; a connected tool may retrieve additional records or perform actions using the permissions it has been given.

There is usually an AI policy somewhere already. There may even be a list of approved tools. A policy can say that customer information must only reach approved services. Someone still needs to implement that rule in every application. Without a shared approach, teams must repeatedly solve access management, destination restrictions and logging, and maintain those controls as their integrations change.

An AI gateway can help teams apply those controls consistently across the applications connected to it. It gives employees a fast, approved way to use AI, instead of a rule they have to remember and a workaround they will find if the approved way is slower.

What is an AI gateway?

In Gartner's May 2026 report Market Overview for AI Gateways, they define an AI gateway as “an intermediary platform that is situated between the AI applications and agents and the AI models, tools and resources they use. Its core function is to provide centralised management of access to AI models and remote Model Context Protocol (MCP) servers.”

The report identifies several routes into this market. Established API management vendors are extending their platforms to support AI workloads. API security vendors bring a focus on discovery and risk oversight. Specialist AI companies and open-source projects approach the problem through the needs of developers building AI applications.

The scope of these products is also expanding. Gartner describes vendors adding MCP and Agent2Agent (A2A) support, moving beyond model access towards governing communication between agents and their environment.

A gateway puts a control door between employees and the models they use. Traffic goes through it, the gateway checks who is asking and what is being sent, picks the right model and writes it all down.

The AI gateway gives you a place to apply shared policies:

• Identity and access. Every request carries an identity, whether it comes from a person, an internal app or an agent. The gateway checks who is asking before it checks anything else, so nobody reaches a model on a shared, anonymous key.
• Data checks and policies. What goes out gets checked on the way. Depending on its capabilities and configuration, the gateway can flag or redact detected personal data and secrets, and block requests to unapproved destinations.
• Routing and usage limits. The gateway can have routing rules to use lower-cost models for suitable tasks and set usage limits by team or application.
• Logs and cost visibility. For traffic routed through the gateway, teams can track usage and costs by team, application or model in one place.

The major security vendors are also moving into this space. Check Point, Palo Alto Networks, Zscaler, Netskope and F5 are introducing AI gateways and related inspection and governance capabilities across their portfolios. Since their products already sit inline, with traffic passing through them, organisations may be able to introduce AI security controls through infrastructure they already operate, including existing firewalls and cloud security platforms. This could lower the barrier to adoption, although the capabilities and deployment requirements vary between products.

LiteLLM as a working example

LiteLLM is one of the better-known open-source AI gateways, and a useful illustration of what this could look like.

One API, many providers. LiteLLM is an open-source library that gives you a single, unified interface to call 100+ LLMs (OpenAI, Anthropic, Vertex AI, Bedrock, and more) using the OpenAI format. A team can switch models, add a fallback provider or move part of its traffic to a cheaper model for simple questions, without rewriting the application that calls it.

Virtual keys, not shared secrets. Instead of handing every project a copy of the real provider key, LiteLLM issues virtual keys scoped to specific models, budgets and rate limits. The actual provider credentials stay in one place, managed centrally and rotated on a schedule.

Spend tracking by default. Every request is logged with the team, project and model behind it, so a sudden spike in cost or usage is visible long before it turns into a surprise on an invoice. Budgets can be set per key, per team or per model, and enforced rather than just monitored after the fact.

A place to add checks. Because every request passes through the same point, this is also where an organisation can attach its own logic. Redacting personal data before it leaves the network, blocking a request bound for an unapproved destination or routing anything tagged as customer data to a separately approved environment. A gateway does not know that a message contains a customer’s incident timeline unless someone tells it so. The application still has to pass that context along, and someone still has to decide what the policy should actually say for that kind of data.

Where the AI gateway's protection ends

An AI gateway adds a layer of control to existing security measures, but its reach is limited to the connections that pass through it. An employee using a personal account in a browser, or an AI feature operating within a software vendor’s infrastructure, may bypass it entirely. Gateway logs therefore show part of the picture, not a complete inventory of AI use across the organisation.

Endpoint detection and response (EDR) on servers and employee devices remains a foundation for visibility. It can help identify agents running locally, scripts calling models directly and unapproved browser extensions. Network traffic analysis and cloud application discovery can provide additional visibility into AI use outside the gateway.

There is also the question of intent or authority. Knowing that someone accessed an AI service does not necessarily reveal what they sent, and inspecting a request does not establish whether it serves a legitimate business purpose.

Non-human identities add another challenge: service accounts and machine identities can accumulate broad, standing permissions that are rarely reviewed. A gateway does not automatically know who owns an identity, why it has access, or whether that access is still appropriate. Those questions require identity and access management that extends beyond the gateway.

Prompt injection exposes another limit. A retrieved webpage, threat report or tool result can contain malicious instructions designed to redirect an agent. Content inspection can detect some attempts, but it cannot guarantee that the model will follow only the intended instructions. Limiting what an agent can access and do remains essential, even when its requests pass through an approved gateway.

Winning the battle against the Agents

Back in The Matrix, Agent Smith kept multiplying until the system could no longer contain him. Neo made a deal with the Machines: he entered the Matrix unarmed and allowed Smith to assimilate him. By copying himself onto Neo, Smith exposed himself through Neo’s connection to the Machines. That connection gave them a way to destroy Smith and his copies without hunting each one down individually.

The question we started with was how to enable employees to use AI safely without losing control. Smith’s copies were deleted through a shared connection. An AI gateway gives an organisation a similar point of control: an approved route where it can enforce shared rules across the applications and agents that use it.

The harder question is what remains outside that route: personal accounts, unapproved tools and AI features built into other services. The AI gateway needs to be supported by visibility into AI use elsewhere, clear ownership and limits to what agents can access and do.

For me, this brings us back to old lessons from shadow IT. People will reach for tools that help them work. Our job is to make the approved route useful enough that they choose it, while maintaining visibility into where they still go around it. AI gateways can provide that route, particularly when they build on controls and security platforms organisations already use. Giving employees a practical way to explore AI without giving up control.