Give a large language model broad access to data and it becomes the perfect insider threat, operating at machine speed and without human judgment.
Large language models (LLMs) have quickly evolved from novelty chatbots to sophisticated agentic AI systems that act on our behalf. Unlike traditional software, these AI agents don’t follow rigid, predefined instructions. They interpret our intent and make autonomous decisions. That elasticity lets enterprises automate everything from support tickets to code reviews, but it also multiplies risk. If an AI agent misinterprets intent or gets manipulated by malicious input, it could access or reveal data it shouldn’t.
Mitigating these risks requires rethinking how we enforce security and authorization in AI-driven applications. Today’s authorization is based on humans clicking well-defined routes, not bots improvising at runtime. We need something better, something that travels with every vector search, API call, and database query an agent makes. One emergent answer is Oso, an authorization engine already powering Duolingo, Intercom, and PagerDuty. Oso isn’t the only answer to AI’s security issues, but it’s increasingly an essential part.
Off-piste and off-script: New risks of agentic systems
Researchers recently hid instructions inside a harmless-looking email. Microsoft 365 Copilot, which auto-summarizes messages, obeyed and leaked internal data—no clicks required. The takeaway is brutal: Give an LLM broad access to data and it becomes the perfect insider threat, operating at machine speed and without human judgment.
Traditional security controls weren’t designed for scenarios like this.
We’ve historically handled security by authenticating the user and then checking what actions or data the user is allowed to access. We might use OAuth scopes or role-based access control (RBAC) to guard each API endpoint. But LLM-powered systems break this model. LLM agents still call ordinary APIs, but they decide the sequence, parameters, and data scope at run time. Because you don’t know that exact combination beforehand, traditional endpoint or scope-based controls alone can’t guarantee that every generated request is authorized. You need authorization that travels with each query and data fetch and is evaluated at the moment the agent generates the request.
Even new AI-specific frameworks haven’t solved this fully. Consider Anthropic’s Model Context Protocol (MCP), which standardizes how AI agents invoke tools. It cleanly separates the “agent” from the tools it can call. But MCP only lets you permit or deny tool usage at a broad level. If you expose a delete_record function, MCP can gate whether an agent may call delete_record at all, not whether it can delete this particular record. As Oso’s Greg Sarjeant points out, this leaves us no better off than with basic OAuth. To enforce true resource-level authorization, the logic has to live closer to the data and action in question.
The guardrails approach
Faced with these challenges, developers have turned to a variety of stopgap measures. One approach is to add guardrails on the AI’s inputs and outputs—essentially, trying to sanitize what goes in and monitor what comes out. These guardrails come in roughly three flavors:
- Prompt injection filters (Rebuff, PromptGuard). These try to detect suspicious strings and embeddings before they hit the model. They are effective against known jailbreaks, but prompt-based exploits are endlessly adaptable. A clever attacker might slip in a new form of jailbreak that the filters don’t recognize.
- Output content filters (Guardrails AI, NeMo Guardrails). These inspect what the LLM is about to output and filter or refuse disallowed content. Output guardrails are great for stopping toxic language or obvious policy violations, but they’re not a foolproof shield against data leakage. If the model was never supposed to have certain data, we’d rather it never be in the output in the first place.
- Controlled tool use and sandboxing (LangChain). This strategy limits what an AI agent is allowed to do in the first place, such as giving an agent access to only a carefully curated set of tools or APIs and then running those in sandboxed environments. While sensible, LangChain punts on data-level filtering; you must hand-roll per-user vector searches and hope you didn’t miss one.
Each of these approaches is useful but insufficient. None ensures that the data an AI is using or the actions it attempts are authorized for that specific user and context. In other words, the AI might still be operating on an honor system, which is a really bad security posture to bet enterprise data on.
This is where fine-grained authorization comes in. Instead of just slapping guards around the AI, we need to weave permission checks into the AI’s decision loop. The ideal solution would automatically enforce “who can see what” every time the AI tries to access some data or perform an operation. That’s exactly the approach Oso takes.
Bringing fine-grained permissions to LLMs
The core idea behind Oso’s methods is simple: An AI agent should be treated like any other user, subject to all the same permissions. So, for example, Oso provides a drop-in solution for secure retrieval-augmented generation (RAG) workflows, which are a common way LLMs are used in enterprise apps. In a RAG setup, when a user asks a question, the system will retrieve chunks of internal data (documents, records, etc.) to feed into the LLM as context so it can give an informed answer. Those chunks might live in a vector database as embeddings. Oso’s new integration with Python’s SQLAlchemy ORM essentially applies your permission rules to that retrieval query automatically.
In practice, that means you define your authorization policy (who can see what) once in Oso’s policy language, Polar, and Oso will ensure every vector similarity search or database lookup is filtered to only return chunks the current user is allowed to see. In other words, you write one Polar rule (“managers can view tickets in their department; creators can view their own”) and Oso rewrites the SQL or vector search so the LLM only sees authorized chunks. No exposure means no leak. This is the same least-privilege doctrine you (hopefully) use for humans, but it’s extended to agents.
Under the hood, Oso compiles policy into query fragments at the ORM layer, so enforcement remains consistent across microservices. Update the rule once and every AI workflow inherits the change. It’s a simple yet powerful approach to AI security.
Balancing innovation and security
There is so much incredible promise in AI right now but also incredible peril. Users and enterprises need to trust that the AI dream won’t become a security nightmare. As I’ve noted, we often sideline security in the rush to innovate. We can’t do that with AI. The cost of getting it wrong is colossally high.
The good news is that practical solutions are emerging. Oso’s permissions model for AI is one such solution, turning the theory of “least privilege” into actionable reality for LLM apps. By baking authorization into the DNA of AI systems, we can prevent many of the worst-case scenarios, like an AI that cheerfully serves up private customer data to a stranger.
Of course, Oso isn’t the only player. Pieces of the puzzle come from the broader ecosystem, from LangChain to guardrail libraries to LLM security testing tools. Developers should take a holistic view: Use prompt hygiene, limit the AI’s capabilities, monitor its outputs, and enforce tight authorization on data and actions. The agentic nature of LLMs means they’ll always have some unpredictability, but with layered defenses we can reduce that risk to an acceptable level.
