Securing RAG Pipelines: Per-User Document Authorization for AI

Evan Sims

Your RAG pipeline has a security hole, and post-retrieval filtering isn’t fixing it. If you’re checking document permissions after retrieval, unauthorized data has already entered the context window. That’s not a minor oversight — it’s a data breach waiting to happen.

Retrieval-augmented generation is now the standard pattern for connecting LLMs to enterprise data. Retrieve relevant documents at query time, inject them into the context window, generate a grounded response. Simple, powerful, and deeply insecure by default.

Without authorization controls on the retrieval step, the model can access any document in the corpus. Every document. Regardless of who’s asking.

Why Post-Retrieval Filtering Is Broken

The most common approach to RAG authorization looks reasonable on a whiteboard: fetch the top-k most relevant documents, then filter out anything the user shouldn’t see.

It’s fundamentally insecure.

The unauthorized documents have already been retrieved. In systems using embedding-based retrieval, the document content is encoded in the embeddings. The retrieval step may log or cache the full document text. If retrieval and filtering happen in the same process, a single bug — a missing check, an exception that bypasses the filter — and confidential information reaches the model.

Then it gets worse. The model generates a response incorporating that information. The response is delivered to the user. You’ve just built a system that launders unauthorized data through an LLM.

Post-retrieval filtering treats authorization as a convenience layer. In any system handling sensitive data — HR records, financial documents, legal files, medical information — it needs to be a security boundary.

Pre-Retrieval Authorization: The Only Secure Approach

InferaDB enables authorization before the retrieval step ever executes. Before the vector store is queried. Before document content enters any cache or log. Before the model can see it.

The approach is straightforward:

1. User submits a query to the RAG pipeline
2. The retrieval system identifies candidate documents
3. InferaDB checks authorization for each candidate against the requesting user
4. Only authorized documents proceed to retrieval and ranking
5. The model generates a response using only permitted context

Unauthorized documents are excluded at the earliest possible stage. They never enter the context window. They never influence the response. They never appear in retrieval logs.

The Authorization Model

Document access maps naturally to InferaDB’s relationship-based schema. The model is simple:

user:alice → viewer → document:quarterly-report

In practice, you don’t model permissions per-document. You model your organizational structure — departments, projects, teams, shared folders — as typed relationships in the Infera Policy Language. A user’s access to a document is derived from their team memberships, project roles, and sharing relationships.

When a new document is ingested into the RAG pipeline, a single relationship tuple places it in the hierarchy. The document inherits access policies from its parent container. No per-document ACL to maintain.

When an employee changes teams, updating their membership in InferaDB automatically updates access to all associated documents. The authorization model stays in sync with organizational structure without batch synchronization jobs or stale cache invalidation.

Sub-Microsecond Latency Makes This Feasible

The obvious objection: “Checking authorization for every document candidate will destroy my retrieval latency.”

Not at 2.8µs per check.

Authorizing a thousand candidate documents adds less than 3 milliseconds to the retrieval pipeline. Compare that to embedding search (50-200ms) and model inference (500ms-5s). Authorization at this speed is noise — it disappears into the latency budget.

This is what makes per-document authorization practical at scale. Previous approaches required either:

• Batch pre-computation of access lists (stale by definition)
• Coarse-grained filtering by department or role (too permissive)
• Post-retrieval checks (insecure, as discussed)

Sub-microsecond latency unlocks a fourth option: real-time, per-document, per-user authorization at query time. No staleness. No over-permissioning. No data leakage.

Chain-of-Thought Permission Traces

When an AI agent generates a response in a regulated environment, “it had access” isn’t a sufficient audit answer. You need to know exactly what was accessed, why it was permitted, and what was denied.

InferaDB records every authorization check in a tamper-evident audit trail, producing a chain-of-thought permission trace for each agent interaction:

• Which documents the retrieval system considered
• Whether access was granted or denied for each
• Which relationship path was traversed to reach the decision
• Which policy rule matched
• At what revision of the authorization state the decision was made

For regulated industries — finance, healthcare, legal — where AI-generated outputs may inform consequential decisions, this trace is the evidence that the system operated within authorized boundaries. It transforms authorization from a silent gatekeeper into a documented participant in the AI pipeline’s decision chain.

Auditors don’t need to trust that your RAG pipeline respects permissions. They can verify it, check by check, with cryptographic proof.

Getting Started

Integrating InferaDB into an existing RAG pipeline is straightforward. The authorization check is a single API call that sits between your candidate generation and retrieval steps:

1. Model your document hierarchy in InferaDB using relationship tuples
2. Write relationship tuples when documents are ingested
3. Check authorization for candidate documents before retrieval
4. Query the audit trail for compliance reporting

The authorization model grows with your organization. New teams, new projects, new document types — they all map to the same relationship-based schema without schema migrations or access list rebuilds.

Your RAG pipeline is only as secure as its authorization layer. Post-retrieval filtering is a patch. Pre-retrieval authorization is a solution.

Start with the quickstart guide or explore InferaDB on GitHub to secure your RAG pipeline today.