Secure RAG: Six Design Patterns for Enterprise Data

Secure RAG protects the retrieval path so a user receives only the source material they are authorized to use.

If an employee cannot open a document in the source system, the assistant should not reveal it through a search result, summary or citation. That means the access decision has to follow the document into its chunks, indexes and caches, and be checked when the employee asks a question.

The employee’s access can change while copies of the document remain in the system. Those copies pass through ingestion, indexes, retrieval and model requests, and may appear again in citations or logs. Deletion needs to reach them too. If any of those handoffs widens access without the owner’s approval, the assistant may answer from information the employee was never allowed to read.

GS Consulting’s private LLM and secure RAG implementation service helps teams work through these choices. For CUI boundaries and contractor requirements, see secure RAG architecture for GovCon. The private LLM security controls guide follows the complete data path, and the private LLM cost model addresses what it takes to operate it. You can find the related guides in the Private LLM and Secure RAG hub, or explore the broader secure AI automation service.

Which secure RAG design fits your data?

The first question is who can read the data. Then consider how often those permissions change and what would happen if a filter failed. That gives you a reason to choose among these six designs. You can combine them, provided someone owns the additional routing, policy and support work.

Whichever design you choose, test it against your actual permissions and failure cases. A separate store, for example, will not protect a customer if the application routes that customer’s query to somebody else’s store.

What does RAG stand for?

RAG stands for retrieval augmented generation. For example, an employee might ask about a policy that changed yesterday. The application searches the approved source material and includes relevant passages in the model request. The model can then use that current information to prepare the answer.

The employee still needs permission to read that policy. Secure RAG checks that permission during retrieval and applies the handling rules for the resulting data. Asking the model to ignore restricted text after it has received the text is too late for that access check. The application must decide what the requester may use before building the model context.

Unauthorized access, leakage and poisoning are among the risks in the OWASP guidance on vector and embedding weaknesses. The NSA AI Data Security guidance also follows provenance, integrity, access and monitoring through the data lifecycle. Your review therefore needs to follow the document and its copies. Looking only at the final answer leaves much of that handling out of view.

For that review, establish who is responsible for these six controls:

1. Source authority. Record which system owns the content. Its access rules determine what the application may retrieve.
2. Identity context. The access check needs the user and service involved in the request. Include the role, tenant, device, purpose and session conditions that affect the decision.
3. Ingestion control. Keep classification, ownership, permissions, version, retention and deletion state with each chunk.
4. Retrieval enforcement. Apply the current access decision before content enters the candidate set or model context.
5. Answer traceability. Give the user usable citations. Retain the source, policy, model and prompt versions needed to investigate the response.
6. Failure handling. An untrusted identity, incomplete metadata or uncertain policy or source state needs a defined response. Specify when the application denies the request, narrows it or sends it for review.

What are the 5 steps of RAG?

Ingestion, indexing, retrieval, context assembly and generation give you five places to examine what happens to a document. They cover how it enters the application, how it becomes searchable, what a query returns, what reaches the model and what comes back to the user. An implementation may combine or separate those stages; the count is an explanatory grouping, not a requirement. Each stage still has to preserve the access rules.

1. Ingest approved sources. Keep the source owner, received version and allowed readers with the incoming material. Check whether the content has been tampered with or contains instructions intended to redirect the assistant.
2. Build the index. Create chunks and, where needed, embeddings. The derived records need the same permission information and source identifiers as the content they came from.
3. Retrieve for the current user. Check current permissions before returning passages. If the application cannot establish a trusted identity or access state, the response is denial or review, rather than an unchecked retrieval.
4. Assemble the model context. Pass only authorized passages to the model. Keep the retrieved text separate from application instructions so a document cannot give itself authority.
5. Generate and review the response. Check citations and apply output controls. Protect the answer and logs, keeping enough evidence to investigate what happened.

The OWASP RAG Security Cheat Sheet provides checks across that process, including permissions, content integrity, output handling and monitoring.

How do you choose a secure RAG design pattern?

The source owner and security owner may have different reasons for requiring separation. Work through those reasons before comparing platforms. These five questions establish what a shared index would need to do and when another retrieval path would be necessary.

• Where does authority live? A source that decides access at query time may change permissions between index updates. Copying those permissions gives your team another list to maintain.
• How much isolation is required? Check the reason for the separation. Two business units may share a policy, while customers, legal matters, export controlled data or regulated records may bring different obligations.
• How quickly do content and permissions change? A daily update may be enough for a policy library. It may leave a case record available long after a person’s access has been removed.
• How many systems and tenants participate? Each additional source or tenant affects the work needed to maintain separate stores, delegate queries and evaluate policy.
• What must an investigator prove? Ask the reviewer which source records, access decisions and events would let them trace an answer. Those records need to be available when a request is investigated.

The per session access decisions and dynamic policy in NIST SP 800-207 provide a useful principle for this decision. Retrieval can use the requester’s current identity and the resource’s current state, even outside a formal Zero Trust implementation. Network location and the permission list saved at indexing time are not substitutes for that current decision.

Six secure RAG design patterns

1. Shared index with metadata filters

With one index serving several teams, the application has to distinguish their content within that index. It does so using fields on each chunk: tenant, business unit, role and sensitivity, together with the owner, source identifier and permission version. The application uses the current user context to build a mandatory query filter against those fields.

Broad internal knowledge with moderate sensitivity and stable metadata can suit this approach, provided the common retrieval service has strong test coverage. A missing field is an important test. If a missing field or defective filter could expose another customer’s data or cross a legal or regulated boundary, choose a different pattern for that scope.

2. Indexes separated by domain

Legal, HR, finance and engineering can maintain separate indexes or collections, with policy routing each request to an approved domain before search begins. This is worth considering when a few durable business boundaries justify the separation.

A filter failure within one domain can then be contained to a smaller set of data. The tradeoff is work on each index: synchronization, maintenance and eventual retirement. Assign that work to a domain owner rather than assuming the shared application team will absorb it.

3. Store per tenant

A customer separation requirement in the product or contract can justify giving each tenant its own vector store or dedicated logical partition. Each store or partition has independently enforced credentials and lifecycle rules. You can share application services while keeping data access specific to that customer.

The store is only one place a tenant mix up can happen. Routing, cache keys, logs, evaluation data and support tools all need the same separation. Test those paths before treating the dedicated store as evidence that customers cannot see each other’s information.

4. Delegated source retrieval

With delegated retrieval, the RAG service asks the source to search or retrieve on behalf of the current user. The source continues to manage its permissions, current content, deletion state and record authority. This can help when permissions change quickly enough that an indexed copy would struggle to stay current.

The source must also be capable of handling the search. Its availability and response time become part of the user’s experience. Microsoft’s secure multi tenant RAG guidance discusses the responsibilities for identity, data isolation and the application. Those responsibilities still need owners when retrieval is delegated.

5. Brokered hybrid retrieval

An enterprise may have a policy library suitable for a shared index, customer records requiring tenant stores and another source whose permissions must be checked live. A policy broker can choose between those routes using identity, purpose, data class, source capability and query risk. It combines only the passages the requester is authorized to use.

That flexibility fits mixed sensitivity across many connected systems. It produces the highest fit score in the GS scenario, along with the highest control burden. Someone has to maintain consistent policy and routing across those systems, retain the evidence and handle failures when a route is unavailable.

6. Isolated local RAG lane

Some work requires a dedicated environment for the content, embeddings, retrieval and model execution. Logs and administration stay in that environment too, and movement into or out of it is explicitly reviewed.

Consider this approach when network separation, sensitivity, mission continuity or local control matters more than model variety and operating simplicity. The team still has to address insiders, poisoned content, excessive privileges and missing evidence. Isolation determines where that work happens; it does not remove it.

GS Secure RAG Pattern Fit Index

GS Consulting used six factors to compare these designs in a defined enterprise scenario. Permission fidelity carries 25 percent of the weight, isolation 20, freshness 18, audit evidence 17, scale 12 and simplicity 8. An analyst assigns each pattern a rating from 1 to 5 for each factor. The result is a planning comparison built from those assumptions.

For mixed sensitivity across many systems, brokered hybrid retrieval scores 92.8. Delegated source retrieval scores 86.4 when source permissions and freshness dominate. A store per tenant scores 83.2 for customer isolation, while isolated local RAG scores 80.8 for disconnected or highly sensitive environments. Separate domain indexes score 78.4. A shared index with metadata filters scores 74.4.

The scores describe the scenario the analyst rated, using the cited guidance and stated assumptions. We did not measure product performance to produce them. A buyer with different requirements needs to revisit the ratings, weights and thresholds before using the comparison. The ordering should not be read as a ranking for every organization.

The alternate weights in the sensitivity check move the leading score by 1.0 point. No pattern moves more than 3.0 points, and the broad ordering stays the same. That is a limited check on the model’s response to those weights. It supplies no measurement of how a chosen design will perform in your environment.

The reference set includes the NIST Generative AI Profile, NIST SP 800-207, OWASP vector guidance and NSA data security guidance. It also includes the AWS generative AI security reference architecture and Google Cloud RAG reference architectures.

What will the design take to operate?

The separate burden model looks at integration, policy complexity, test load, operations and cost. Brokered hybrid retrieval scores 100.0. Isolated local RAG scores 95.6, delegated source retrieval 93.2 and a store per tenant 86.8. Domain separation scores 72.0, compared with 61.2 for shared metadata filtering.

The shared design is the least burdensome in this comparison because there is less to operate. Your access requirement still comes first. If the design depends on every metadata field and every filter being correct to keep two customers apart, its lower score is not evidence of safety. Compare the work needed to enforce the boundary and keep it current, then choose the least burdensome option that meets that requirement.

How do you turn the access rules into a retrieval design?

1. Find the source owner. Establish which system owns the content and its permissions. Retention, deletion and version state need an authoritative system too.
2. Define what must stay separate. Distinguish ordinary business access from tenant, legal, regulated and mission boundaries.
3. Check the update delay. Measure the time between a source change and the corresponding index update, including changes to permissions.
4. Select a pattern for that boundary. Use those findings to select shared, separated, delegated, hybrid or isolated retrieval.
5. Test requests that should not succeed. Include missing identity, stale permissions, malformed metadata, cross tenant queries, deleted content and an unavailable source.

A role can mean one thing in the identity service and something different in the source. When the index, broker and model gateway act on that role, the difference can change who gets access. Assign an owner to the translation, test it across the connected systems and retain the policy interpretation in the evidence. Matching role names alone does not establish that the rules match.

Where can a secure RAG design fail?

An allowed query may return exactly the right answer while a revoked user can still retrieve the same material. Include incomplete and out of date access information in the tests, rather than judging the design only by successful answers.

• Permission drift: the source removes or changes access, but the index is still working from the earlier permission list.
• Missing metadata: an absent access field lets a passage through when the application cannot establish who may read it.
• Cache mixing: a passage or answer saved for one request is served to a different user or tenant.
• Deletion lag: a document is removed, but its chunks, index entries or cached copies are still available.
• Citation mismatch: following the citation does not substantiate what the assistant told the user.
• Unlogged context: an investigator opens the records and cannot determine which material was sent to the model.

Test prompt injection alongside those failures. A document might tell the assistant to ignore its instructions or reveal other context. Being retrieved gives that text no authority to approve an action or grant access. The retrieval service, tools and model gateway must continue treating it as content, however convincingly it is written as an instruction.

A 90 day plan for one secure RAG workflow

Days 1 through 30: understand the data and permissions

Choose one business question for the initial work. That gives the source and security owners a specific answer to examine: which documents could supply it, and which users should receive it?

• List all the sources that could contribute to that answer.
• For those sources, identify the content owners, identity providers and permission rules. Include their data classes, retention requirements and deletion paths.
• Check how often the documents and permissions change. Compare those changes with how long the index takes to catch up.
• Write down requests the application should refuse before putting together the queries it should answer.

Days 31 through 60: build the chosen path and its records

By this point, the source and permission review should give you a reason to choose a pattern. Use the simplest one that preserves the boundary. As you build it, keep the information needed both to decide access and to explain that decision during a later review.

• A chunk needs to retain its source identifiers, access fields, version and classification.
• Apply mandatory policy using the requester’s current identity and purpose.
• Record the citations, policy decision, model version and prompt version, including the path used to handle an exception.

Days 61 through 90: test failures and assign ongoing ownership

Give a reviewer an allowed answer and a denied request. Can they follow each through the records and explain what the application did? Resolve anything they cannot account for before adding users or connecting more sources.

• Test valid, denied and revoked access, as well as deleted content, missing metadata, source outages and cross tenant requests.
• Measure unsupported answers, citation validity, denial accuracy, stale content delay and exception volume.
• Agree who owns policy, sources, indexes, models and evidence, and how often those owners will review the workflow.

Which records should your team keep?

Keep eight records together: the source register, access model, ingestion rules, retrieval policy, model context record, test results, exception decisions and owner review. They should let another person follow a request from the source access decision through the response and any subsequent review.

The Microsoft RAG enrichment guidance explains chunk metadata and update design. OpenFGA’s RAG authorization guidance addresses retrieval and relationship based authority. Use those implementation references alongside your organization’s decisions about ownership, retention, evidence and acceptable failure.

Changing a source or its permission model calls for another look at these records, as does changing the indexing process. Include the embedding model, retrieval logic, prompt, generation model and output controls in that change review. A user may see the same interface throughout, even though the changes affect what reaches the model.

What should a retrieval review demonstrate?

For any passage used in an answer, the reviewer needs its source, version and the reason access was allowed. They also need to see what reached the model. For a denied request or an exception, the records should show what happened and who handled it. Encryption, private endpoints and citations help protect and explain the workflow; the operating records are still needed to examine whether the access boundary held.

Sources and method

• NIST Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile
• NIST SP 800-207, Zero Trust Architecture
• OWASP Vector and Embedding Weaknesses
• NSA AI Data Security Guidance
• Microsoft Secure Multi Tenant RAG
• AWS Generative AI Security Reference Architecture
• Google Cloud RAG Reference Architectures

GS Consulting derived these planning scores from public sources and the stated analyst assumptions. They do not report measured security outcomes and should not be used as legal advice, compliance determinations or product endorsements. Check the model’s assumptions against your own data, contracts, access rules, architecture and risk process before relying on the comparison.

Frequently asked questions

Related reading

• Private LLM and Secure RAG Hub
• Secure RAG Architecture for GovCon
• Preventing CUI Leakage in LLMs
• Data Classification Before AI Automation
• Secure AI Automation