Summary
Retrieval-Augmented Generation (RAG) grounds a model's responses in your own data — but "RAG" is not one thing to secure, it's seven stages: Ingestion, Storage, Indexing, Retrieval, Prompt Assembly, Model Inference, and Output. Each stage has a distinct risk profile, and a control that protects one stage (say, access control on the vector index) does nothing for another (say, a poisoned document making it into the index in the first place). This chapter walks the pipeline stage by stage.
Core concepts
1. Ingestion
→
2. Storage
→
3. Indexing
→
4. Retrieval
5. Prompt Assembly
→
6. Model Inference
→
7. Output
The reason to think in stages rather than "the RAG system" as one unit: a document can be perfectly clean at Ingestion and still leak data at Retrieval if access control isn't enforced there too. Securing one stage never implies the others are secure.
Architecture discussion — stage by stage
1. Ingestion
| Risk | What can go wrong | Azure-native controls |
|---|---|---|
| Poisoned documents | Malicious instructions embedded in PDFs/docs | Defender for Storage (malware scanning); Azure AI Content Safety (pre-index scan); manual review workflow for high-risk sources |
| Malware | Embedded exploits in files | Defender for Storage malware detection |
| Credential leakage | Secrets embedded in docs | Content Safety + regex scanning; Key Vault secret scanning policies |
| Unauthorized ingestion | Anyone can upload content | Entra ID auth; managed identity for ingestion pipelines; RBAC on storage |
2. Storage
| Risk | What can go wrong | Azure-native controls |
|---|---|---|
| Data exfiltration | Public blobs or shared keys | Disable public access; Private Endpoints; RBAC-only access |
| Unauthorized access | Broad contributor roles | Least-privilege RBAC; separate containers for raw vs. processed data |
| Compliance violations | Data retention / deletion gaps | Immutable blob policies; lifecycle management rules |
| Key compromise | Storage keys leaked | Azure Policy: disable key-based auth |
3. Indexing (Azure AI Search)
| Risk | What can go wrong | Azure-native controls |
|---|---|---|
| Over-indexing | Sensitive fields indexed unintentionally | Field-level controls (searchable, retrievable); index schema reviews |
| Cross-tenant leakage | One user sees another tenant's data | Tenant ID stored as metadata; mandatory filter expressions |
| Index abuse | Direct access to search endpoint | Private Endpoint only; Entra ID auth; disable API keys |
4. Retrieval
| Risk | What can go wrong | Azure-native controls |
|---|---|---|
| Over-retrieval | Too many documents returned | Top-K limits enforced in code / APIM |
| Unauthorized data access | User retrieves data beyond clearance | Entra ID claims → filter expressions; row-level security (RLS) |
| Prompt injection via retrieved text | Instructions hidden in documents | System-prompt isolation; Content Safety scan on retrieved chunks |
5. Prompt Assembly
| Risk | What can go wrong | Azure-native controls |
|---|---|---|
| Prompt override | User says "ignore system prompt" | APIM prompt firewall; fixed system-prompt templates |
| Context stuffing | Excessive data injected | Prompt size limits in APIM |
| Tool abuse | Model calls unintended tools | Tool allow-listing; explicit tool schemas |
| Data leakage | Sensitive data added to prompt | Prompt-inspection policies; structured prompt formats |
6. Model Inference
| Risk | What can go wrong | Azure-native controls |
|---|---|---|
| Prompt injection | Model follows malicious instructions | Defender for AI runtime detection |
| Data exfiltration | Model outputs sensitive data | Defender for AI alerts; token anomaly detection |
| Abuse | Excessive or automated usage | Rate limiting (APIM); quotas in Azure OpenAI |
| Shadow AI | Unmanaged model usage | Defender for Cloud AI discovery |
7. Output
| Risk | What can go wrong | Azure-native controls |
|---|---|---|
| PII leakage | SSNs, emails, PHI in response | Azure AI Content Safety (PII detection); optional Presidio on Azure |
| Unsafe content | Toxic or policy-violating output | Content Safety filters |
| Hallucinated actions | Model invents actions/commands | Output schema validation; human-in-the-loop approvals |
Key security considerations
- The most dangerous risks span multiple stages. Prompt injection via retrieved text appears at both Retrieval (stage 4) and Model Inference (stage 6) — an attacker who successfully poisons a document at Ingestion is exercising a multi-stage attack chain, not a single exploit.
- Cross-tenant leakage at the Indexing stage is a governance failure, not just a technical one — mandatory filter expressions only work if every query path actually enforces them; a debugging tool or admin console that bypasses the filter reintroduces the exact risk it was built to prevent.
- Output validation is the last line of defense, not the first. By the time a hallucinated action or PII leak reaches Output, five prior stages have already had a chance to catch it — treat a repeated Output-stage failure as a signal to audit earlier stages, not just tighten the output filter.
- Rate limiting and quotas (stage 6) are a security control, not just a cost control — unmetered inference is also unmetered exfiltration bandwidth.
Recommended practices
- Map your own RAG pipeline to these seven stages explicitly, and confirm you have at least one control per stage before calling the pipeline "secured" — a single strong control at one stage does not compensate for a missing control at another.
- Enforce tenant and clearance filtering (Indexing and Retrieval stages) as mandatory query parameters the search service itself rejects requests without — not as an application-layer convention that can be bypassed.
- Scan retrieved content for injected instructions at Retrieval, before it's assembled into a prompt — waiting until Model Inference to catch it means the model has already seen the malicious content.
- Apply the same "log hash, not content" telemetry discipline from Chapter 6 to RAG-specific logs — prompt assembly and retrieval logs are just as capable of leaking sensitive data as application logs.
- Require human-in-the-loop approval for any output that triggers a downstream action (not just a response) — hallucinated actions are a distinct risk from hallucinated facts.
Technologies referenced
- Microsoft Defender for Storage — malware scanning at Ingestion.
- Azure AI Content Safety — pre-index scanning, retrieved-chunk scanning, and PII/unsafe-content detection at Output.
- Azure Key Vault — secret scanning policies at Ingestion.
- Azure AI Search — field-level access controls and mandatory filter expressions at Indexing.
- Azure API Management (APIM) — Top-K limits, prompt firewall, prompt-size limits, and rate limiting across Retrieval, Prompt Assembly, and Model Inference.
- Microsoft Defender for AI — runtime prompt-injection and exfiltration detection at Model Inference.
- Microsoft Entra ID — claims-based filtering enforced at Retrieval.
Key takeaways
- RAG is seven distinct security boundaries — Ingestion, Storage, Indexing, Retrieval, Prompt Assembly, Model Inference, Output — not one system to secure once.
- Multi-stage attacks (a poisoned document at Ingestion that triggers prompt injection at Retrieval and Model Inference) are the highest-risk pattern; single-stage controls don't stop them.
- Output-stage filtering is the last line of defense; recurring output failures are a signal to audit ingestion, indexing, and retrieval controls, not just tighten the output filter.