Securing RAG and AI Agents

Part 4 of the KIS AI Security Series: a practical guide to retrieval-augmented generation security, AI agent permissions, access control and monitoring for enterprise AI.

Why RAG and agents change the risk

Retrieval-augmented generation (RAG) connects a large language model (LLM) to your documents, databases or search index. AI agents go further: they can plan a task, call tools and take multi-step actions. Both patterns can make enterprise AI useful, but they also create more places where data, instructions and permissions can be confused.

RAG security is not simply “encrypt the vector database.” LLM security is not simply “write a stricter system prompt.” A trustworthy AI workflow must preserve the user’s identity, the source document’s permissions and the business owner’s approval at every step.

Part 1 covered AI security foundations, Part 2 mapped data flows, and Part 3 showed how prompt injection can make content fight back. This article joins those lessons into a design approach for AI agents and retrieval-based applications.

Four boundaries to design deliberately

1. Knowledge is curated, not simply connected

An index can faithfully retrieve outdated, duplicated or incorrectly classified material. A model may then produce a confident answer from a source your policy team retired last year. Retrieval quality is a security and governance concern, not only a relevance metric.

Remediation: assign an owner for every knowledge collection, record source version and review date, remove duplicates, tag sensitivity and exclude documents without an accountable owner. Treat deletion and retention changes as index changes that need testing.

2. Retrieval must respect the requesting user

A shared service identity can accidentally turn a personal assistant into a privileged search engine. If the application retrieves all documents first and asks the model to filter them later, the model has already seen data the user should not receive.

Remediation: authenticate the user before retrieval, apply object-level authorisation in the source or retrieval layer, and pass only permitted passages to the model. Test two accounts with different access rights and verify that denied content is absent from prompts, citations and logs.

3. Tools need narrower permissions than people

An agent that can read a calendar may not need to create meetings. An agent that can draft a purchase request should not be able to approve payment. Broad tools turn a small prompt-injection mistake into a business-impacting event.

Remediation: expose task-specific functions, allow-list destinations, validate parameters server-side, use short-lived credentials and require explicit approval for sending, deleting, publishing, changing production or moving money. Start with read-only tools while the workflow is being tested.

4. Multi-step plans need checkpoints

Agent behaviour is a sequence, not one answer. A harmless search can be followed by an unsafe export; a correct draft can be followed by an unauthorised send. A final response alone does not show what happened in between.

Remediation: define maximum steps, timeouts and spend limits. Add checkpoints after retrieval, before sensitive tool calls and before any external side effect. Store a correlation ID that links the user request, sources, tool arguments, approvals and outcome.

Real-world example: an IT operations agent

RAG security controls that teams can implement

Curate the source estate

Separate authoritative policy, working drafts and user-generated content. Keep document owner, classification, effective date and retention metadata alongside the content. When a document is withdrawn, remove it from retrieval and confirm the change with a test query.

Protect the embedding and index layer

Apply encryption, network controls, tenant separation and backup protection to embeddings and vector indexes. Embeddings can still reveal sensitive relationships, so do not treat them as anonymous. Restrict who can rebuild, export or debug the index.

Make citations part of the experience

Show the source title, version or link for answers that influence a decision. A citation is not proof of correctness, but it gives a reviewer a fast way to challenge a stale or irrelevant passage. Do not expose citations the user could not access directly.

Defend against indirect prompt injection

Scan and test documents, web pages and messages for instructions that conflict with the task. Keep untrusted content in a separate context, use delimiters, strip active content where appropriate and never let a retrieved passage change permissions or tool policy.

Monitor the workflow, not only the model

Track retrieval denials, unusual source combinations, repeated tool failures, new destinations, long agent plans and approval overrides. Alert on patterns that indicate boundary crossing. Protect telemetry because prompts, documents and outputs may contain sensitive business data.

A 30-day remediation plan

Questions for an enterprise AI security review

Can a user retrieve only what they could access in the source system? Can a document change the agent’s tool policy? Which agent actions require human approval? Are credentials short-lived and scoped to one task? Can investigators reconstruct every retrieval and tool call? What happens when a source is revoked, a model changes or monitoring detects a boundary crossing?

Use these questions with your application-security, privacy, cloud-security and AI-governance teams. For organisations in Singapore, align the workflow with your existing data-protection, regulatory-compliance and incident-response obligations rather than creating a separate AI silo.

Guidance to support your programme

The OWASP GenAI Security Project offers practical guidance for LLM security, prompt injection and agentic applications. The NIST AI Risk Management Framework provides a useful lifecycle lens for identifying, measuring and managing AI risk. Combine these references with least privilege, Zero Trust, secure software development and your organisation’s existing control framework.

Coming next: measuring AI security in operation

Part 5 will turn controls into operating metrics: what to measure, which alerts deserve attention, how to evidence human oversight and how to demonstrate that an AI security programme is improving over time.

KIS AI Security Series · Part 4. Examples are illustrative and intended to help teams discuss practical controls.