RAG on sensitive documents: how to define the data scope

For a RAG on sensitive documents, the data scope is not just a list of connected folders. It must state which task is assisted, which documents can be searched, who can view them, which transformations are allowed and how a change or a withdrawal takes effect. Without this decision file, the team can neither limit the pilot, nor explain an answer, nor extend the system in a controlled way.

In short

• Start with the task and the authorised population. A connector alone does not define an acceptable use.
• Describe each source all the way to the system's outputs: document, extraction, chunks, index, context sent to the model, answer and traces.
• Separate admitted, excluded, conditional and pending data. An indexable document is not automatically usable in every journey.
• Test withdrawal, rights change and missing-source cases before opening a wider scope.

Here, "sensitive" means information whose exposure, misinterpretation or excessive retention can have a significant impact on the organisation or on individuals. This can cover trade secrets, HR records, financial data, legal information or personal data. This working definition does not replace a legal qualification or the organisation's own classification policy.

Why is a list of documents not enough?

The same document can be useful to one team, reserved for another role, outdated, partly shareable or authorised only in a given context. Turning it into chunks and making it searchable also adds new places where its reach must remain understood: extraction, metadata, index, generation context, citations, logs and backups.

The right working object is therefore not an inventory of connectors. It is a sheet that answers an operational question: "can this information be used by this system, for this task, by this person, at this stage of the chain?"

The CNIL, France's data protection authority, recommends starting from the system's objective, the context of use, the excluded contexts, the sources used and the strictly necessary data. Its guidance concerns the design and development of AI systems involving personal data. For a RAG already planned for deployment, it does not replace the analysis that applies to the concrete case, but it provides a useful discipline to avoid starting from the available volume rather than the real need.

Which eight dimensions should you decide before indexing?

A data scope sheet can stay short. It must, however, cover the decisions that really change what the system sees or returns.

Dimension	Decision to write down	Example evidence to keep
Task	Which search or answer preparation is assisted?	Task description approved by the business owner
Population	Which roles use the RAG, and in what context?	Roles, spaces and excluded use cases
Source	Which system is authoritative for the document and its rights?	Owner, identifier, version and publication status
Categories	Which data families are admitted, conditional or excluded?	Internal classification and reasons for the decision
Transformations	Which steps are allowed: OCR, extraction, chunking, enrichment, summarisation?	Flow diagram and processing rules
Access	How do the source's rights limit retrieval and citations?	Authorisation rule and refusal scenarios
Outputs and traces	What can an answer, a citation, a log or a feedback item contain?	Answer contract and logging policy
Life cycle	How are updates, withdrawal, retention and incidents handled?	Trigger, owner and revocation test

This sheet must be specific to a use case. "Internal documentation" is too broad a label. It says nothing about applicable versions, access rules, attachments, drafts, pending decisions or information that must not leave a team.

How do you classify sources without creating an abstract register?

For each candidate source, assign a usage status rather than a simple "connected" flag:

1. Admitted. The source is useful for the use case, its owner is identified and its rules can be applied and then tested.
2. Admitted under conditions. A restriction must be respected, for example a set of roles, part of the corpus, a business validation or a specific data processing rule.
3. Excluded. The content does not serve the task, its status is uncertain or its exposure would create a risk not accepted for this pilot.
4. Pending decision. The team does not yet have an answer on the owner, the life cycle, the rights, the transformations or the withdrawal path.

This classification avoids a common mistake: treating the absence of a decision as a temporary authorisation. A pilot can move forward with a reduced corpus. It should not silently absorb sources that do not yet have an owner or a verifiable rule.

The method for scoping a RAG project helps tie this choice to a business decision. The data scope is its concrete side: it makes visible what the prototype is allowed to learn from the corpus and what it must ignore.

How do you follow a document through the whole RAG chain?

A document can be authorised in its original application without being authorised in every representation created for the RAG. Map the full path before plugging in a model:

1. Read by the connector. Which identity reads the source, and with what reach?
2. Extraction and normalisation. Which text, table, metadata or attachment is kept after conversion?
3. Chunking and enrichment. Which chunks are produced, and to which document, version and access rule do they remain attached?
4. Indexing and retrieval. Which filter limits results to the role asking the question?
5. Generation context. Which passages can be sent to the model to answer?
6. Answer and citation. What can be rephrased, displayed or referenced to the user?
7. Traces and feedback. What is kept to diagnose a gap, with which access and for how long?

This map turns a vague discussion about confidentiality into testable controls. If the team does not know where a passage is stored, transmitted or reused, it cannot demonstrate that a withdrawal or a restriction has taken effect.

The security recommendations of ANSSI, France's national cybersecurity agency, on generative AI systems point out that the architecture involves several populations, retrieval components, filters and connections to business resources. They address the security of the architecture, not the full range of privacy or data protection questions. These two readings must therefore be organised without confusing them.

How do you decide what to exclude from the first pilot?

The first corpus should not aim for maximum coverage. It must make it possible to test a useful task, realistic sources and the system's limits. Prefer explicit exclusions to promises of later processing.

A source can be excluded from the pilot when at least one question remains unanswered:

• the business owner does not confirm its use for the chosen task;
• the authoritative version cannot be identified;
• the rights cannot be propagated or checked on refusal cases;
• the document contains a category of data that requires a decision or review not yet obtained;
• the team does not know how to withdraw the content, its chunks and its derived representations;
• the quality of the content does not let you distinguish applicable information from a draft, a preparatory note or a superseded version.

This rule does not mean that an excluded source is permanently unusable. It means it must not become a side effect of the first connector switched on. A reduced pilot can produce a more defensible decision than a large index whose boundaries nobody controls.

Why do access rights not replace the scope?

A rights filter is necessary, but it answers only one part of the decision: who can retrieve what. On its own, it does not answer these questions: is this source relevant to the task, is this version still applicable, can the answer cite this passage, can the content be sent to an external service or kept in the traces?

Document the business rule first, then the technical path that enforces it. Our guide on access control architecture choices in a RAG details the elements to link between identity, source, index, refusal, revocation and audit. The scope sheet completes this work by deciding which objects are allowed to enter that chain.

When should you involve the security and data protection functions?

The answer depends on the nature of the data, the use and the level of risk. Do not turn this page into legal advice. However, trigger an explicit review when the scope reveals personal data, sensitive or highly personal information, a vulnerable population, combined datasets, decisions that affect individuals, or an exposure that is hard to control.

The CNIL states that a data protection impact assessment may be required when processing is likely to result in a high risk to rights and freedoms. It also specifies that this assessment must evolve with the characteristics of the processing and the risks observed. For a RAG, the scope sheet is a practical support: it makes readable the operations, data, populations, controls and assumptions that the relevant functions must examine.

How do you check that the scope remains valid in production?

The scope is not a frozen launch document. It must be replayed when a change opens a new risk or alters the observed behaviour. Prepare simple triggers: new source, new document category, new role, connector change, access rule change, new vendor, change in logging or a reported incident. To carry this change through versions, chunks, indexes and tests, also see our method to version, update and withdraw a source document in a RAG.

At each change, the team must be able to answer:

1. which scope decision changes;
2. which data, users and outputs are affected;
3. which retrieval, refusal and withdrawal tests must be rerun;
4. who approves the extension, the restriction or the rollback;
5. which version of the corpus and rules explains the result.

NIST's generative AI profile proposes a voluntary risk management approach to adapt to the organisation's context, objectives and resources. This logic is especially useful here: the scope is not meant to certify a system, but to link its risks, evidence and decisions to the use case actually opened.

To check the effects of a change on retrieval, answers, rights, latency and cost, also use our method to evaluate a RAG in production. Tests do not prove that a scope is perfect. They show that the declared limits are still being applied.

Which evidence should you obtain before opening a wider corpus?

Before widening the scope, gather at least the following seven items:

1. a clearly described task and population;
2. a list of sources with owner, status, version and usage rules;
3. explicit exclusions and the reasons for them;
4. a map of transformations and of the places where content is stored or transmitted;
5. a testable access rule, with authorised and refused scenarios;
6. an update and withdrawal mechanism whose effect can be verified;
7. a written decision on open risks, owners and conditions for extension.

If any of these items is missing, the team can still learn with a narrower scope. It must not present the system as ready to search all of the organisation's sensitive documentation.

FAQ

• Can you index a whole document space to go faster?

  Not by default. Start with the sources needed for a specific task and check their owner, status, rights, transformations and life cycle. A reduced but testable corpus helps you decide more than a very large set whose limits remain unknown.

• Should documents containing personal data be excluded?

  Not automatically. Their processing must be justified, limited to the need, tied to the applicable rules and reviewed by the relevant functions depending on the context. When a decision is missing, classify the source as pending or exclude it from the pilot rather than assume that technical access is enough.

• Is an access filter in the vector database enough?

  No. It can limit retrieval, but it does not decide whether the source is relevant, up to date, authorised for the use case, transmissible to the model or retainable in the traces. The scope must cover the whole chain, from the source to the answer and to withdrawal.

• When should you update the scope sheet?

  At every change that affects sources, data categories, roles, rights, connectors, outputs, logs or retention rules. The update must lead to a review and appropriate tests, not just a new date on a document.

Methodology and sources

• CNIL, Tenir compte de la protection des données dans la conception du système (taking data protection into account in system design, in French), 8 April 2024, consulted on 21 September 2026. The guidance covers, among other things, the objective, excluded contexts, sources, minimisation and the validation of design choices for AI systems involving personal data.
• CNIL, Réaliser une analyse d'impact si nécessaire (carrying out an impact assessment where necessary, in French), 8 April 2024, consulted on 21 September 2026. The guidance presents the situations where a DPIA may be required and how it evolves in light of the processing risks.
• ANSSI, Recommandations de sécurité pour un système d'IA générative (security recommendations for a generative AI system), 29 April 2024, consulted on 21 September 2026. The guide covers the security of a generative AI system's architecture, including retrieval components and connected business resources.
• NIST, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, July 2024, consulted on 21 September 2026. This voluntary profile proposes governance, mapping, measurement and risk management actions to adapt to the context.

This page is a technical scoping method. It does not legally qualify data, does not determine the applicable obligations and is not a promise of security or compliance. Scope decisions must be validated with the business, security, data protection and operations owners concerned.

---

Do you need to decide which data a RAG can really consult before opening a connector? Describe your context to IgnitionAI. A first conversation can help turn the sources, rights, exclusions and expected evidence into a decision scope.