Scoping an enterprise RAG project: 7 decisions to document before the prototype

A RAG project is scoped before you choose a model, a vector database or an interface. The useful deliverable is not a list of components. It is a decision file that lets you answer, with evidence: which problem to solve, which data can be used, who can see what, how to measure quality and under which conditions to stop or continue.

In short

• Start with a business decision and a user scope, not with the technology.
• Require seven documented items before the prototype: use case, sources, rights, expected answer, evaluation, operations and go/no-go criteria.
• Treat the prototype as a decision test. If it cannot fail against criteria written in advance, it does not really reduce risk.

This guide offers a working framework for a CTO or Head of Data/AI. It replaces neither the security analysis nor the legal analysis specific to your context.

What should scoping produce?

Scoping must make a decision possible. At the end of this phase, the team should be able to write one of three conclusions:

1. Proceed to a prototype, with a known scope, data and evaluation criteria.
2. Reduce or change the use case, because a data, access or operations constraint makes the first hypothesis too risky.
3. Do not build a RAG, because the problem would be better solved by existing search, better documentation or a non-generative workflow.

This is the angle that avoids the most expensive false start: producing a convincing demo, then discovering that nobody can deploy it with the real permissions, data or responsibilities.

The NIST AI Risk Management Framework, published in 2023, offers a voluntary framework for building trustworthiness considerations into the design, development, use and evaluation of AI systems. Its playbook organises actions around four functions: govern, map, measure and manage. For a RAG, these four verbs are a good lens for scoping, without turning it into automatic compliance.

When scoping is also used to compare approaches or vendors, our RAG partner selection scorer (in French) helps prepare the evidence and questions to examine. It does not replace a review of the scope, the technical documents or the tests.

Which business decision should the RAG improve?

A use case worded as "build a chatbot on our documents" is too broad. Start from an observable decision or task: for example, finding a maintenance procedure, preparing a support answer from authorised documentation, or helping a team locate the applicable version of an internal policy.

For each case, record:

• the user concerned and their work context;
• the question or task they must complete;
• the cost of the current way of working;
• the expected answer and what the system must not do;
• the consequence of a wrong, incomplete or out-of-scope answer.

That last line matters. An assistant that points to a procedure and an assistant that proposes an operational decision do not have the same level of requirements, nor the same human validation protocol.

Which data sources are really in scope?

A list of connectable tools is not a data scope. For each candidate source, create a short sheet: owner, content type, freshness, quality, access rights, versioning and deletion procedure.

The useful question is not only "can we extract these documents?". It is also:

• Which version is authoritative in case of contradiction?
• How do you detect an update or a deletion?
• Which documents must be excluded at ingestion?
• Does the source already contain what is needed to cite an answer?

A credible prototype does not index everything. It selects a representative corpus, with its real problems: duplicated documents, contradictory versions, badly extracted tables, heterogeneous rights or outdated content. Otherwise the test mainly measures how artificially clean the demo set is.

To turn this selection into a usable decision, also document the data scope of a RAG on sensitive documents: admitted or excluded sources, transformations, rights, outputs and withdrawal mechanism. When the corpus changes, our method to version, update and withdraw a source document helps tie that decision to the chunks, indexes, caches and tests that must follow the change.

How do you define access rights before indexing?

Access control must not be added after the first demo. It determines which metadata to keep, how passages are retrieved and which tests to run.

Document at least:

Decision	Question to settle	Evidence expected in scoping
Identity	How does the system identify the user?	Authentication flow and available attributes
Authorisation	Which rule determines access to a document?	Mapping between business rights and indexed data
Revocation	What happens when a right is removed?	Delay, synchronisation mechanism and test scenario
Traceability	What must you be able to explain after an answer?	Logged events and validated retention period

Our article on access control in an enterprise RAG details the possible architectures. To prepare the decision before choosing a mechanism, also use our method to document access control architecture choices. Here the scoping point is simpler: no test set should be declared representative until it includes users with different rights and denied access attempts.

What answer is the system allowed to produce?

Define an answer contract before talking about prompts. It specifies, among other things:

• whether each answer must cite the documents used;
• when the system must answer that it does not know;
• which actions require human validation;
• which types of information must not be rephrased, aggregated or exported;
• how to behave when sources are missing, contradictory or too old.

This rule separates linguistic quality from operational quality. A fluent answer with no source does not let the user check the information. An answer that cites an unauthorised source is just as unacceptable. The answer contract must therefore be tested with the chosen permissions and corpus, not only with easy questions.

How do you prepare an evaluation that can invalidate the prototype?

The prototype must answer representative cases prepared before its evaluation. Build a test set with real questions or questions reviewed by business teams. For each one, store the expected answer, the authorised sources, the forbidden sources and a scoring rule.

A minimal grid separates five dimensions:

Dimension	Test question
Accuracy	Is the answer supported by the right document?
Grounding	Do the citations let you verify the answer?
Authorisation	Does the system expose only authorised passages?
Usefulness	Does the answer help complete the task without excessive rewording?
Robustness	Does the system recognise its limits on an out-of-corpus or ambiguous question?

Also measure negative cases: out-of-scope question, deleted document, contradictory source, user without rights and a malicious instruction placed in a document. These cases are not security details to postpone. They determine whether the behaviour can be operated.

To go further on building a test corpus, see our guide to building a representative RAG test set. The principle stays the same: the test set belongs to the use case, not to the model vendor.

Which operating conditions must you know before the build?

A RAG is a living system. Scoping must specify who updates the sources, who monitors incidents and who arbitrates a drop in quality. Write down from the start:

1. the business owner of the content and the technical owner of the system;
2. the frequency and mechanism for updating sources;
3. the events to trace in order to investigate an answer;
4. the thresholds that trigger a review, a restriction or a shutdown;
5. the user feedback path when an answer is wrong.

This preparation connects with production decisions: ingestion, retrieval, observability and permissions. The technical trade-offs are detailed in the five critical decisions of a production RAG.

Which risks should you treat as design assumptions?

Scoping is not about declaring a system "compliant" or "secure". It makes risks explicit and assigns an owner to each decision. NIST's GenAI profile, published in 2024, complements the AI RMF with considerations specific to generative AI. It can serve as a working source to qualify risks, but its use remains voluntary and must be adapted to the organisation's context.

For an enterprise RAG, examine at least:

• undue access to data through poor permission propagation;
• an unsupported answer or a citation of an outdated source;
• a hostile instruction contained in an indexed source;
• data disclosure in traces or observability tools;
• dependence on a vendor, a model or a critical source;
• the absence of an owner responsible for fixing the corpus and the evaluations.

The expected result is a list of risks tied to the prototype's scope, each with a mitigation, an owner and a test. It is not a generic register reused without reading.

Which go/no-go criteria should you set before launching the prototype?

End scoping with a decision page. It must contain thresholds suited to the use case, with no invented universal threshold. For example:

• the test corpus covers the chosen roles and critical documents;
• evaluated answers can be traced to authorised sources;
• denied-access tests have been run and documented;
• the business owner accepts the answer criteria and refusal cases;
• the technical team has an ingestion, operations and rollback plan;
• open risks are known, assigned and accepted at the appropriate level.

At this stage, technology becomes a means to test hypotheses. The prototype can then produce a decision: proceed, change the scope or stop. That is a more useful output than a simple demo.

Which checklist should you use for your next scoping meeting?

Bring these seven pieces of evidence to the decision meeting:

1. a use case worded as a task and a business decision;
2. a representative corpus, with its life cycle and exclusions;
3. a testable access rule, including for refusals;
4. an answer and citation contract;
5. an evaluation set that can reveal a failure;
6. an operating model with named responsibilities;
7. written criteria to proceed, reduce or stop.

If one of these items is missing, the prototype can still be useful for learning. It simply must not be presented as proof of production feasibility.

FAQ

• Should you build a prototype before scoping a RAG project?

  No. Scoping comes before the prototype and defines what the prototype must demonstrate. It can include limited technical exploration, but that exploration must serve a written decision hypothesis.

• How many documents do you need to scope an enterprise RAG?

  There is no universal number. Above all, the corpus must represent the document types, the hard cases and the access rights of the chosen scope. A small, varied and realistic corpus is more useful than a large but artificially clean one.

• What are the success criteria for a RAG prototype?

  They depend on the use case. Set them before the test: verifiable accuracy, citations to authorised sources, respect for permissions, business usefulness and safe behaviour outside the scope.

• Does the NIST AI RMF make a RAG project compliant?

  No. NIST presents the AI RMF as a voluntary risk management framework. It helps structure decisions, but it does not replace the regulatory, contractual, security or data protection analysis that applies to your organisation.

Methodology and sources

• NIST AI Risk Management Framework, version 1.0 published on 26 January 2023, consulted on 10 September 2026. NIST describes a voluntary framework to better manage risks related to AI systems.
• NIST AI RMF Playbook, consulted on 10 September 2026. It organises suggested actions under the Govern, Map, Measure and Manage functions and states that it is not a checklist to apply in full.
• NIST AI RMF: Generative Artificial Intelligence Profile, published on 26 July 2024, consulted on 10 September 2026.

The method presented in this article is an editorial scoping grid for enterprise RAG systems. It contains no client data, no mission metrics and no promise of results. Go/no-go thresholds must be defined with the business, security and compliance teams concerned.

---

Do you need to decide whether a RAG use case deserves a prototype, and on what scope? Describe your context to IgnitionAI. The goal of a first conversation is to clarify the decisions, dependencies and evidence to gather before committing to the build.