Moving from an LLM prototype to production: go/no-go criteria
An LLM prototype alone does not justify a deployment. Decide on a scope, evaluation evidence, controls, operations and a documented decision.
An LLM prototype can show that an interaction is possible. It does not yet show that the system can be deployed for a real business scope. A useful go/no-go decision rests on written evidence: use case, data and rights, representative evaluation, expected behaviour when something deviates, operating conditions and the person accountable for the decision.
In short
- Decide on a defined use case, set of users and risk level, not on a general demo.
- Evaluate the candidate version on representative tasks and refusal cases before widening the scope.
- NIST recommends establishing performance or assurance thresholds and reviewing them in deployment decision processes. These thresholds must be adapted to the context, not copied from another project.
- A no-go or a reduced scope can be a valid scoping conclusion when a necessary piece of evidence is missing.
This grid helps a team prepare a technical and business decision. It does not replace the security, compliance or data protection analysis, or the organisation's own approvals.
Why is a successful demo not enough?
A demo usually shows a few chosen inputs, with a limited corpus, rights and load. Production raises additional questions: who uses the system, which sources it can consult, what is acceptable when it cannot answer, how a change is evaluated and who steps in when quality degrades.
NIST's generative AI profile proposes establishing minimum performance or assurance thresholds and reviewing them in deployment approval policies and processes. It provides neither a universal threshold nor a certification. The expected level depends in particular on the task, the people affected and the consequences of an incorrect answer.
The go/no-go question therefore becomes: "do we have enough evidence to open this system to this scope, with these limits and these owners?" It is more demanding than: "did the prototype give a good answer?"
Which scope should the decision cover?
A production decision must name the scope rather than promise a general capability. Start by documenting the following:
| Element | Question to settle | Evidence to prepare |
|---|---|---|
| Business task | Which decision, search or action should the system help complete? | Concrete scenarios and expected behaviour |
| Users | Which roles may use the system? | Testable identity and authorisation rules |
| Data and tools | Which sources or operations are in the first version? | Inventory, owners, exclusions and update cycle |
| Limits | What must the system do out of scope or without a sufficient source? | Refusal cases, expected message and escalation path |
| Risk level | Which errors are unacceptable or require human review? | Decision criteria approved by the owners concerned |
This work avoids confusing a model capability with permission to use it in any process. For a document system, our guide on scoping an enterprise RAG project details how to link a use case, a corpus, rights and a prototype decision.
Which evidence should you gather before a go?
A decision file does not need to be long. It must be readable and let you find the evidence behind each conclusion. Six categories are usually needed.
- A verifiable business intent. Describe the task, the users and what would make the result useful or unusable.
- A data and rights scope. List the sources, their owners, the exclusions and the controls that must stay in force.
- A representative evaluation set. Prepare normal, ambiguous, out-of-scope and forbidden cases before comparing versions.
- Interpretable results. Keep the outputs, scoring rules, tested versions and the gaps that require a decision.
- An operations plan. Specify the useful traces, monitoring, alerts, incident handling and the ability to restrict or roll back.
- An assigned decision. Name the person or body that accepts the scope and the open risks, with the review conditions.
This list is a working summary, not a new standard. It makes operational the governance, mapping, measurement and risk management topics described by NIST.
How do you build an evaluation that can actually lead to a no-go?
A test only reduces risk if it can invalidate a hypothesis. Build the evaluation set from real business tasks or tasks reviewed by people who know the domain. Add the cases that unsettle the demo: an ambiguous request, an outdated source, contradictory data, a user without rights, a hostile instruction in a document or a tool call that goes beyond the planned scenario.
For each case, record the request, the role used, the authorised sources or actions, the expected behaviour and the review rule. Microsoft Foundry's observability documentation describes pre-production evaluation as a step that tests, among other things, edge cases, robustness, task adherence, groundedness, relevance and safety before deployment.
For a RAG, it helps to separate document retrieval, the generated answer, rights, latency and cost. Our method to evaluate a RAG in production explains why a single good answer does not let you diagnose a complete system.
Which go/no-go criteria should you write down?
The criteria must answer observable questions. They must not reuse generic percentages found in a benchmark. A decision table can take this form:
| Area | Decision criterion to adapt | Decision if the evidence is missing |
|---|---|---|
| Usefulness | Priority tasks have been tested with a review rule the business understands. | Reduce the scope or complete the test set. |
| Data and access | Sources, exclusions and expected refusals are known and verifiable. | Block opening to roles or sources not covered. |
| Quality and safety | Critical cases have an expected answer, a trace and a review owner. | Fix, add a control or keep the case out of scope. |
| Operations | The team can observe useful errors, restrict the system and investigate a gap. | Prepare operations before widening access. |
| Change | A change of model, prompt, index or tool can be compared with a reference. | Keep the reference version and organise a new evaluation. |
A go can be conditional: open to a defined user group, limit the data or disable an impactful action. A no-go is not necessarily abandoning the use case. It can conclude that a control, a dataset, an access rule or an operations mechanism must be handled before the next evaluation.
How do you prepare operations before opening?
Production does not start with a dashboard. The team must know which information it will keep to understand an answer or an action: system version, input and output in line with data rules, sources or tools used, control decisions, duration and execution result.
Microsoft describes production monitoring as tracking operational metrics, continuous evaluations on a sample of traffic, scheduled evaluations on test sets, and alerts when quality or safety thresholds are not met. The choice of metrics and trace data remains specific to the context. It must respect confidentiality rules and avoid turning observability into an uncontrolled copy of business data. Our method to measure the cost of an LLM system details the units to link to each journey before comparing versions.
For agents that call tools, operations must also let you find the proposed operation, the control applied, any human decision and the result. Our article on human approvals before an AI agent's actions covers this control point by type of action. The guide on AI agent traceability explains how to link these events to a single execution.
Who should make the final decision?
The person who approves the production release must be able to read the evidence and know the accepted limits. Depending on the case, the decision can involve a business owner, a technical lead, security, data protection, operations or the owner of the connected tool. The point is not to have every role approve every detail, but to make sure no known constraint is left without an owner.
Document at least:
- the authorised scope and the exclusions;
- the criteria used and the results observed;
- the open risks explicitly accepted or postponed;
- the owner responsible for monitoring and responding to a deviation;
- the date or event that will trigger a review of the decision.
This file distinguishes a deployment decision from a favourable impression after a demo. It also makes a rollback or a scope reduction easier to explain. To turn this capability into an operating procedure, define the conditions for suspending, rolling back and escalating a GenAI system before opening.
A go decision remains incomplete if the team that must operate the system cannot apply the procedures, prepare a change or reduce the scope. Our guide on handing over a GenAI system to internal teams proposes a takeover exercise and a handover file to check this capability before delivery.
FAQ
Is a successful LLM prototype ready for production?
No. It can demonstrate a technical capability, but production also requires a defined scope, representative evaluations, access or action controls, operating conditions and an assigned decision.
Which thresholds should you use for an LLM system go/no-go?
There is no universal threshold. Criteria must be set according to the task, the users, the accepted risk and the consequences of a deviation. They must be written before the decision and tied to verifiable evidence.
Does a no-go mean abandoning the use case?
Not necessarily. A no-go can lead to reducing the scope, completing the test set, handling an access rule or preparing operations before a new evaluation. The decision must state what is missing and who is responsible for it.
Should a system be re-evaluated after a model or data change?
Yes. A change of model, prompt, corpus, index or tool can alter the system's behaviour. Compare the candidate version with a reference on the relevant cases and record the decision before rolling out the change more widely.
Methodology and sources
- NIST, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, published on 26 July 2024, consulted on 13 September 2026. Among other things, the profile presents establishing performance or assurance thresholds and reviewing them in deployment approval processes.
- Microsoft Foundry, Observability in Generative AI, consulted on 13 September 2026. The documentation describes pre-production evaluation and post-deployment monitoring of generative AI applications and agents.
- Microsoft Azure Architecture Center, Generative AI Operations for Organizations with MLOps Investments, consulted on 13 September 2026. This guide presents the evaluation, deployment, inference, monitoring and feedback loops for GenAI systems.
The method proposed is a decision grid for teams preparing an enterprise GenAI system. It contains no client results, prices, timelines or promises of security, compliance or performance. Criteria and responsibilities must be validated for each organisation.
Do you need to turn an LLM prototype into a defensible deployment decision? Describe your scope to IgnitionAI. A first conversation can help identify the evidence, controls and responsibilities to document before opening to production.