Direct answer: Do not use a demo set as the acceptance set. Build a separate "golden set" for AI acceptance: sample real tasks, stratify by scenario/risk, include negative and safety cases, double-label and adjudicate, freeze the version, then set launch gates for task success, citation correctness, refusal/safety, latency, human takeover and cost per task. Re-run regression whenever the model, prompt or knowledge base changes. Acceptance is not one demo; it is whether this set can be reproduced.
This report is for AI project owners, product/business, data/labelling, QA, procurement and vendor management. It covers acceptance evaluation sets for language-based AI projects such as agents, RAG, AI customer service and document Q&A. Information is current to 30 October 2026. It is public standards, governance frameworks and project method, not certification or a promise about any model, vendor, product, accuracy or compliance; specific methods, sample sizes, gates and contract terms are governed by business risk, the standards, contracts and third-party evaluation/legal advice. Visual quality-inspection sample size and confidence intervals are covered separately; this article does not repeat them.
A demo set is not an acceptance set; the acceptance set must be independent, frozen and reproducible. Acceptance is not one accuracy number—it is task success, citations, safety, latency, takeover and cost by scenario/risk. A golden set needs an owner, version, labelling guide and adjudication record; otherwise it is just another subjective answer.
Why can't a demo set be the acceptance set?
Demo sets usually come from training/prompt-tuning samples, making them prone to overfitting, cherry-picking, missing negatives and safety boundaries. The acceptance set must be independent, frozen and reproducible.
In a vendor demo, questions are often selected, prompts are tuned, the knowledge base is cleaned, and answers may even be prepared by a person. Such a "demo set" cannot prove post-launch performance. Common problems: overlap with training/tuning data causing leakage; only high-frequency, easy questions, with no long tail, negatives, refusals or safety cases; no frozen version, so results are not comparable after a model/prompt change; no labelling guide or adjudication, so the "gold answer" is one person's judgement. Acceptance is not "can it demo"; it is "does it still reproduce on another set of real tasks".
| Dataset | Source | Use | Not for |
|---|---|---|---|
| Demo set | Vendor-selected/tuned cases | Show capability, discuss scenarios | Acceptance conclusion |
| Training/tuning set | Historical tasks, synthetic data, annotations | Training, prompt/knowledge-base tuning | Independent acceptance |
| Development validation set | Samples held out during development | Parameter tuning, vendor selection, iteration | Repeatedly "gaming" it and still calling it acceptance |
| Golden acceptance set | Stratified sample of real tasks, independent of training/tuning | Launch acceptance, vendor comparison | Giving it to the vendor for training/tuning |
| Regression set | Golden set + historical failures/incidents | Regression after model/prompt/knowledge-base changes | Replacing one-off acceptance |
Dataset split is an editorial method; isolation, holdout and version control depend on project risk, vendor contracts and internal policy.
What goes into a golden set?
A golden set must cover main scenarios, long-tail/boundaries, negatives, safety/refusal, multilingual and human takeover. Every sample needs input, expectation, criteria and risk level.
A golden set is not "export a batch of historical questions". It must represent the post-launch distribution and cover low-frequency, high-risk scenarios. Each sample should include at least: task/scenario, user input (with context), expected result/criteria, risk level, source channel, annotator/adjudicator and version. For an agent, also record available tools, permissions, expected tool path and forbidden actions. For RAG, record the document/passage that should be cited and the "don't know" boundary.
| Sample type | Example | What to judge | Suggested share |
|---|---|---|---|
| Main/high-frequency | Quote, product specs, after-sales, order status | Task success, factual correctness, format | 40–50% |
| Long tail/boundary | Multi-condition, ambiguity, missing parameters, cross-document | Clarification, refusal, citation | 20–25% |
| Negatives/should not answer | No evidence, unauthorised, out of scope, outdated | No fabrication, correct refusal/escalation | 10–15% |
| Safety/compliance | Privacy, permissions, sensitive actions, prompt injection | No leakage, no privilege escalation, auditable | 10–15% |
| Multilingual/channel | English, dialect, email/chat/form | Language consistency, channel fit | 5–10% |
| Human takeover | Low confidence, complaint, complex negotiation | Timely escalation, complete context | 5–10% |
Shares are editorial examples, not a standard; adjust to business distribution, risk and historical incidents, and state the assumptions in the report.
How to stratify and size the sample?
Stratify by scenario/risk/channel first, then set a minimum per stratum. High-risk scenarios need full coverage of critical cases plus statistical sampling; one "30" or "100" does not fit all.
There is no universal sample-size number. Low-frequency, high-risk scenarios (privilege escalation, privacy, safety) cannot be under-tested because they are rare; high-frequency, low-risk scenarios can use statistical sampling to estimate overall performance. Start with 8–12 main scenarios, then classify into three risk tiers: high-risk critical items (full coverage of critical cases + random sample), medium-risk core items (e.g. 30–50 per scenario as an example), low-risk long tail (sample by distribution). The table below is an example assumption, not a standard; actual counts depend on variance, risk, budget and acceptable confidence.
| Tier | Sampling | Example size | Note |
|---|---|---|---|
| High-risk critical | All critical cases + random supplement | 100% critical + 50–100 per type | Safety/permissions/privacy/amount/contract |
| Medium-risk core | Stratified random by scenario | 30–50 per scenario; 300–500 total | Quotes/specs/after-sales/orders |
| Low-risk long tail | By channel/language/intent | 30+ per stratum | Dialect/email/multilingual |
| Negatives/refusal | By failure type | 30–50 per type | No evidence/unauthorised/outdated/ambiguous |
| Regression set | Historical failures + incidents + golden subset | Cover all fixed defects | Run on every change |
Sample sizes are illustrative assumptions, not an industry standard or statistical guarantee; high-value/high-risk projects should involve data/statistics/QA and state confidence intervals and limitations.
How to manage labelling, adjudication and versions?
Use a labelling guide, double review, disagreement adjudication, frozen versions and access control together. A "gold answer" without an adjudication record is just another person's opinion.
Golden-set quality depends on the labelling process, not just the number of labels. Write a guide first: what counts as task success, correct citation, when to refuse/escalate, what counts as leakage/privilege escalation, and how to judge multilingual answers. Then use at least two annotators/reviewers, send disagreements to adjudication, and feed the conclusions back into the guide with a version. Freeze the golden-set version, restrict access, and give the vendor only a masked test interface or controlled samples—not the acceptance set for training/tuning.
| Step | Practice | Evidence | Owner |
|---|---|---|---|
| Labelling guide | Define task success/citation/refusal/safety/format with positive and negative examples | Guide version, training record | Product/QA |
| Double labelling | At least two independent annotators/reviewers | Labels, agreement statistics | Data/business |
| Disagreement adjudication | Third reviewer/expert decides; update the guide | Adjudication record, revision log | Business owner |
| Version freeze | Bind samples, guide, model/prompt/knowledge-base versions | Version number, change log | QA |
| Access control | Acceptance set not used for training/tuning; vendor access controlled | Permission list, access log | Data/security |
| Holdout/rotation | Keep an independent holdout; add new samples periodically | Holdout list, rotation record | QA/data |
Labelling process is an editorial suggestion; specific agreement metrics, adjudication levels and access control depend on project risk, data sensitivity and contracts.
How to set metrics and launch gates?
Do not look only at accuracy. Task success, citation correctness, refusal/safety, format, latency, human takeover and cost per task need scenario-specific gates and failure handling.
"Accuracy" in language AI can hide problems: the answer looks right but cites the wrong passage; it should refuse but fabricates; the format is right but the action exceeds permission; the task succeeds but human takeover is high; latency is too long for customers to use. Choose 3–5 primary metrics per scenario, then set hard gates (safety/permissions/citations) and soft gates (latency/cost/experience). Gates must be written into the acceptance plan/contract before results, not explained afterwards.
| Metric | Definition | Applies to | Example gate |
|---|---|---|---|
| Task success | Completes the user task within the defined scope | Support/quote/query | Low risk ≥90%; high risk by scenario |
| Citation/grounding | Answer has a verifiable source and the cited passage is correct | RAG/document Q&A | Critical items 100% traceable |
| Refusal/safety | Correctly refuses or escalates no-evidence/unauthorised/sensitive requests | All scenarios | Safety-critical 100% pass |
| Format/tool use | Structured output, tool parameters and permission boundaries correct | Agent/automation | Critical tools 100% correct |
| Latency | P50/P95 response time | Online support/sales | Set by channel experience |
| Human takeover | Share requiring human takeover/correction | Support/sales | Set cap and review reasons |
| Cost per task | Model/tool/human/ops per successful task | Scale-up | Compare with human baseline/budget |
Gates are illustrative assumptions, not standards; set them by business risk, human baseline, customer experience and budget, and write them into the acceptance plan.
How to handle regression and change management?
Any change to model, prompt, knowledge base, tools or data triggers regression. Use the frozen golden set + regression set + small-traffic canary; roll back if it fails.
AI projects do not stay stable: model versions update, prompts change, knowledge bases add documents, tools/permissions change and business rules change. Every change can degrade previously good answers. Split a regression set from the golden set, covering historical failures, incidents, critical safety and core scenarios; run regression before the change, canary after the change, roll back on anomalies, and add new failures to the regression set.
| Change | Trigger | Regression scope | Pass standard |
|---|---|---|---|
| Model/version | Model swap, upgrade, routing change | Full golden + regression set | Hard gates not lower; core metrics not lower |
| Prompt/policy | Role, format, refusal policy change | Affected scenarios + safety | Task success/safety/format pass |
| Knowledge base | Add/remove documents; chunk/index change | Citation/Q&A/conflict samples | Correct citation; no outdated answer |
| Tools/permissions | New API; permission/parameter change | Tool calls + privilege cases | Correct parameters; no escalation |
| Business rules | Price/policy/process change | Affected scenarios + boundaries | New rule covered; old answer removed |
Regression scope is an editorial suggestion; change management, canary share and rollback time depend on system risk and operational capability.
What should supplier contracts and acceptance deliverables include?
Contracts should specify golden-set ownership, test access, no-training, reproducibility, defect repair, acceptance gates and exit handover. Otherwise acceptance can only look at a demo.
AI acceptance disputes are often not "is it good" but "on which set, which version, which gate, and who reproduces it". Write the evaluation set and method into the contract: who owns the golden set, whether the vendor can access it, whether it promises not to train on it, the test environment/API, result reproduction, defect repair, what happens if acceptance fails, and how data/logs/configuration are handed over at exit. Do not accept "we tested 95% internally" without samples, versions and definitions.
| Clause | What to write | Acceptance evidence |
|---|---|---|
| Golden-set ownership | Company-owned/jointly owned; vendor use controlled | Inventory, authorisation, access log |
| No training | Acceptance set not used for training/tuning/prompt optimisation | Undertaking, isolated environment, audit right |
| Test access | Test API/environment, masked samples, call limits | API docs, account/permission record |
| Reproduction | Model/prompt/knowledge-base versions, parameters, randomness settings | Experiment record, version list |
| Acceptance gates | Hard/soft metrics, stratified results, failure handling | Acceptance plan, sign-off page |
| Defect repair | Repair deadline for failed samples; regression requirement | Defect ticket, regression report |
| Exit handover | Data/logs/configuration/prompts/knowledge base/accounts handover and deletion | Handover list, deletion proof |
| Third-party evaluation | Whether a third party can review disputes; cost/confidentiality | Third-party report, NDA |
Contract clauses are editorial suggestions, not a substitute for legal review; specific IP, data, acceptance and exit terms depend on the project/vendor negotiation.
90-day build and pre-launch checklist
Weeks 1–2 define scenarios and risk; weeks 3–4 sample and write criteria; weeks 5–6 label/adjudicate; weeks 7–8 run gates; weeks 9–12 regress, canary, accept and archive.
If the project is late in POC, treat "building the set" as a separate workstream, not something the vendor does on the side. The company needs an owner, labelling resources, business experts and an acceptance signatory; the vendor can provide tools and masked samples but cannot be both player and referee.
| Phase | Actions | Outputs | Owner |
|---|---|---|---|
| Weeks 0–2 | Scenario list, risk tiers, data sources, vendor isolation plan | Scenario/risk table, sampling plan | Product/business |
| Weeks 3–4 | Sample real tasks, mask, define sample fields and criteria | Golden set v0.1, labelling guide | Data/business |
| Weeks 5–6 | Double labelling, adjudication, add negatives/safety/long tail | Golden set v1.0, adjudication record | QA/business |
| Weeks 7–8 | Run models/vendors, calculate metrics, stratified report | Evaluation report, gap list | QA/data |
| Weeks 9–10 | Regression after fixes/tuning, reproduce, check contract gates | Regression report, acceptance conclusion | Product/procurement |
| Weeks 11–12 | Small-traffic canary, monitoring, rollback plan, next evaluation | Canary report, operations handover | Ops/owner |
| Acceptance item | Pass standard | Evidence | If failed |
|---|---|---|---|
| Independence | Acceptance set not used for training/tuning; vendor access controlled | Permission/access log, undertaking | Replace/isolate/audit |
| Coverage | Main/long-tail/negative/safety/multilingual/takeover complete | Stratification table, sample list | Add/re-sample |
| Labelling quality | Double review/adjudication/guide version complete | Labels, agreement stats | Re-label/adjudicate |
| Reproducibility | Model/prompt/KB/parameter versions reproducible | Version list, experiment record | Add record/re-run |
| Gates | Hard gates 100%; soft gates pass or accepted in writing | Stratified report, sign-off | Fix/reduce scope/reject |
| Regression | Historical failures/incidents pass | Regression report | Fix/rollback |
| Canary/rollback | Small-traffic monitoring and rollback plan usable | Canary report, plan | Pause launch |
| Handover | Data/logs/config/accounts/knowledge base handed over | Handover list, deletion proof | Complete handover/deduct |
The 90-day plan and acceptance checklist are editorial suggestions; specific methods, gates, contracts and launch decisions depend on project risk, the standards and internal governance.
Who should read this, who should not, and what decision does it support?
It fits companies selecting, piloting or launching AI agents, RAG, customer service or document Q&A. It does not fit teams without a business owner/labelling resources, or those wanting a demo to replace acceptance.
Who should read: AI project owners, product/business, data/labelling, QA, procurement and vendor management; teams doing agent, RAG, AI customer-service or document Q&A selection, POC or launch acceptance; companies needing to put "effectiveness" into contracts, acceptance plans and regression. Who should not: teams without a business owner, labelling resources or real task samples; projects wanting to replace acceptance with a vendor demo or internal screenshot; teams treating one accuracy number as the whole conclusion without safety or human takeover. Decisions supported: build a golden/regression set, stratified sampling, labelling/adjudication, metrics/gates, supplier contract, 90-day plan and launch acceptance.
FAQ
How large should the golden set be?
There is no universal number. Stratify by scenario/risk first, then set a minimum per stratum; high-risk safety/permission/amount scenarios need full coverage of critical cases plus random supplement; high-frequency core scenarios can start at 30–50 per stratum, often 300–500 total as an example assumption. The final size depends on distribution, variance, acceptable confidence and budget; state the limitations in the report—do not use one "30" or "100" for everything.
The vendor says it tested 95% internally. Can we trust it?
Ask: which set, how many samples, which version, who labelled, what definition, and can it be reproduced. A "95%" without a sample list, labelling guide, version record and raw results is not an acceptance conclusion. Re-test on your own golden set/controlled environment, covering at least core scenarios, negatives and safety items.
Can the golden set be given to the vendor for tuning?
Not advisable. Once the acceptance set enters training/tuning/prompt optimisation, it loses independence. Give the vendor a masked test interface or controlled samples, but write "not for training/tuning" into the contract and keep access logs/audit rights; use a separate development set for tuning.
Is 90% accuracy enough to launch?
Not necessarily. It depends on scenario and failure cost: 90% may be acceptable for low-risk Q&A, but critical safety/permission/amount/contract items must pass 100%; also check citation correctness, refusal, format, latency, human takeover and cost per task. Write gates by scenario into the acceptance plan—not one overall accuracy.
Do we need to re-test after model/prompt updates?
Yes. Changes to model, prompt, knowledge base, tools/permissions or business rules can change results. Run the regression set (historical failures/incidents + core scenarios) on every change; run the full golden set for high-risk changes; canary small traffic, roll back on anomalies, and add new failures to the regression set.
We have no labelling team. Can the vendor label for us?
The vendor can provide tools/first-pass labels, but the company needs business-expert review/adjudication and final sign-off; otherwise the "gold answer" becomes the vendor's standard. At minimum, have a labelling guide, double review/sampling, disagreement adjudication and version records; critical safety/permission samples should be labelled by the company.
Bottom line
An AI acceptance evaluation set is not a demo set: sample real tasks by stratum and build an independent, frozen, reproducible golden set covering main/long-tail/negative/safety/multilingual/takeover cases; double-label and adjudicate with versions; set gates for task success, citations, safety, format, latency, takeover and cost; regress on every change; and write ownership, no-training, reproduction and exit into the contract.
Method and limits
This article is public standards, governance frameworks and editorial method, not certification, a promise or legal advice for any model, vendor, product, accuracy, evaluation set or project. It uses no unpublished client data, evaluation sets or vendor contracts. The dataset split, sample sizes, labelling process, metrics/gates, regression/contract checklists and 90-day plan are editorial suggestions; specific methods, sample sizes, gates and contract terms are governed by business risk, the standards, third-party evaluation/legal advice. Standards/governance frameworks change; the latest official text governs.
Sources
Information is current to 30 October 2026; each official/primary page governs its own scope and version.
- AI Safety Governance Framework 3.0 (TC260, 14 September 2026): tc260.org.cn. Risk governance/assessment framework for designing risk tiers and acceptance boundaries.
- GB/T 45654-2025, Cybersecurity technology—Basic security requirements for generative artificial intelligence service (effective 1 November 2025): National Standards Full-text Platform. Generative-AI service security requirements and assessment baseline.
- GB/T 45288.1-2025, Artificial intelligence—Large-scale model—Part 1: General requirements (effective 28 February 2025): National Standards Full-text Platform. General large-model requirements for capability/safety/lifecycle baseline.
- National AI Industry Comprehensive Standardization System Construction Guide (2024 edition) (four departments, July 2024): gov.cn. Policy context for AI standards/evaluation/service specifications.