The main finding: enterprise AI agents have entered production in 2026, but operating capability has not kept up. LangChain’s 2026 survey finds that 57.3% of respondents have agents in production and 89% have some observability, but only 52.4% run offline evaluations and 37.3% run online evaluations. In Splunk’s 2026 survey, 56% cannot measure LLM response quality and 54% have not tied AI to clear business outcomes.
This is an industry analysis based on public sources. It is written for CIOs and IT leaders, AI platform owners, and agent product and operations teams. Research cut-off: 24 October 2026. The data comes from public reports and documentation from LangChain, Splunk/Cisco, and OpenAI. It is neither first-party research nor a China sample. This article does not treat survey data as an industry benchmark and does not promise rankings, cost reductions, or outcomes.
Shipping is not operating. Observability is not evaluation. Call volume is not business value. Define what a successful task is before discussing models, platforms, and scale.
What changed in enterprise agent operations in 2026?
The shift from pilot to production is real, but operating capability is uneven: 57.3% have agents in production and 89% have observability, while only 37.3% run online evaluations. Seeing is easier than judging.
| Observation | Public data | Sample / period | Note |
|---|---|---|---|
| Agents in production | 57.3% | LangChain, 1,340 respondents, Nov–Dec 2025 | Another 30.4% are in development with a defined launch plan |
| Have observability | 89% | LangChain, same | 94% among teams with production agents; 71.5% have full tracing |
| Offline / online evaluation | 52.4% / 37.3% | LangChain, same | Among teams with production agents, online evaluation is 44.8% |
| Cannot measure LLM quality | 56% | Splunk/Cisco, 2,050 respondents, 10 countries, Feb–Mar 2026 | “AI workload metrics like LLM response quality” |
| No clear business outcomes | 54% | Splunk/Cisco, same | 87% consider monitoring AI a priority |
| Cannot control AI cost | 55% | Splunk/Cisco, same | 60% find unexpected bills more painful than a high baseline cost |
Note: the two surveys differ in sample, geography, industry, and their definitions of “production” and “evaluation”. They cannot be combined into a single adoption rate; the table presents them side by side. LangChain’s sample is 63% technology, 49% companies with fewer than 100 people, and about 3% manufacturing.
Why does observability still not answer whether an agent is good?
Observability tells you what the agent did; evaluation tells you whether it was right. With logs and traces but no success criteria or sampled evaluation, teams can see failures but cannot quantify quality.
LangChain’s data makes the gap clear: 89% have observability, but only 52.4% run offline evaluations and 37.3% run online evaluations. Logs, traces, and metrics are a microscope; evaluation is an exam. The former reconstructs the call chain. The latter answers whether the task was completed, whether the answer was correct, and whether a human should have taken over.
Splunk’s survey adds the business side: 56% cannot measure LLM response quality, and 54% have not tied AI to clear business outcomes. If metrics stop at call volume, token count, and average latency, teams see a busy system but not a better business.
OpenAI’s guide to building agents makes a similar point: design human handover for failure thresholds and high-risk actions, and use evaluation to establish a performance baseline before optimizing cost and latency. In other words, human handover and evaluation are part of agent design, not post-launch patches.
Which operating metrics matter most?
Task success rate, cost per task, human handover rate, adoption, evaluation coverage, P95 latency, and incident or guardrail trigger rate. Start with 3–5 metrics tied to business outcomes.
| Metric | Formula / definition | Why it matters | Data source |
|---|---|---|---|
| Task success rate | Successful tasks ÷ completed tasks; success defined per scenario | Separates “sounds right” from “task actually done” | Business system + agent trace + evaluation |
| Automated closed-loop rate | (Tasks completed end to end with no human intervention and with a successful outcome) ÷ total tasks | Stricter than success rate; reflects the real degree of automation | Agent event stream + tickets |
| Human handover rate | Tasks handed to a human ÷ total tasks; separate approval from failure escalation | Measures fallback cost and user experience | Agent trace + human tickets |
| Cost per task | (Model + tools + infrastructure + human + operations) ÷ successful tasks | Prevents “smarter but less profitable” | Billing + labor hours + allocated operations |
| Adoption | Weekly active target users ÷ total target users | Without users, technical metrics are meaningless | Product telemetry / logins / tickets |
| Evaluation coverage | Evaluated traces or tasks ÷ total traces or tasks; also regression-set coverage | Prevents “we only tested the demo questions” | Evaluation platform + sampling records |
| P95 latency | Upper bound of completion time for 95% of requests | Averages hide long-tail stalls | APM / trace |
| Incident / guardrail trigger rate | Serious incidents or guardrail blocks per 1,000 tasks | Measures safety and availability, not only success | Security logs + human review |
Note: these definitions are editorial advice, not a unified industry standard. A “successful task” needs a reviewable acceptance rule; otherwise the success rate can be manipulated through definitions.
How do you calculate adoption, cost per task, and human handover rate?
Adoption uses weekly active target users. Cost per task divides total cost by successful tasks. Human handover rate divides handed-off tasks by total tasks. All three must be read with the definition of task success.
A hypothetical example, with numbers that are not an industry benchmark: a company runs 10,000 tasks a month; 8,000 succeed (80%) and 1,500 require human handover (15%). Model and tool cost is RMB 20,000, human cost is RMB 12,000 (1,500 × RMB 8), infrastructure is RMB 3,000, and operations is RMB 5,000. Total cost: RMB 40,000.
| Metric | Calculation | Result | How to read it |
|---|---|---|---|
| Task success rate | 8,000 ÷ 10,000 | 80% | Confirm the definition of “success” and the sampling method first |
| Human handover rate | 1,500 ÷ 10,000 | 15% | Separate approval from failure escalation; only the latter is a quality signal |
| Cost per task | 40,000 ÷ 8,000 | RMB 5.00 per successful task | Not RMB 4.00 per total task; failed tasks also consume cost |
| Success rate improvement | 9,000 successful, cost still RMB 40,000 | RMB 4.44 per successful task | Quality improvement directly dilutes cost per task |
| Handover doubles | 3,000 handovers, human cost RMB 24,000, total cost RMB 52,000 | RMB 6.50 per successful task | Human cost is the hidden bulk; tokens alone are not enough |
Note: this is an editorial calculation in RMB, per month, assuming stable prices and labor costs. Real projects should add platform allocation, failed retries, tool calls, and operations staff.
Adoption has two layers: licensed adoption (people with accounts ÷ target users) and task adoption (people who actually complete tasks each week ÷ target users). A login without a completed task is not adoption. For example, with 200 target users, 120 log in weekly (60%), and 60 of those complete a task weekly (30%). Report both figures.
How should targets differ by scenario?
Customer service: resolution, handover, and satisfaction. Knowledge assistants: accuracy, citations, and time saved. Manufacturing: task closure, exception escalation, and audit. Office work: adoption and rework. Set a baseline first; do not copy universal numbers.
| Scenario | Core metrics | Supporting metrics | Do not rely on |
|---|---|---|---|
| Customer service / after-sales | First-contact resolution, failure handover rate, CSAT | Cost per ticket, P95 response, complaint rate | Conversation turns, reply speed |
| Knowledge / office assistant | Answer accuracy, citation coverage, time saved | Weekly active users, adoption, rework rate | Document count, token consumption |
| Manufacturing / operations | Task closed-loop rate, exception escalation rate, audit coverage | Cost per order, P95, incident rate | Number of dashboards, model parameters |
| Engineering / coding assistant | Suggestion acceptance, rework rate, review time | Weekly active users, defect rate, P95 | Lines generated, number of conversations |
Note: scenario metrics are editorial advice, not industry benchmarks. Different task risk, user tolerance, and labor costs change the target. High-risk write-back and approval workflows should not aim for zero human involvement.
How do you run a quarterly review that changes something?
Week 1: get the data right. Week 2: quality and failure taxonomy. Week 3: cost and adoption. Week 4: decide to expand, stop, or fix. Change 1–2 variables at a time; do not swap the model and the process together.
| Week | What to review | Output |
|---|---|---|
| Week 1: data | Task definitions, success criteria, log completeness, cost allocation | Metric dictionary and data-gap list |
| Week 2: quality | Failure taxonomy, evaluation samples, handover reasons, incidents | Top failure modes and fix priorities |
| Week 3: cost and adoption | Cost per task, human cost, active users, frequency of use | Cost structure and adoption bottlenecks |
| Week 4: decisions | Expand, stop, or fix; who owns it and when to recheck | 1–3 actions, not a pile of metrics |
Note: the cadence is editorial advice. A monthly operations meeting plus a quarterly business review is usually enough. Add a dedicated review after a model change, tool change, scenario expansion, or incident.
Which metrics mislead most?
Call volume, token cost, average latency, demo success rate, and agent count all look good. Watch the cost per successful task, human handover, and business results.
| Good-looking but useless | Problem | Replacement |
|---|---|---|
| Tokens / call volume | Shows usage, not task completion | Cost per task, number of successful tasks |
| Average latency | Hides long-tail stalls and timeouts | P95 / P99 latency, timeout rate |
| Demo success rate | Questions are hand-picked; not production | Production sampled evaluation, regression set |
| Number of agents | Count is not closure; it may add coordination cost | Task closed-loop rate, incident rate |
| Human handover = 0 | May hide failures or remove necessary approvals | Handover rate by type + incident rate + audit |
| Logins / accounts | An account is not a completed task | Weekly active task users, task adoption |
| Fixed evaluation set | Goes stale after model or process changes | Rolling regression set + production trace sampling |
Note: the replacements are editorial advice. Each scenario should keep the business metrics it needs; do not delete risk signals for the sake of a single standard.
What should you ask a vendor when buying or building?
Can traces be exported, how is evaluation done, how is human handover triggered, how is cost split, how are models routed, and how is data retained. Without these, operations only sees demos.
| Question | Why ask | Red flag |
|---|---|---|
| Can traces be exported in full? | Without a call chain you cannot locate failures or regressions | Only aggregate reports, no single-task view |
| Do you support offline and online evaluation? | Evaluation closes the quality loop; logs alone do not | Shows an “accuracy” screenshot but no evaluation method |
| How is human handover triggered? | High-risk actions need approval or escalation | Only “fully automatic” or “fully manual” |
| How is cost broken down? | Model, tools, human, and infrastructure must be attributable | Quotes only a token price, not cost per task |
| Can models and tools be replaced? | Avoid lock-in; support routing by task | Changing the model requires rebuilding every flow |
| Data retention and permissions? | Logs contain business data and need audit and isolation | Logs shared by default; cannot delete or export |
| Exit and migration? | Avoid platform lock-in | Cannot export traces, evaluation sets, or configuration |
Note: this question list is procurement advice, not an assessment of any vendor. Business, legal, and security teams should review the specific terms together.
For decision authority, data and supply-chain security, and tiered governance, see our enterprise procurement and compliance checklist for the agent implementation opinion.
Which judgments are still uncertain?
The two surveys use different samples and definitions. LangChain leans toward the technology community; Splunk leans toward global IT operations. They cannot be combined into a China market benchmark, and correlation cannot be read as causation.
- Sample bias: of LangChain’s 1,340 respondents, 63% work in technology, 49% are at companies with fewer than 100 people, and about 3% are in manufacturing. They do not represent Chinese manufacturing or large enterprises.
- Regional differences: Splunk covers 10 countries but not mainland China. AI cost, compliance, and labor cost baselines differ.
- Definition differences: “production”, “evaluation”, and “observability” mean different things in different teams. The 57.3% and 89% figures do not share the same denominator.
- Self-report bias: the surveys are self-reported. Teams may overestimate their evaluation coverage and cost control.
- Correlation is not causation: teams with observability may already be more mature. That does not prove that buying a tool causes success.
What should different companies do next?
Companies already live should finalize their evaluation and cost definitions. Those preparing to launch should define success criteria and handover processes. Multi-agent teams should standardize traces and the metric dictionary. Do not buy a large platform first.
- Already live: pick one high-value scenario, add 200–500 regression evaluation cases, and record success rate, handover rate, cost per task, and P95 for four consecutive weeks.
- Preparing to launch: write task success criteria, failure thresholds, and a high-risk action list before choosing a model or platform. Do not substitute demo success rate for production metrics.
- Multi-agent / multi-team: standardize trace fields, task IDs, cost allocation, and the metric dictionary first, or reports cannot be merged.
- Procurement: put trace export, evaluation, human handover, cost breakdown, and exit terms in the contract. Do not just compare feature lists.
If private execution and end-to-end tracing are the priority, see LingHarness AI foundation. For office collaboration, see the LingWork office agent. For the implementation and acceptance checklist before launch, see AI agents from POC to production.
FAQ
Does the 57.3% production adoption rate represent Chinese companies?
Not directly. Of LangChain’s 1,340 respondents, 63% come from the technology industry and 49% from companies with fewer than 100 people; manufacturing is only about 3%. This is a technology-community sample, not a sample of Chinese companies. It can illustrate trends and operating gaps, but it is not a Chinese market share or industry average.
What is the difference between observability and agent evaluation?
Observability answers “what did the agent do”; evaluation answers “did it do it correctly”. Logs, traces, and metrics reconstruct the call chain, but without task success criteria, sampling, and a regression set, quality cannot be quantified. Use them together, and do not treat “we have logs” as “we have evaluated”.
Should cost per task include human labor?
Yes. Cost per task should include model or API, tools, infrastructure, human review or handover, operations, and allocated platform cost, divided by successful tasks. Counting only tokens or model fees underestimates the real cost, especially where the human handover rate is high.
Is a lower human handover rate always better?
Not necessarily. High-risk approvals, refunds, and production write-back need human confirmation. Distinguish “compliance approval” from “failure handover”. Removing necessary approvals to push the handover rate down raises risk. Read the metric alongside incident rate, audit, and business results.
What targets should we set for an AI agent?
There is no universal number. Define the task success criteria and a baseline first, then set targets by task risk, user tolerance, and human cost. Reasonable targets for customer service, knowledge assistants, manufacturing write-back, and coding assistants are completely different. Do not copy another company’s success rate or cost.
How often should agent operating metrics be reviewed?
Review operating metrics monthly and hold a business review quarterly. Add a dedicated review after a model change, tool change, scenario expansion, or incident. The point of the quarterly review is not whether the numbers look good, but deciding to expand, stop, or fix, with a clear owner and deadline.
AI agent operating metrics are not “how many tokens were used” or “how many agents went live”. They are whether the task was completed, what each success costs, how many need human takeover, and whether the business improved.
Research method and limitations
This article is a study of public sources, not first-party interviews, surveys, or project audits. The LangChain and Splunk surveys are self-reported, and their samples, regions, industries, and definitions differ. This article presents them side by side rather than combining them into a single market conclusion, and it does not present correlation as causation. The metric formulas, worked example, scenario targets, and review cadence are editorial recommendations and should be adjusted to each company. This article does not promise cost reductions, rankings, indexing, or business growth. Version: Q4 2026, first observation; future updates will be made at the same URL with material changes noted on the page.
Sources and caveats
Research cut-off: 24 October 2026. All sources are public pages; the original text governs the specific sample and statistical basis.
- LangChain, “State of Agent Engineering” (12 June 2026). langchain.com. 1,340 respondents, surveyed 18 November to 2 December 2025: 57.3% have agents in production, 89% have observability, 52.4% run offline evaluations, 37.3% run online evaluations; among production teams, 94% have observability and 71.5% have full tracing.
- Splunk/Cisco, “State of Observability 2026: Avoiding the Hidden Costs of AI Agents” (20 October 2026). cisco.com. 2,050 ITOps and engineering respondents across 10 countries, surveyed February to March 2026: 56% cannot measure LLM quality, 54% have no clear business outcomes, and 55% cannot control AI cost.
- OpenAI, “A practical guide to building agents” (15 January 2026). openai.com. Key points: design human handover for failure thresholds and high-risk actions; use evaluation to establish a performance baseline, then optimize cost and latency; combine guardrails with human review to control risk.