The short answer: for a small or mid-sized company in 2026, a serious private AI deployment usually starts at RMB 150,000–800,000 in the first year. Multi-scenario production commonly reaches RMB 800,000–1,600,000. A true group-level 671B deployment typically starts around RMB 5 million over three years. Quotes of RMB 40,000, 400,000, 2 million, and 5 million can all be real — they describe different scopes: “it runs,” “one department uses it,” “multiple systems run on it,” and “enterprise-grade with compliance and redundancy.”
Here, “private” means model inference, data storage, and business integration stay in an environment the company controls. It can be on-premises, in a dedicated cloud, or in a private cloud. It does not automatically mean buying racks of GPUs. Prices below are public, tax-inclusive reference ranges as of 24 October 2026, excluding data-center construction, security certification, and dedicated network lines. Every project should be scoped from a requirements list and a proof of concept.
RMB 150,000 is validation. RMB 800,000 is production. RMB 5 million is enterprise. Most manufacturers land in the first two tiers. The third tier solves a different problem: absolute data separation, high concurrency, and compliance.
Why quotes range from RMB 40,000 to 5 million
Three variables drive the gap:
- Model size. A 7B or 14B model can run quantized on one 24–32 GB GPU. A 70B model needs about 70 GB at FP8 and is usually deployed on two cards in production. A 671B mixture-of-experts model needs roughly 700 GB at FP8 — eight H200-class cards. Hardware alone ranges from tens of thousands to several million RMB.
- Deployment boundary. A public API, a dedicated instance, a private cloud, and an offline deployment have very different requirements for redundancy, security, and operations. Stronger boundaries cost more.
- Integration depth. Installing a model is not the same as connecting it to ERP, MES, and OA, enforcing field-level permissions, building an evaluation set, and running it. The latter is often two to three times the cost of the former.
| Quote tier | Typical delivery | Can do | Not suitable for |
|---|---|---|---|
| RMB 40,000–100,000 | One RTX 5090/4090, 7B–14B quantized model, open-source RAG, single machine | Single-department knowledge base, document summaries, POC | High concurrency, long context, ERP write-back, dual-machine redundancy |
| RMB 300,000–800,000 | 2–4× PRO 6000 (96 GB), 32B–70B, vector database, permissions, monitoring | 50–200 daily users, multi-scenario agents, read-only integration | 671B full model, core transaction writes, financial-grade active-active |
| RMB 1.5–3 million | 8× L20/H20 or 4× H100, 70B–235B, redundancy and compliance as needed | Multiple plants, multiple systems, production SLA | 671B FP8, ten-thousand-level concurrency |
| RMB 5 million+ | 8× H200 141 GB, 671B FP8, storage, network, security, implementation, three-year operations | Group AI foundation, strict compliance, high concurrency | Unvalidated scenarios, no operations team |
The last tier is usually a single node. Active-active redundancy roughly doubles GPU and storage budgets. Do not assume high availability is included.
Where the money actually goes
Hardware is usually 35–45% of a private deployment. Integration, data governance, and three-year operations are where projects overrun — and in manufacturing they often exceed the hardware cost.
| Cost line | Typical share | What it covers | Common omission |
|---|---|---|---|
| Compute and hardware | 35–45% | GPU servers, storage, switches, racks | Counting GPUs but not storage, network, or redundancy |
| Platform software | 10–15% | Inference framework, vector database, RAG, monitoring, model management | Open source is not zero cost; customization and operations need people |
| System integration | 20–30% | ERP/MES/OA/CRM interfaces, permissions, single sign-on | Legacy systems without APIs can double the effort |
| Data governance | 10–20% | Document cleanup, chunking, labeling, evaluation sets | A knowledge base is not “dump the documents in” |
| Implementation and training | 5–10% | Scenario design, prompt and agent tuning, training | No operating owner; the system is unused after launch |
| Three-year operations | 15–25% of hardware | Power, maintenance, model upgrades, security checks | Budgeting only year one |
| Contingency | 10% | Rework, model replacement, scope changes | No buffer; the project stalls mid-way |
What 7B, 70B, and 671B actually need
Calculate memory first, then choose cards. FP16 weights are roughly 2 GB per billion parameters; INT8 halves that, INT4 halves it again. Leave another 20–50% for KV cache and framework overhead. The table shows weight minimums, not production configuration.
| Model | FP16 weights | INT8 | INT4 | Typical hardware | Typical use |
|---|---|---|---|---|---|
| 7B–8B | ~14 GB | ~7 GB | ~4 GB | 1× RTX 5090 32 GB / 1× PRO 6000 | Department Q&A, classification, summaries |
| 14B | ~28 GB | ~14 GB | ~7 GB | 1× 32 GB / 1× 48 GB | Knowledge base, document processing |
| 32B | ~64 GB | ~32 GB | ~16 GB | 1× PRO 6000 96 GB / 2× 48 GB | Multi-scenario agents, code and reports |
| 70B | ~140 GB | ~70 GB | ~35 GB | 2× H200 141 GB / 4× 48 GB | Complex reasoning, production assistant |
| 671B MoE | ~1.3 TB | ~670 GB | ~340 GB | 8× H200 141 GB at FP8 (~700 GB) | Group-level foundation, full model |
“It fits” does not mean “it survives production.” Production adds redundancy, storage and backup, monitoring, model version management, and security audit. Those typically add 30–60% to the hardware line. Multi-turn agents also multiply token consumption by 3–5× compared with single-turn Q&A, changing memory, latency, and cost.
The three budget tiers
| Tier | First-year budget | Three-year TCO | Model and hardware | Users and scenarios |
|---|---|---|---|---|
| Validation | RMB 150,000–400,000 | RMB 200,000–550,000 | 7B–14B quantized; 1× RTX 5090 or PRO 6000 | 10–50 users, 1–2 scenarios: knowledge base, summaries, POC |
| Production | RMB 800,000–1,600,000 | RMB 1.2–2.2 million | 32B–70B; 2–4× PRO 6000 or 8× L20 | 50–200 users, multiple scenarios, read-only ERP/MES integration |
| Enterprise | RMB 5–8 million | RMB 6–10 million | 671B MoE at FP8; 8× H200 141 GB | Group, multiple plants, high concurrency, strict compliance |
The dividing line is not “how expensive are the cards.” It is how many people and how many problems the system must serve. The validation tier serves one department. The production tier connects multiple systems and permissions. The enterprise tier carries group-level stability, compliance, and peak load. There is a transition band between RMB 400,000 and 800,000: a 32B single machine with one read-only ERP interface, or a quantized 70B for a small team.
If 20 people just need help writing reports and searching policies, the validation tier may be too expensive. Use an API first and validate the scenario. If drawings, process parameters, and financial data cannot leave the intranet, and ERP/MES integration is required, production is the sensible starting point.
Is private cheaper than API?
On pure token cost, most small and mid-sized companies never reach break-even. Private deployment buys data boundaries, low latency, system integration, and long-term control — not cheaper tokens.
Assume an effective API cost of RMB 20–60 per million tokens, already accounting for multi-turn agents, long context, and retries. A 70B production deployment with a three-year TCO of RMB 1.2–2.2 million costs RMB 400,000–730,000 per year. At RMB 20–60 per million tokens, that requires roughly 670 million to 3.7 billion tokens per year to break even on tokens alone. A team of 100 daily users at 20,000 tokens each, 250 working days a year, consumes about 500 million tokens. Most manufacturers are below that level.
The right question is not “can private beat API pricing?” It is “which data and which scenarios justify paying for a boundary?” If the answer is unclear, do not buy GPUs yet.
Three costs manufacturers miss
1. System integration
ERP holds orders, planning, procurement, inventory, and cost. MES holds scheduling, dispatch, reporting, quality, and equipment. OA holds approvals and collaboration. To answer “why is this work order late,” the model needs data from all three. Interface work usually consumes 20–30% of the budget. If the legacy system has no API, middleware or screen-level automation adds another step. Non-invasive integration can save up to 40% of the connection cost, but only after master data, permissions, and processes are clean.
2. Data governance
A knowledge base is not a file share. Ten years of policies, process documents, and inspection reports dumped into a vector database will produce fast answers that may be wrong. Documents need deduplication, chunking, tagging, and permissions. You need 200–500 real questions and answers to measure correctness, not fluency. This is usually 10–20% of the budget and is the difference between a demo and a usable system.
3. Model refresh and operations
Models change every 6–12 months; inference frameworks, quantization, and context windows change with them. A card bought today may not run next year’s full model. You need one or two people for GPUs, Kubernetes, model versions, monitoring, security, and incidents. Budget 15–25% of hardware per year. Choose a model-agnostic inference layer, leave 30% memory headroom, and assume a three-year depreciation cycle.
A practical sequence for 2026
- Validate with an API (2–4 weeks). Pick 3–5 real tasks and 200–500 evaluation examples. Establish a baseline. Cost: RMB 0–30,000.
- Go hybrid (4–8 weeks). Keep sensitive data on a local 7B–32B model with RAG; send non-sensitive tasks to a cloud model; unify routing and audit. Cost: RMB 150,000–500,000.
- Expand private capacity (8–12 weeks). Move to 70B or larger, integrate ERP/MES, add field-level permissions and redundancy. Cost: RMB 800,000–3 million.
- Operate quarterly. Re-evaluate models, update the knowledge base, and review cost. If usage stays low for two quarters, scale down.
1. Do not choose a private deployment without 200+ real evaluation examples. 2. Do not choose a fully self-hosted system without 1–2 part-time administrators. 3. Do not sign a hardware purchase without a three-year TCO and an exit plan.
FAQ
What is the minimum for a private AI deployment?
RMB 100,000–150,000 can build a single-machine POC. Supporting more than 50 production users usually requires RMB 300,000–800,000 in the first year. A 671B enterprise deployment starts around RMB 5 million.
How many GPUs does a 70B model need?
FP8 weights are about 70 GB. One H200 141 GB can hold the weights; production usually recommends two cards for KV cache, concurrency, and redundancy. INT4 can reduce weights to about 35 GB, but concurrency and quality may suffer.
Is private always cheaper than API?
No. On pure token cost, most small and mid-sized companies are better off with APIs. Private deployment buys data separation, low latency, integration, and control.
Can we do this without an IT team?
You can choose a managed private cloud or an appliance, but at least one part-time administrator is needed. A fully self-hosted deployment usually needs one or two operations or platform engineers.
Which scenario should a manufacturer start with?
Start with high-frequency, low-risk scenarios where data already exists: knowledge-base Q&A, report and work-order assistance, and quality-traceability lookups. Do not start with end-to-end autonomous workflows.
Will the hardware become obsolete?
Use a model-agnostic inference layer, leave 30% memory headroom, and depreciate over three years. Avoid binding to one model or vendor; when the model changes, replace weights, not the architecture.
A reasonable 2026 budget for private AI at a mid-sized company is RMB 150,000–1,600,000: RMB 150,000–400,000 for single-department validation, RMB 800,000–1,600,000 for multi-scenario production. Above RMB 5 million is usually a 671B, high-compliance, high-concurrency group deployment. Validate with APIs, then go hybrid, then privatize what must stay inside.
Sources and caveats
Research cut-off: 24 October 2026. GPU and API prices move quickly; verify current official quotes before signing. The budget tiers, break-even volumes, and acceptance lists are editorial estimates, not vendor quotes. They exclude data-center construction, dedicated lines, and security certification. Custom scope determines the actual price.
- DeepSeek API documentation, “Models & Pricing,” accessed 24 October 2026. api-docs.deepseek.com.
- Alibaba Cloud Bailian, “Model pricing,” accessed 24 October 2026. help.aliyun.com.
- Volcano Engine, “Doubao models,” accessed 24 October 2026. volcengine.com.
- IDC, “Beyond Tokens: China’s Private AI Enters Deep Water,” 18 May 2026. idc.com.
- SkySeraph, “Open-source LLM local deployment hardware guide,” 2026. cnblogs.com.
- Public GPU rental quotes, August 2026: 8× H100 around RMB 90,000/month; 8× H200 around RMB 150,000/month. Verify with the provider.