Direct answer: using official on-demand list prices retrieved on 7 November 2026 (Linux, AWS us-east-1 / Azure eastus, excluding storage, egress, tax and discounts), an 8-GPU H100 node costs USD 55.04/hour on AWS p5.48xlarge (USD 6.88 per GPU-hour, about USD 40,179/month at 730 hours) versus USD 98.32/hour on Azure ND96isr H100 v5 (USD 12.29 per GPU-hour, about USD 71,774/month). Azure's node-hour list price is about 78.6% higher. In the single-GPU inference tier, AWS g6.xlarge (1x L4 24GB) is USD 0.8048/hour (about USD 588/month) and g6e.xlarge (1x L40S 48GB) is USD 1.861/hour (about USD 1,359/month); Azure NV36ads A10 v5 (1x A10 24GB) is USD 3.20/hour (about USD 2,336/month). Note that Azure NV12ads A10 v5 provides only one-third of an A10: USD 0.908/hour looks cheap, but the full-GPU-equivalent rate is USD 2.724/hour. Conclusion: normalise to cost per GPU-hour, actual utilisation, interconnect and memory before comparing; never compare node sticker prices or fractional GPUs with full GPUs directly.
This guide is for architecture, ML, operations and procurement teams budgeting AI training/inference, selecting a cloud provider and negotiating purchases. Prices come from the official AWS on-demand pricing JSON and the Azure Retail Prices API (both retrieved 7 November 2026); specifications come from AWS product pages and Microsoft Learn instance-series documentation. Official price facts, editorial calculations and excluded items are kept separate.
(1) For an 8-GPU H100 node, cost per GPU-hour matters more than the node price: AWS USD 6.88 vs Azure USD 12.29 at list. (2) In the single-GPU tier, check memory and GPU fraction first: one-third of an A10 is not comparable to a full card. (3) Below 60% utilisation, idle time eats the apparent price advantage; calculate effective cost per GPU-hour before discussing reserved or savings plans.
Scope: what these prices are and are not
This comparison uses only on-demand, Linux, public list prices: AWS us-east-1 (N. Virginia) and Azure eastus, as of 7 November 2026. Prices exclude cloud disks/local SSD, object storage, public egress, NAT/load balancers, snapshots and backup, Windows licensing, enterprise support and taxes; they also exclude reserved instances, savings plans, Spot/low-priority pricing and any private discount. 730 hours is the average month (365 x 24 / 12); actual billing is per second or minute, so month length causes small differences. List prices at both providers can change at any time; re-query before purchasing.
H100 8-GPU nodes: AWS vs Azure
Both offer eight H100 80GB GPUs, but the node-hour prices differ substantially: AWS p5.48xlarge is USD 55.04/hour and Azure ND96isr H100 v5 is USD 98.32/hour, or USD 6.88 and USD 12.29 per GPU-hour respectively. At 730 hours, the monthly costs are about USD 40,179 and USD 71,774, a difference of about USD 31,594 per month. The specifications are not identical: AWS provides 192 vCPU, 2,048 GiB of memory and 3,200 Gbps EFA; Azure provides 96 vCPU, 1,900 GiB and 400 Gbps InfiniBand per GPU. The gap may reflect networking, generation, region and commercial discounts, so it is not simply "one provider is more expensive"; but at list price Azure's cost per GPU-hour is higher.
| Instance | GPU | vCPU / memory | Interconnect | On-demand (USD/h) | Per GPU-hour | 730h monthly |
|---|---|---|---|---|---|---|
| AWS p5.48xlarge | 8x H100 80GB | 192 / 2,048 GiB | 3,200 Gbps EFA | 55.04 | 6.88 | about 40,179 |
| Azure ND96isr H100 v5 | 8x H100 80GB | 96 / 1,900 GiB | 8x 400 Gbps InfiniBand | 98.32 | 12.29 | about 71,774 |
Sources: AWS EC2 on-demand pricing JSON (us-east-1, Linux, on-demand, 7 November 2026); Azure Retail Prices API (eastus, Linux, Consumption, 7 November 2026); specifications from the AWS P5 product page and Microsoft Learn ND-H100-v5 documentation. Monthly = hourly x 730; excludes storage, egress, tax and discounts. Azure's list price is about 78.6% higher per node-hour.
Single-GPU / small-scale inference: L4, L40S and A10
The most common mistake in the single-GPU tier is the "GPU fraction". AWS g6.xlarge has one full L4 (24GB) and g6e.xlarge has one full L40S (48GB); Azure NV36ads A10 v5 has one full A10 (24GB), while NV12ads A10 v5 provides only one-third of an A10 with 8GB of framebuffer. So NV12ads at USD 0.908/hour cannot be compared with a full-card price: its full-GPU-equivalent rate is USD 2.724/hour, higher than AWS's full L4. If the model fits in 8GB and concurrency is very low, a one-third card can still make sense; if you need 24GB or stable throughput, compare full cards.
| Instance | GPU | GPU fraction / memory | vCPU / memory | On-demand (USD/h) | Per full GPU-hour | 730h monthly |
|---|---|---|---|---|---|---|
| AWS g6.xlarge | 1x L4 | 1 / 24GB | 4 / 16 GiB | 0.8048 | 0.8048 | about 588 |
| AWS g6e.xlarge | 1x L40S | 1 / 48GB | 4 / 32 GiB | 1.8610 | 1.8610 | about 1,359 |
| Azure NV12ads A10 v5 | 1/3x A10 | 1/3 / 8GB | 12 / 110 GiB | 0.9080 | 2.7240 | about 663 |
| Azure NV36ads A10 v5 | 1x A10 | 1 / 24GB | 36 / 440 GiB | 3.2000 | 3.2000 | about 2,336 |
Sources: AWS EC2 on-demand pricing JSON (us-east-1, Linux), Azure Retail Prices API (eastus, Linux), AWS G6/G6e product pages and Microsoft Learn NVadsA10_v5 documentation (accessed 7 November 2026). L4, L40S and A10 are not the same performance class; the table normalises price and specification only and makes no performance-equivalence claim. "Per full GPU-hour" = hourly price / GPU fraction.
Utilisation: the hidden multiplier
On-demand pricing buys occupied time, not completed work. If a node runs only 30% of the month, the effective cost per GPU-hour is 3.3x the list price; at 60% utilisation it is 1.67x. For training clusters, queueing, debugging, data loading and failure recovery all reduce effective utilisation; for inference, peaks, troughs and cold starts create idle time too. Measure your stable utilisation first, then decide between reserved/savings plans and an on-demand plus Spot mix, rather than looking only at the hourly rate.
| Option | 100% utilisation | 60% utilisation | 30% utilisation |
|---|---|---|---|
| AWS p5.48xlarge (per GPU-hour) | 6.88 | 11.47 | 22.93 |
| Azure ND96isr H100 v5 (per GPU-hour) | 12.29 | 20.48 | 40.97 |
| AWS g6.xlarge (per GPU-hour) | 0.8048 | 1.34 | 2.68 |
| Azure NV12ads A10 v5 (per full GPU-hour) | 2.724 | 4.54 | 9.08 |
Editorial calculation: effective cost per GPU-hour = list price / utilisation; excludes storage, egress, tax and discounts. Utilisation should be measured as GPU time actually executing work divided by billed time, not CPU or whole-host uptime.
Beyond the list price: what else appears on the bill
A cloud GPU bill is rarely just instance hours. Training and inference also generate cloud disks, images, object storage, logs, public egress, cross-region replication, load balancers, NAT, backup and monitoring; multi-node training adds cluster networking and orchestration overhead. Classifying these items by whether they scale linearly with GPU count prevents a budget that covers only the host and then overruns on storage and traffic. The table is an editorial checklist, not a vendor quote.
| Cost item | Trigger | Estimation method |
|---|---|---|
| Cloud/local disk | Datasets, weights, checkpoints, images | Capacity x monthly rate; checkpoints x retained copies x size |
| Object storage | Data lake, model registry, backup | Storage volume + requests + retrieval traffic |
| Public egress | Inference APIs, model downloads, cross-region transfer | GB x egress rate; estimate peaks and monthly volume |
| Network and gateways | Load balancers, NAT, dedicated/VPN links | Instance hours + processed traffic; multi-AZ doubles |
| Snapshots/backup | Compliance, rollback, disaster recovery | Snapshot capacity + cross-region copies |
| Support plan | Production SLA, 24x7 response | Percentage of cloud spend or fixed tier |
| Idle and queueing | Debugging, data loading, recovery | Convert by effective utilisation, as above |
Editorial synthesis; specific rates and billing rules follow the AWS/Azure official pricing pages and invoices. Taxes, exchange rates, migration labour and self-managed operations are not included.
Choosing by scenario: training, inference, graphics and POC
Start with the bottleneck: memory, interconnect or per-GPU compute. Large-model training and multi-node HPC need the high-bandwidth interconnect (EFA/InfiniBand) and local storage of an 8-GPU node; inference and fine-tuning care more about memory capacity, batching efficiency and cost per token, where L40S 48GB and L4 24GB often fit better than an 8-GPU H100; graphics and virtual desktops need GRID licensing and framebuffer, which Azure NV-series includes; POC or low-frequency jobs should start with a single GPU or serverless GPU instead of buying a full month for peak capacity. Without benchmarks, do not infer "which GPU is faster" from price alone.
| Scenario | Suggested tier | Key constraint | Validate first |
|---|---|---|---|
| Large-model pre-training / multi-node HPC | 8x H100/H200 node | GPU interconnect, topology, EFA/IB, checkpoint storage | Multi-node scaling efficiency, recovery time |
| Inference / fine-tuning | L40S 48GB, L4 24GB, A10 24GB | Memory, batching, quantisation, concurrency | Cost per token/request, P95 latency |
| Graphics / virtual desktops | A10 series with GRID | Framebuffer, concurrent users, licensing | Users per host, encoding latency |
| Small POC | Single GPU on-demand or serverless GPU | Cold start, memory, data compliance | Whether monthly/reserved is justified |
| Steady production inference | Reserved/savings plan plus on-demand buffer | Commitment term, region, portability | Effective cost per GPU-hour after discount |
Editorial selection framework, not a vendor recommendation; GPU performance, memory bandwidth and interconnect must be tested with the actual model and framework.
Procurement and decision checklist
Asking the following seven questions usually prevents "looks cheap, bills higher". The priorities are to align billing units, confirm GPU fraction and interconnect, include storage/egress/support in the total, and clarify the commitment term and exit conditions behind any discount.
| Question | Why ask | Expected answer |
|---|---|---|
| Which region, OS and billing mode? | Region, Windows and on-demand/reserved differ widely | Region, Linux/Windows, on-demand/reserved/spot |
| Full GPU or fractional? How much memory? | A one-third card is not comparable to a full card | GPU model, count/fraction, memory in GB |
| What GPU interconnect and bandwidth? | Training scaling depends on interconnect | NVLink/PCIe, EFA/IB, bandwidth per GPU |
| Are storage, egress, snapshots and support included? | List prices usually exclude them | Itemised rates or an explicit "billed separately" |
| Reserved/savings discount and term? | Discounts may bind region, family and term | Discount rate, term, refund/exchange conditions |
| Is capacity/quota guaranteed? | Availability can delay training plans | Availability zones, quota, reserved-capacity terms |
| Tax, currency and payment terms? | Cross-border purchasing changes landed cost | Tax inclusive/exclusive, currency, payment term |
Editorial procurement question list, not commercial or legal advice; contract terms and written vendor quotes govern.
FAQ
Why is Azure's H100 node so much more expensive than AWS?
At public list prices retrieved on 7 November 2026, Azure ND96isr H100 v5 is USD 98.32/hour versus USD 55.04/hour for AWS p5.48xlarge, about 78.6% higher per node-hour. Specifications, networking, region and commercial discounts differ, so list prices are not the same as negotiated transaction prices; compare cost per GPU-hour at the same region, term and support level.
An 8-GPU node is expensive; why not buy eight single-GPU hosts?
Large-model training needs high-bandwidth GPU-to-GPU interconnect (NVLink/EFA/InfiniBand) and consistent topology; eight single-GPU hosts usually have only ordinary networking, so scaling efficiency drops sharply. Single GPUs suit inference, fine-tuning, graphics and POC; eight single-GPU hosts are not equivalent to one 8-GPU node.
Can one-third of an A10 run inference?
Yes, if the model and concurrency fit in 8GB of memory. NV12ads A10 v5 at USD 0.908/hour is a one-third card, equivalent to USD 2.724 per full GPU-hour; if the model needs 24GB or stable throughput, compare full-card tiers such as NV36ads.
Why use 730 hours for the monthly estimate?
730 is the average month (365 x 24 / 12), useful for turning an hourly rate into a monthly budget. Actual billing is per second or minute, and month length varies; if the system runs only on weekdays, use actual powered-on hours instead of 730.
How much can reserved instances or savings plans save?
This article excludes reserved instances, savings plans, Spot and private discounts because they vary by region, instance family, term and commitment. Ask the provider for a like-for-like quote and compare the effective cost per GPU-hour after discount with the on-demand rate and your utilisation.
Can we use Spot/low-priority instances?
They suit interruptible batch processing, rendering and offline training with checkpoints, and are usually much cheaper than on-demand; they are not suitable for online inference with a strict SLA. Design checkpointing, retries and capacity fallback before relying on them.
Do these prices include storage and traffic?
No. The tables cover instance hours only. Cloud/local disks, object storage, public egress, NAT/load balancers, snapshots, support and taxes may be billed separately. Include the items above in the total cost of ownership before launch.
Why is Google Cloud not included?
At retrieval time, Google Cloud's public pricing pages were calculator/dynamic-table based, and a verifiable static list price could not be obtained without signing in or using the key-required Cloud Billing Catalog API. To avoid inventing numbers, this article compares AWS and Azure only; use Google Cloud's official calculator or a commercial quote for GCP pricing.
Research method and limitations
This article separates "official price facts" from "editorial calculations": AWS hourly prices come from the official EC2 on-demand pricing JSON (manifest timestamp 2026-11-07T00:28:52Z, us-east-1, Linux, on-demand); Azure hourly prices come from the Azure Retail Prices API (eastus, Linux, Consumption, 7 November 2026); GPU model, count, memory, vCPU, RAM and interconnect come from the AWS P5/G6/G6e product pages and Microsoft Learn ND-H100-v5 and NVadsA10_v5 documentation. Monthly costs, per-GPU-hour figures, utilisation conversions and cost items are editorial calculations.
Seven limitations: (1) only AWS us-east-1 and Azure eastus on-demand Linux list prices are covered, excluding other regions, Windows, reserved/savings plans, Spot, tax and private discounts; (2) storage, egress, networking, backup and support are not included; (3) L4, L40S and A10 are not the same performance class, and this article normalises price/specification only, not performance; (4) 730 hours is an average-month convention while actual billing is per second/minute; (5) prices change and should be re-queried after publication; (6) Google Cloud and other providers are not covered, for the reason in the FAQ; (7) this is not procurement, commercial or legal advice. Performance, memory bandwidth and interconnect efficiency must be tested on your own models and frameworks.
Sources and basis
- AWS, EC2 On-Demand pricing JSON (us-east-1, Linux, on-demand): b0.p.awsstatic.com/.../US East (N. Virginia)/Linux/index.json. p5.48xlarge 55.04, g6.xlarge 0.8048, g6e.xlarge 1.8610 USD/hour; manifest 2026-11-07. Accessed 7 November 2026.
- AWS, Amazon EC2 P5 Instances: aws.amazon.com/ec2/instance-types/p5/. p5.48xlarge: 192 vCPU, 2 TiB memory, 8x H100 80GB, 3,200 Gbps EFA. Accessed 7 November 2026.
- AWS, Amazon EC2 G6 / G6e Instances: aws.amazon.com/ec2/instance-types/g6/, aws.amazon.com/ec2/instance-types/g6e/. g6.xlarge: 1x L4 24GB, 4 vCPU, 16 GiB; g6e.xlarge: 1x L40S 48GB, 4 vCPU, 32 GiB. Accessed 7 November 2026.
- Microsoft, Azure Retail Prices API (eastus, Virtual Machines, Consumption): prices.azure.com/api/retail/prices. ND96isr H100 v5 98.32, NV12ads A10 v5 0.908, NV36ads A10 v5 3.20 USD/hour. Accessed 7 November 2026.
- Microsoft Learn, ND-H100-v5 sizes series: learn.microsoft.com/.../ndh100v5-series. 8x H100 80GB, 96 vCPU, 1,900 GiB, 400 Gbps InfiniBand per GPU. Accessed 7 November 2026.
- Microsoft Learn, NVadsA10_v5 sizes series: learn.microsoft.com/.../nvadsa10v5-series. NV12ads is 1/3x A10 8GB, NV36ads is 1x A10 24GB; the series includes GRID licensing. Accessed 7 November 2026.
Data cut-off 7 November 2026; prices and specifications follow vendor official pages/APIs; not procurement, commercial or legal advice.