67 percent of all AI workloads are now run outside the cloud. 88 percent of companies run at least one AI workload on-premises. It’s not about leaving the cloud entirely—it’s about hybrid AI, where control, security, and intellectual property are best protected on-premises.
Target Audience: CTO, Infrastructure Manager, AI Lead | Reading Time: 6 minutes
TL;DR: 67 percent of all AI workloads are now running outside the cloud. 88 percent of companies run at least one AI workload on-prem. It’s not about leaving the cloud entirely—it’s about hybrid AI, where control, security, and IP are best protected on-premises. GPU clusters based on the NVIDIA DGX platform, such as NVIDIA DGX systems, are becoming strategic investments, not experiments.
What’s happening with AI and the cloud right now?
Three years ago, the answer was obvious: AI runs in the cloud. AWS, Azure, and GCP offer on-demand GPU instances, and startups built their entire businesses around API calls to OpenAI and Anthropic.
But in 2026, the tide has turned. Dell Technologies World in May presented figures that silenced the audience: 67 percent of all AI workloads are now running outside the public cloud. Four out of five companies—88 percent—run at least one AI workload on-premises, in a colocation facility, or in an edge environment.
This is not a minor shift. It is a fundamental reevaluation of how AI infrastructure should be built.
Why are companies moving AI from the cloud to on-premises?
There are five driving forces behind this shift:
1. Costs Are Skyrocketing in the Cloud
GPU hours in the cloud are expensive. An NVIDIA H100 instance on AWS costs about $4–5 per hour. For a training job that requires 1,000 GPU hours per day, the annual cost can easily reach $1.5–2 million—just for compute.
With an on-premises NVIDIA DGX H100 system, the break-even point is often reached in 18–24 months for constant workloads. Over 5–7 years, the difference amounts to several million dollars.
2. Data Residency and Regulatory Requirements
Nordic companies in the finance, healthcare, and public sectors have strict requirements regarding where data may be stored and processed. The GDPR, the Patient Data Act, and defense-related confidentiality regulations mean that sensitive data cannot leave the country—or even the organization’s own data centers.
The cloud can solve this with region-specific instances, but you never have the same level of control as when you own the infrastructure yourself.
3. Intellectual Property Protection and Competitive Advantage
When training an AI model on proprietary data, the model itself is a competitive advantage. Allowing it to be trained in a shared cloud environment—where the cloud provider could theoretically observe or replicate patterns—is a business risk that more and more companies are refusing to take.
4. Predictable performance
The cloud is shared. Other users’ workloads affect yours. Latency varies. Bandwidth is not guaranteed.
“On-prem” means dedicated GPUs, dedicated network bandwidth, and full control over the entire stack. For real-time inference—especially agent-based AI that interacts with internal systems—this is crucial.
5. Scalability Without Lock-in
Cloud environments lock you into a vendor’s ecosystem. On-premises solutions based on open standards allow you to switch vendors, migrate workloads, and maintain control over your technology roadmap.
Dell, IDC, and Gartner: What Do the Numbers Say About 2026?
| Metric | Number | Source |
|---|---|---|
| NVIDIA DGX system (8×B200 GPUs) | 4.8–5.3 MSEK | 0.5–0.8 MSEK |
| Network (400/800 GbE, leaf+spine) | 1.5–3.0 MSEK | Included |
| Storage (500 TB, NVMe tier) | 3.0–6.0 MSEK | 0.4–0.8 MSEK |
| Electricity and cooling (0.5 MW) | — | 2.0–4.0 MSEK |
| Operations and Support (24/7) | — | 1.0–2.0 MSEK |
| Total 5–10 racks | 45–90 MSEK | 7.9–13.6 MSEK |
The numbers speak for themselves: AI infrastructure is the fastest-growing category in IT. And it’s growing the fastest on-premises.
Gartner’s latest forecast shows that AI infrastructure now accounts for 31.7 percent of the total IT budget—up from 13.7 percent as recently as 2024. That’s a doubling in just one year. And that’s just the beginning.
IDC reports that non-x86 servers, powered by AI chips with Arm cores, now account for 47.9 percent of server market revenue, representing a 107 percent increase from the previous year. The market is transforming in real time.
Which workloads are best suited for on-premises vs. the cloud?
On-prem is superior for:
- ›Model trainingusing proprietary data (fine-tuning of LLMs and industry-specific models)
- ›Agent-basedAI with system access (internal APIs, databases, documents)
- ›Workloadswith strict data residency requirements (finance, healthcare, defense)
- ›Real-time inferencewith low-latency requirements (industry, autonomy, sensor fusion)
- ›Constant, predictable workloads (the cloud is expensive when the GPUs are running around the clock)
The cloud has the following advantages:
- ›Experimentationand prototyping (test a model for two weeks without tying up capital)
- ›Burst-basedworkloads (seasonal peaks, campaigns)
- ›Accessto hyperscale models (GTP-5 class, requires data centers with 10,000+ GPUs)
- ›Organizationswithout in-house infrastructure expertise
The reality for most Nordic companies: a hybrid strategy in which training and fine-grained inference take place on-premises, while experiments and peak loads are handled in the cloud.
The Cost Picture: The Price Tag for a Modern AI Data Center
Building an AI data center is no longer just for hyperscalers. With the DGX platform—such as NVIDIA DGX systems—a company can get started with a rack-scale AI factory.
Price estimates for the Nordic region (all amounts exclude VAT):
* Operating costs for electricity and cooling vary significantly depending on the utilization rate, electricity pricing agreements, and the data center’s PUE (Power Usage Effectiveness). The prices above are indicative estimates, excluding VAT.
For a complete system consisting of 5–10 racks of NVIDIA DGX systems—which provides capacity equivalent to the AI computing clusters of medium-sized cloud environments—the total cost amounts to 45–90 MSEK in capital expenditure and 8–14 MSEK per year in operating costs.
Compared to the cloud: Equivalent GPU capacity on AWS would cost 20–35 MSEK per year. The break-even point for on-premises solutions is often reached in 18–30 months. The break-even point for on-premises solutions is often reached in 18–30 months.
“It’s not about choosing between the cloud and on-premises. It’s about placing the right workload in the right place at the right price.”
5 Steps for a Successful Migration from Cloud to Hybrid AI
Step 1: Take inventory of your workloads
Map out all AI workloads: what is being trained, what is running in inference, what data is being used, and what are the latency requirements? This provides the basis for deciding what should be moved on-premises.
Step 2: Choose the Right Data Center Partner
For most companies, colocation or managed AI data centers are a better option than building their own. Look for a partner that offers:
- ›Experiencewith GPU-intensive environments (NVIDIA DGX system certification is a plus)
- ›Climate-neutraloperations (important for ESG reporting and costs)
- ›Networking expertise(InfiniBand/RoCE)
- ›SwedishData Residency
Step 3: Design for a hybrid
The cloud isn’t going away. Design the architecture so that models can be trained on-premises, deployed to the cloud for burst processing, and then brought back. Kubernetes with GPU support (Run:ai, NVIDIA KAI Scheduler) is the key.
Step 4: Ensure data flows
AI models rely on data. Ensure that training data can be moved efficiently between storage and compute. WEKA, VAST, or Pure Storage are common choices for AI-grade storage.
Step 5: Build internal expertise
On-premises solutions require different skills than cloud solutions. Ensure that your team has the capacity to handle GPU scheduling, network optimization, and model deployment. Alternatively, purchase managed services from a partner.
Aixia’s Perspective: Hybrid AI Made Simple
Aixia builds and operates NVIDIA DGX systems in the Nordic region with full support for hybrid AI. Our climate-neutral data centers and the DGX platform make on-premises AI accessible to Swedish and Nordic companies.
- › NVIDIA DGX System : Validated design from NVIDIA with up to 5 petaFLOPS AI performance per rack
- › Climate-neutral operation : ISO 14001-certified data centers with sustainable energy
- › Swedish data residency : Data stays in Sweden, without cloud gaze
- › Managed AI : We handle the operations — you focus on the models
“Would you like an unbiased assessment? Contact Aixia’s AI infrastructure team for a workshop where we’ll analyze your workloads and recommend the right hybrid strategy.”
Summary: Three Things to Do This Month
- 1 Take inventory of your AI workloads. Which ones are running in the cloud today? What data is being used? Which ones are costing the most?
- 2 Calculate break-even. Compare the cost of the cloud equivalent to the on-prem investment over 3-5 years. Don’t forget to factor in data residency, IP protection, and performance.
- 3.Schedule an NVIDIA DGX system demo. See what a validated AI factory can do for your performance—before you sign your next cloud contract.
Sources and Further Reading
- › Dell Technologies World: AI Infrastructure Shifts to On-Premises — Next Platform, May 2026 (67% outside the cloud, 88% at least one on-prem)
- › Data and Storage at the Center of the AI Stack — Next Platform, May 2026
- › IDC Worldwide Quarterly Server Tracker Q1 2026 — IDC, June 2026 (non-x86 servers +107%)
- › Gartner AI Spending Forecast 2026 — Gartner, June 2026 (31.7% of IT budget)
- › TeraWulf AI Datacenter Build-out — Next Platform, May 2026 ($7-10M per MW)
Aixia AB — The Nordic region’s exclusive NVIDIA DGX system partner. We simplify complex AI infrastructure for Swedish and Nordic companies.
Contact: petter.ahlen@aixia.se | aixia.se
Ready to explore hybrid AI?
Let Aixia’s AI infrastructure team assess your workloads and recommend the right strategy.
Aixia AB — The Nordic Region’s Exclusive NVIDIA DGX System Partner
