When Compute Sits and Waits: Fixing the Hidden Storage Bottleneck in EDA and HPC with Azure NetApp Files

Hello Folks!

If you have ever stared at an HPC pipeline and wondered why the queue depth keeps climbing while every CPU graph looks lazy, this session from the Microsoft Azure Infrastructure Summit 2026 is going to feel very familiar. Ron Hogue from the Azure Storage team and Ranga Sankar from NetApp spend twenty-seven minutes diagnosing the problem that nobody wants to admit out loud, that the bottleneck is not compute, not the scheduler, and not licenses. It is storage. And then they walk through exactly how Azure NetApp Files (ANF) solves it.

📺 Watch the session:

Why IT Pros Should Care

If you support a semiconductor, simulation, rendering, or scientific computing team, you have lived this story. Engineers ask for more cores. You buy them. Wall-clock times barely move. Managers then ask for more tool licenses, because surely that must be the constraint. Spoiler, it usually is not.

Here is why this matters to anyone running shared infrastructure on Azure:

EDA and HPC pipelines hammer storage with millions of tiny reads and writes, metadata operations like file creation, rename, and unlink, all happening concurrently across thousands of processes.
That pattern breaks generic cloud file storage that was tuned for large sequential I/O.
ANF brings the same NetApp ONTAP data services that EDA teams have used on-premises for decades, delivered as a native Azure service.
You can move design environments to Azure without re-architecting tools, schedulers, or scratch path conventions.
The economics changed in the last twelve months with the Flexible service level and Cool Access, so the old “ANF is too expensive for scratch” argument deserves a fresh look.

In short, if your job involves keeping expensive engineers and expensive licenses busy, the storage layer deserves your attention.

What Azure NetApp Files Brings to EDA and HPC, the technical overview

ANF is a first-party Azure service running NetApp ONTAP on bare-metal infrastructure inside Azure datacenters. It speaks NFS v3, NFS v4.1, and SMB, with dual-protocol options, and it preserves the file semantics that EDA tools assume. Things like POSIX permissions, fast metadata operations, snapshots, and consistent low latency.

Ron and Ranga framed the session in eight layers (Problem, Foundation, Accelerator, Scale, AI Ready, Guardrails, Optimize, Proof). Three pieces did most of the heavy lifting.

Foundation, Migration Assistant with SnapMirror. This is how you get the data into Azure without rewriting your inventory. SnapMirror replicates from on-premises ONTAP or Cloud Volumes ONTAP into ANF while preserving metadata, permissions, snapshots, and directory structure, with continuous synchronization and minimal downtime. For EDA flows where a missing ACL can invalidate an entire run, that fidelity is not optional.

Accelerator, Cache Volumes. Built on NetApp FlexCache, these volumes front your authoritative dataset (whether it lives in on-prem ONTAP or in Cloud Volumes ONTAP) and pull hot reads close to your Azure compute. Tool libraries, PDKs, shared reference data, all served at sub-millisecond latency without copying petabytes around. There is still one source of truth, which keeps your data governance story clean.

Scale, Large Volumes with Breakthrough mode. A single ANF large volume in Breakthrough mode scales to 2 PiB and delivers throughput in the tens of GiB per second by fanning I/O across six storage endpoints. That lets you collapse sharded namespaces (the classic /proj1, /proj2, /proj3 split that nobody loves) into fewer high-throughput volumes that behave predictably under contention.

How it works, under the hood

Cache Volumes are the FlexCache pattern most ONTAP customers already know. You peer a cluster, point a cache at an origin, and the cache propagates data on demand. Ron demoed this live: cluster peering with on-prem ONTAP, a cache volume created in ANF, and reads served from cache while the authoritative copy stayed home.

Large Volumes in Breakthrough mode are where the architecture gets interesting. Instead of a single mount point pinned to a single storage endpoint, Breakthrough mode exposes six storage endpoints for one logical volume. Clients can mount, balance I/O, and aggregate throughput across all six. Microsoft published Linux scale-out benchmarks showing a single 50 TiB large volume in Breakthrough mode sustaining roughly 50,000 MiB/s of sequential reads and approaching 1.8 million 8 KiB random read IOPS using twelve VMs (see the Microsoft Learn link in Resources).

For shared environments, ANF added user and group quotas with real-time consumption reporting and hard limits. Ranga demoed quota rules that stopped a runaway simulation generating millions of scratch files from starving the rest of the team. If you have ever had to send the “who filled the scratch volume” email at 2am, this feature alone might justify the trip.

On the cost side, the Flexible service level decouples capacity from throughput. You buy a capacity pool, you pick throughput independently, with 128 MiB/s of baseline throughput included and a ceiling of up to 640 MiB/s per provisioned TiB (which is roughly five times the Ultra service level). Cool Access transparently tiers cold blocks to Azure Blob behind the same file mount point.

In short, you stop paying for throughput you do not need on archival volumes, and you stop overprovisioning capacity to chase throughput on small scratch volumes.

Real-world value, use cases, ROI, scenarios

The Proof section closed with SPEC Storage 2020 EDA Blended results that are worth reading carefully:

A single large volume in Breakthrough mode sustained 2,880 EDA jobsets at about 0.51 ms overall response time.
Six volumes scaled linearly to 17,280 jobsets at about 0.60 ms.
FIO measurements approached 2 million IOPS.

The honest tradeoff: those numbers come from a benchmark, not your environment, and the SPEC tables show latency climbing at the highest load points. So treat the result as evidence that the platform behaves predictably under EDA-style concurrency, not as a guarantee for every workflow. That predictability is the part that matters. EDA leads do not lose sleep over peak throughput, they lose sleep over latency that drifts when concurrency rises.

Practical scenarios where this lands:

Burst regression and verification runs to Azure during tape-out crunches, with Cache Volumes keeping your on-prem golden tree authoritative.
Full migration of EDA environments to Azure for teams whose datacenters are out of capacity or out of lease.
HPC simulation workloads (CFD, weather, seismic, life sciences) that share the same metadata-heavy I/O profile.
AI-adjacent pipelines that need to read design data with file semantics and surface the same bytes as objects to Fabric, OneLake, or Databricks via the dual file and object access pattern Ron mentioned in the AI Ready section.

Getting Started, concrete first steps

You do not need a multi-quarter project to get a useful pilot moving.

Register the Azure NetApp Files resource provider in your target subscription and request quota in the regions you care about.
Stand up a small capacity pool, start with the Flexible service level so you can dial throughput independently.
Create a test volume, mount it from a representative VM (HBv4 or Ev5 family are good starting points for HPC and EDA respectively).
Run your real workload, not just FIO. Use a representative regression batch or simulation job and watch the metadata patterns in the ANF metrics blade.
If you have on-premises ONTAP, peer a Cache Volume against it to test FlexCache behavior with your actual datasets before committing to a full migration.
Layer in user and group quotas before you open the volume to a wider team. Trust me on this one.

Resources

Keep Learning at the Summit

Catch the full Microsoft Azure Infra Summit 2026 session playlist here.

Cheers!

Pierre Roman

This article was originally published by Microsoft's ITOps Talk Blog. You can find the original article here.

Tags: Artificial Intelligence, Cluster, Linux, Server Message Block [SMB], Storage, Virtual Machine [VM]