pauledwards

Distributing model weights to your AI cluster: a faster pre-flight on AKS and Slurm

Azure / By pauledwards / Published 05/06/2026 / Updated 05/06/2026 / 2 minutes of reading

Why this exists You’ve provisioned a multi-node GPU cluster on Azure. NDR InfiniBand. The training script is ready. Before any GPU does any useful work, every node needs the same 400 GB model checkpoint on its local NVMe. The naive approach — azcopy (or our Rust equivalent, azcp) on every node, in parallel — has three problems: You pay Azure […]

Distributing model weights to your AI cluster: a faster pre-flight on AKS and Slurm Continue Reading

mpi-stage: High-Performance File Distribution for HPC Clusters

Azure / By pauledwards / Published 01/09/2026 / Updated 01/09/2026 / 5 minutes of reading

When running containerized workloads on HPC clusters, one of the first problems you hit is getting container images onto the nodes quickly and repeatably. A .sqsh is a Squashfs image (commonly used by container runtimes on HPC). In some environments you can run a Squashfs image directly from shared storage, but at scale that often turns the shared filesystem

mpi-stage: High-Performance File Distribution for HPC Clusters Continue Reading