Choosing between Azure ML's persistent compute and SageMaker's on-demand approach isn't really about cloud preference, it's about how you want to think about work itself. Azure treats compute as a fixture of your workspace, always there, always configured, waiting for you to return. SageMaker treats compute as a consumable, spinning up exactly what a job needs and tearing it down when the job ends. Neither is wrong, but each pulls your team toward a different rhythm of development.
If your team iterates in long, exploratory sessions, tweaking hyperparameters, inspecting intermediate results, collaborating on shared notebooks, Azure's persistent clusters make sense. You log in, your environment is ready, and the context you built yesterday is still there. That continuity reduces friction for researchers who think in hours, not minutes. SageMaker's job-specific model, by contrast, forces intentionality. Every training run is a discrete unit: you define the instance type, the container, the script, and the data source upfront. That discipline can prevent the "it worked on my machine" problem, but it also means every new experiment requires spinning up infrastructure from scratch. For teams running many parallel, well-defined experiments, that overhead is trivial. For teams still figuring out what to try, it can feel like starting the car every time you want to adjust the radio.
The environment customization story reinforces the same divide. Azure offers curated environments and custom environments, giving you a spectrum from "just work" to "I built this Dockerfile myself." SageMaker offers three levels of customization, from fully managed frameworks to bring-your-own-container. Both platforms let you get what you need, but the underlying philosophy differs. Azure's approach assumes you want to build a consistent environment once and reuse it across projects. SageMaker's approach assumes each job may need a unique environment, and that's fine. If your organization standardizes on a single ML stack, Azure's persistence reduces maintenance. If your teams experiment with different frameworks or versions depending on the model, SageMaker's per-job flexibility avoids version collisions.
Here is the practical takeaway: choose the model that matches how your team actually works, not the one that sounds more modern. Persistent compute rewards continuity and collaboration. On-demand compute rewards isolation and reproducibility. Neither is a sign of being stuck in the past or chasing hype. The best infrastructure is the one that gets out of your way so you can focus on the model, not the machine.