Toolset update: Fadsv7, x64/ARM64 phase 2, 48/64-core VMs

[Stephan T. Lavavej (StephanTLavavej)]

Overview

This adds an Fadsv7 pool, which is the fastest available x64 SKU: non-SMT Zen 5, with NVMe temp storage.

And now that we have more and larger pools, this restructures the dependencies between stages. I'm replacing a complicated dependency graph with simpler phases, where each phase needs to succeed before the next phase begins.

We start with a Phase 1 of initial builds, which verify that the PR conforms to our formatting and validation requirements, that the STL builds for all architectures (with /analyze), and that the tests successfully configure. We now rely on ARM64 and ARM64EC native builds, and we don't bother with cross builds.

Then we have a Phase 2 of primary test architectures, where x64 and ARM64 run simultaneously. Finally, we have a Phase 3 of secondary test architectures: x86 and ARM64EC. (Previously, we gated x64 to run first, followed by x86, ARM64, and ARM64EC simultaneously.)

By spreading out the test architectures like this, we reduce each PR's maximum pressure on the pools (particularly the ARM64 pool). For the first time, I'm also giving the initial stages a dedicated pool, so even if the PR/CI system is under heavy load, initial checks won't be held up behind larger test runs.

For PRs with failing tests in x64, this will waste an ARM64 test run compared to the previous structure, but we still save the secondary test runs (which is better than other PR systems which I shall not discuss here). On the plus side, ARM64-only failures will now avoid wasting x86/ARM64EC test runs, and will deliver feedback faster.

I'm also changing our VMs to be larger: 48-core when we're limited by quota, 64-core when we're less constrained. The SKUs with temp storage (Fadsv7 and Dpdsv6) especially benefit from higher core counts. We'll unlock further gains when we improve long-running tests, which waste a lot of time at the end of each test pass with low parallelism.

I also had to change create-1es-hosted-pool.ps1 to prepare prototype VMs with only 16 cores, before imaging them to create the pools with larger VMs. I'm adding a comment explaining why we're doing this and why it works.

Pool structure

• Medium pool (Fasv7)
  • Used for the Phase 1 initial builds only.
  • 13 VMs, each 48-core. We use 4 VMs per PR, so this gives us throughput for 3 PRs. (Phase 1 is very fast, so this is unlikely to be a bottleneck.)
• Fast pool (Fadsv7)
  • Used for the Phase 2 x64 tests only.
  • 21 VMs, each 48-core. We use 10 VMs per PR, so this gives us throughput for 2 PRs. (As the name implies, this one is fast, and each PR uses it for only one phase, so it shouldn't be a bottleneck.)
• Slow pool (Fasv6)
  • Used for the Phase 3 x86 tests (plus ASan CI).
  • 32 VMs, each 64-core. We use 10 VMs per PR, so this gives us throughput for 3 PRs.
  • Note: 64-core Fasv6 really is substantially slower than 48-core Fadsv7.
• ARM64 pool (Dpdsv6)
  • 32 VMs, each 64-core. We use at most 10 VMs simultaneously (2 in Phase 1, 10 in Phase 2, 10 in Phase 3), so this gives us throughput for 3 PRs.

If and when I can obtain more quota, we may simplify this pool structure in the future.

Commits

• Cycle locations.
• Got quota for 2048 cores of Fadsv7 in australiaeast.
• Change initial stages to use a Medium pool.
• Split work root usage into Slow/Medium/Fast.
• Split pools and work roots into Slow/Medium/Fast.
• Move Test_x86 down, no other changes.
• Move Build_ARM64EC_Cross down, no other changes.
• Add Phase 1/2/3 comments.
• Drop Build_ARM64_Cross.
• Rename Build_ARM64_Native => Build_ARM64, drop outdated comments.
• Change Build_ARM64EC_Cross to be native, now named Build_ARM64EC.
• Change dependencies to actually implement the phase structure.
• Adjust VM/pool sizes.
• Split $VMSize into $ProtoVMSize (16-core) and $PoolSkuName (48-core or 64-core).
• Final pools.
• Add a comment about 16-core embiggening to 48/64-core, and make Notes open-ended.

[Copilot]

Pull request overview

Updates CI infrastructure with larger VM pools and a phased architecture-testing workflow.

Changes:

• Adds the Fadsv7 pool and enlarges existing pool VMs.
• Splits CI into initial-build, primary-test, and secondary-test phases.
• Uses native ARM64/ARM64EC builds and pool-specific work roots.

Show a summary per file

File	Description
azure-pipelines.yml	Implements phased CI dependencies and pool assignments.
azure-devops/create-1es-hosted-pool.ps1	Adds Fadsv7 and separates prototype and pool VM sizes.
azure-devops/config.yml	Defines new pools and work roots.
azure-devops/asan-pipeline.yml	Uses the renamed slow-pool work root.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

• Files reviewed: 4/4 changed files
• Comments generated: 0
• Review effort level: Medium

[Stephan T. Lavavej (StephanTLavavej)]

I'm mirroring this to the MSVC-internal repo. Please notify me if any further changes are pushed, otherwise no action is required.