diff options
| author | Ethan Morgan <ethan@gweithio.com> | 2026-02-14 16:44:06 +0000 |
|---|---|---|
| committer | Ethan Morgan <ethan@gweithio.com> | 2026-02-14 16:44:06 +0000 |
| commit | 54409423f767d8b1cf30cb7d0efca6b4ca138823 (patch) | |
| tree | d915ac7828703ce4b963efdd9728a1777ba18c1e /vcpkg/scripts/azure-pipelines/osx | |
Diffstat (limited to 'vcpkg/scripts/azure-pipelines/osx')
| -rw-r--r-- | vcpkg/scripts/azure-pipelines/osx/README.md | 300 | ||||
| -rw-r--r-- | vcpkg/scripts/azure-pipelines/osx/azure-pipelines.yml | 109 | ||||
| -rw-r--r-- | vcpkg/scripts/azure-pipelines/osx/register-guest.sh | 23 | ||||
| -rw-r--r-- | vcpkg/scripts/azure-pipelines/osx/setup-box.sh | 26 | ||||
| -rw-r--r-- | vcpkg/scripts/azure-pipelines/osx/setup-guest.sh | 21 | ||||
| -rw-r--r-- | vcpkg/scripts/azure-pipelines/osx/start-parallels-vm.zsh | 47 | ||||
| -rw-r--r-- | vcpkg/scripts/azure-pipelines/osx/stop-delete-all-vms.zsh | 17 |
7 files changed, 543 insertions, 0 deletions
diff --git a/vcpkg/scripts/azure-pipelines/osx/README.md b/vcpkg/scripts/azure-pipelines/osx/README.md new file mode 100644 index 0000000..7164f23 --- /dev/null +++ b/vcpkg/scripts/azure-pipelines/osx/README.md @@ -0,0 +1,300 @@ +# `vcpkg-eg-mac` VMs + +This is the checklist for what the vcpkg team does when updating the macOS machines in the pool. + +## Creating new base images + +### Prerequisites + +- [ ] A Parallels license for amd64 or [macosvm](https://github.com/s-u/macosvm) allow-listed + by macOS for arm64. Note that the directory 'Parallels' is still used when using `macosvm` + just so that scripts know where to find the VM and friends. +- [ ] An Xcode .xip - you can get this from Apple's developer website, + although you'll need to sign in first: <https://developer.apple.com/downloads> + If you are doing this from a local macos box, you can skip to the "update the macos host" step. +- [ ] An Xcode Command Line Tools installer + +### Instructions (AMD64) + +- [ ] Go to https://dev.azure.com/vcpkg/public/_settings/agentqueues , pick the current osx queue, + and delete one of the agents that are idle. +- [ ] Go to that machine in the KVM. (Passwords are stored as secrets in the CPP_GITHUB\vcpkg\vcpkgmm-passwords key vault) +- [ ] Open the Parallels Control Center, and delete the active VM. +- [ ] Update the macos host +- [ ] Update or install parallels +- [ ] Download the macOS installer from the app store. See https://support.apple.com/en-us/102662 + Note: This portion of the instructions is that which breaks most often depending on what Parallels and macOS are doing. + You might need to use `softwareupdate --fetch-full-installer --full-installer-version 15.6.1` and point Parallels + at that resulting installer in 'Applications' instead. +- [ ] Run parallels, and select that installer you just downloaded. Name the VM "vcpkg-osx-<DATE>", for example "vcpkg-osx-2025-09-11". +- [ ] When creating the VM, customize the hardware to the following: + * 12 processors + * 24000 MB of memory + * 350 GB disk + * Disable sound output + * Disable microphone + * Do not share mac camera +- [ ] Install MacOS like you would on real hardware. + * Set up as new + * Apple ID: 'Set Up Later' / Skip + * Account name: vcpkg + * A very similar password :) + * Don't enable Location Services + * Share crashes with app developers + * Don't enable Ask Siri +- [ ] Install Parallels Tools +- [ ] Restart the VM +- [ ] Set the desktop wallpaper to a fixed color from Settings -> Wallpaper . (This makes the KVM a lot easier to use :) ) +- [ ] Disable automatic updates in the VM: Settings -> General -> Automatic Updates -> Disable them all +- [ ] Enable remote login in System Settings -> General -> Sharing -> Remote Login +- [ ] Update the Azure Agent URI in setup-box.sh to the current version. You can find this by going to the agent pool, selecting "New agent", picking macOS, and copying the link. For example https://download.agent.dev.azure.com/agent/4.261.0/vsts-agent-osx-x64-4.261.0.tar.gz +- [ ] In the guest, set the vcpkg user to be able to use sudo without a password. From a dev machine: + ```sh + scp path/to/Xcode.xip vcpkg@HOSTMACHINE:/Users/vcpkg/Xcode.xip + ssh vcpkg@HOSTMACHINE + rm ~/.ssh/known_hosts + export GUEST_IP=`prlctl list --full | sed -nr 's/^.*running *([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}).*/\1/p'` + export SSH_COOKIE=vcpkg@$GUEST_IP + scp Xcode.xip $SSH_COOKIE:/Users/vcpkg/Xcode.xip + ssh $SSH_COOKIE # and then enter the password here + printf 'vcpkg\tALL=(ALL)\tNOPASSWD:\tALL\n' | sudo tee -a '/etc/sudoers.d/vcpkg' + sudo chmod 0440 '/etc/sudoers.d/vcpkg' + sudo mdutil -ad + xip --expand Xcode.xip + sudo mv Xcode.app /Applications/Xcode.app + rm Xcode.xip + exit + ``` +- [ ] Open Xcode from Applications in the guest GUI. Accept the EULA. +- [ ] Update the Azure Agent URI in setup-box.sh to the current version. You can find this by going to the agent pool, selecting "New agent", picking macOS, and copying the link. For example https://download.agent.dev.azure.com/agent/4.261.0/vsts-agent-osx-x64-4.261.0.tar.gz +- [ ] Copy setup-box.sh and the xcode installer renamed to 'clt.dmg' to the host. For example from a dev workstation: + ```sh + scp ./setup-guest.sh vcpkg@HOSTMACHINE:/Users/vcpkg + scp ./setup-box.sh vcpkg@HOSTMACHINE:/Users/vcpkg + scp path/to/console/tools.dmg vcpkg@HOSTMACHINE:/Users/vcpkg/clt.dmg + ssh vcpkg@HOSTMACHINE + chmod +x setup-guest.sh + ./setup-guest.sh + rm setup-guest.sh + rm setup-box.sh + rm clt.dmg + exit + ``` +- [ ] Shut down the VM cleanly. +- [ ] Ensure that there is no host system specific 'hardware' like macOS install .isos attached. +- [ ] Set the VM 'Isolated' +- [ ] Boot and then cleanly shut down the VM. +- [ ] In Parallels control center, right click the VM and select "Prepare for Transfer" +- [ ] In Parallels control center, right click the VM and remove it, but "Keep Files" +- [ ] Copy the packaged VM to Azure Storage, with something like: + ```sh + ssh vcpkg@HOSTMACHINE + brew install azcopy + azcopy copy ~/Parallels/vcpkg-osx-2025-09-11.pvmp "https://vcpkgimageminting.blob.core.windows.net/pvms?<SAS>" + azcopy copy ~/Parallels/vcpkg-osx-2025-09-11.sha256.txt "https://vcpkgimageminting.blob.core.windows.net/pvms?<SAS>" + exit + ``` +- [ ] Go to https://dev.azure.com/vcpkg/public/_settings/agentqueues and create a new self hosted Agent pool named `PrOsx-YYYY-MM-DD` based on the current date. Grant microsoft.vcpkg.ci and microsoft.vcpkg.pr access. +- [ ] Remove the macOS installer from Applications +- [ ] Follow the "Deploying images" steps below for each machine in the fleet. + +### Instructions (ARM64) + +- [ ] Go to https://dev.azure.com/vcpkg/public/_settings/agentqueues , pick the current osx queue, + and delete one of the agents that are idle. +- [ ] Go to that machine in the KVM. (Passwords are stored as secrets in the CPP_GITHUB\vcpkg\vcpkgmm-passwords key vault) +- [ ] Update the macos host +- [ ] (Once only) install `macosvm` to `~` (this tarball is also backed up in our `vcpkg-image-minting` storage account). For example from a dev workstation: + ```sh + ssh vcpkg@HOSTMACHINE + curl -L -o macosvm-0.2-1-arm64-darwin21.tar.gz https://github.com/s-u/macosvm/releases/download/0.2-1/macosvm-0.2-1-arm64-darwin21.tar.gz + tar xvf macosvm-0.2-1-arm64-darwin21.tar.gz + rm macosvm-0.2-1-arm64-darwin21.tar.gz + exit + ``` +- [ ] Download the matching `.ipsw` for the macOS copy to install. See https://mrmacintosh.com/apple-silicon-m1-full-macos-restore-ipsw-firmware-files-database/ ; links there to find the .ipsw. Example: https://updates.cdn-apple.com/2025SummerFCS/fullrestores/093-10809/CFD6DD38-DAF0-40DA-854F-31AAD1294C6F/UniversalMac_15.6.1_24G90_Restore.ipsw +- [ ] Determine the VM name using the form "vcpkg-osx-<date>-arm64", for example "vcpkg-osx-2025-09-11-arm64". +- [ ] Open a terminal and run the following commands to create the VM with vcpkg-osx-2025-09-11-arm64 and UniversalMac_15.6.1_24G90_Restore.ipsw replaced as appropriate. This must be run in the KVM as it uses a GUI: + ```sh + mkdir -p ~/Parallels/vcpkg-osx-2025-09-11-arm64 + cd ~/Parallels/vcpkg-osx-2025-09-11-arm64 + ~/macosvm --disk disk.img,size=500g --aux aux.img -c 8 -r 12g --restore ~/UniversalMac_15.6.1_24G90_Restore.ipsw ./vm.json + ~/macosvm -g ./vm.json + ``` +- [ ] Follow prompts as you would on real hardware. + * Set up as new. + * Account name: vcpkg + * A very similar password + * Do not allow computer account password to be reset with your Apple Account. + * Apple ID: 'Set Up Later' / Skip + * No location services + * Yes send crash reports + * Set up screen time later + * Only download updates automatically +- [ ] Set the desktop wallpaper to a fixed color from Settings -> Wallpaper . (This makes the KVM a lot easier to use :) ) +- [ ] Disable automatic updates in the VM: Settings -> General -> Automatic Updates -> Disable them all +- [ ] Enable remote login in the VM: Settings -> General -> Sharing -> Remote Login +- [ ] Set the vcpkg user to be able to use sudo without a password, and install Xcode. For example from a dev workstation: + ```sh + scp path/to/Xcode.xip vcpkg@HOSTMACHINE:/Users/vcpkg/Xcode.xip + ssh vcpkg@HOSTMACHINE + rm ~/.ssh/known_hosts + scp Xcode.xip vcpkg@vcpkgs-Virtual-Machine.local:/Users/vcpkg/Xcode.xip + ssh vcpkg@vcpkgs-Virtual-Machine.local + printf 'vcpkg\tALL=(ALL)\tNOPASSWD:\tALL\n' | sudo tee -a '/etc/sudoers.d/vcpkg' + sudo chmod 0440 '/etc/sudoers.d/vcpkg' + sudo mdutil -ad + xip --expand Xcode.xip + sudo mv Xcode.app /Applications/Xcode.app + rm Xcode.xip + exit + ``` +- [ ] Open Xcode from Applications in the guest GUI. Uncheck the "code completion model" and accept the EULA. +- [ ] Update the Azure Agent URI in setup-box.sh to the current version. You can find this by going to the agent pool, selecting "New agent", picking macOS, and copying the link. For example https://download.agent.dev.azure.com/agent/4.261.0/vsts-agent-osx-arm64-4.261.0.tar.gz +- [ ] Copy setup-box.sh and the xcode installer renamed to 'clt.dmg' to the host. For example from a dev workstation: + ```sh + scp ./setup-guest.sh vcpkg@HOSTMACHINE:/Users/vcpkg + scp ./setup-box.sh vcpkg@HOSTMACHINE:/Users/vcpkg + scp path/to/console/tools.dmg vcpkg@HOSTMACHINE:/Users/vcpkg/clt.dmg + ssh vcpkg@HOSTMACHINE + chmod +x setup-guest.sh + ./setup-guest.sh + rm setup-guest.sh + rm setup-box.sh + rm clt.dmg + exit + ``` +- [ ] Shut down the VM cleanly. +- [ ] Mint a SAS token to vcpkgimageminting/pvms with read, add, create, write, and list permissions. +- [ ] Package the VM into a tarball. For example from a dev workstation: + ```sh + ssh vcpkg@HOSTMACHINE + cd ~/Parallels + aa archive -d vcpkg-osx-<date>-arm64 -o vcpkg-osx-<date>-arm64.aar -enable-holes + brew install azcopy + azcopy copy vcpkg-osx-<date>-arm64.aar "https://vcpkgimageminting.blob.core.windows.net/pvms?<SAS>" + exit + ``` +- [ ] Go to https://dev.azure.com/vcpkg/public/_settings/agentqueues and create a new self hosted Agent pool named `PrOsx-YYYY-MM-DD-arm64` based on the current date. Grant microsoft.vcpkg.ci and microsoft.vcpkg.pr access. +- [ ] Follow the "Deploying images" steps below for each machine in the fleet. + +## Deploying images + +### Running the VM (AMD64) + +Run these steps on each machine to add to the fleet. Skip steps that were done implicitly above if this machine was used to build a box. + +- [ ] If this machine was used before, delete it from the pool of which it is a member from https://dev.azure.com/vcpkg/public/_settings/agentqueues +- [ ] Log in to the machine using the KVM. +- [ ] Check for software updates in macOS system settings +- [ ] Check for software updates in Parallels' UI +- [ ] Mint a SAS token URI to the box to use from the Azure portal if you don't already have one, and download the VM. (Recommend running this via SSH from domain joined machine due to containing SAS tokens). From a developer machine: + ```sh + ssh vcpkg@HOSTMACHINE + brew install azcopy + cd ~/Parallels + azcopy copy "https://vcpkgimageminting.blob.core.windows.net/pvms/vcpkg-osx-<DATE>.pvmp?<SAS>" . + azcopy copy "https://vcpkgimageminting.blob.core.windows.net/pvms/vcpkg-osx-<DATE>.sha256.txt?<SAS>" . + exit + ``` +- [ ] Open the .pvmp in Parallels, and unpack it. +- [ ] Open the 'Configure' screen for the VM, and under Options -> Startup and Shutdown, set the following: + * Custom + * Start Automatically: When Mac Starts + * Startup View: Headless + * On VM Shutdown: Close Window (this one should already be set) + * On Mac Shutdown: Shut Down + * On Window Close: Keep Running in the Background +- [ ] Under 'More Options': + * Time: Do not sync + * Uncheck 'Update Parallels Tools automatically' +- [ ] Close and restart Parallels Desktop. The VM should start automatically. If it does not, start it. +- [ ] [grab a PAT][] if you don't already have one +- [ ] Copy the guest deploy script to the host, and run it with a first parameter of your PAT. From a developer machine: + ```sh + scp register-guest.sh vcpkg@HOSTMACHINE:/Users/vcpkg/register-guest.sh + ssh vcpkg@HOSTMACHINE + rm .ssh/known_hosts + chmod +x register-guest.sh + ./register-guest.sh PAT GOES HERE + rm register-guest.sh + ``` +- [ ] In the VM, open a terminal on the host and run: + ``` + ssh -i ~/Parallels/*/id_guest vcpkg@`prlctl list --full | sed -nr 's/^.*running *([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}).*/\1/p'` + ~/myagent/run.sh + ``` +- [ ] Check that the machine shows up in the pool. +- [ ] Lock the screen on the host. +- [ ] Update the "vcpkg Macs" spreadsheet line for the machine with the new pool. + +[grab a PAT]: #getting-an-azure-pipelines-pat + +### Running the VM (ARM64) + +Run these steps on each machine to add to the fleet. Skip steps that were done implicitly above if this machine was used to build a box. + +- [ ] If this machine was used before, delete it from the pool of which it is a member from https://dev.azure.com/vcpkg/public/_settings/agentqueues +- [ ] Log in to the machine using the KVM. +- [ ] Check for software updates in macOS system settings +- [ ] (Once only) install `macosvm` to `~` (this tarball is also backed up in our `vcpkg-image-minting` storage account). From a developer machine: + ```sh + ssh vcpkg@HOSTMACHINE + curl -L -o macosvm-0.2-1-arm64-darwin21.tar.gz https://github.com/s-u/macosvm/releases/download/0.2-1/macosvm-0.2-1-arm64-darwin21.tar.gz + tar xvf macosvm-0.2-1-arm64-darwin21.tar.gz + rm macosvm-0.2-1-arm64-darwin21.tar.gz + exit + ``` +- [ ] Skip if this is the image building machine. Mint a SAS token URI to the box to use from the Azure portal if you don't already have one, and download the VM. (Recommend running this via SSH from domain joined machine due to containing SAS tokens). From a developer machine: + ```sh + ssh vcpkg@HOSTMACHINE + brew install azcopy + mkdir -p ~/Parallels + cd ~/Parallels + azcopy copy "https://vcpkgimageminting.blob.core.windows.net/pvms/vcpkg-osx-<DATE>-arm64.aar?<SAS>" vcpkg-osx-<DATE>-arm64.aar + aa extract -d vcpkg-osx-<DATE>-arm64 -i ./vcpkg-osx-<DATE>-arm64.aar -enable-holes + exit + ``` +- [ ] Open a separate terminal window on the host and start the VM by running: + ```sh + cd ~/Parallels/vcpkg-osx-<DATE>-arm64 + ~/macosvm ./vm.json + ``` +- [ ] [grab a PAT][] if you don't already have one +- [ ] Copy the guest deploy script to the host, and run it with a first parameter of your PAT. From a developer machine: + ```sh + scp register-guest.sh vcpkg@HOSTMACHINE:/Users/vcpkg/register-guest.sh + ssh vcpkg@HOSTMACHINE + rm .ssh/known_hosts + chmod +x register-guest.sh + ./register-guest.sh PAT GOES HERE + rm register-guest.sh + ``` +- [ ] That will cleanly shut down the VM. In the KVM's terminal, relaunch the VM in ephemeral mode with: + ```sh + ~/macosvm --ephemeral ./vm.json + ``` +- [ ] Open a terminal window on the host and run the agent + ```sh + ssh -i ~/Parallels/*/id_guest vcpkg@vcpkgs-Virtual-Machine.local + ~/myagent/run.sh + ``` +- [ ] Check that the machine shows up in the pool, and lock the vcpkg user on the host. +- [ ] Lock the screen on the host. +- [ ] Update the "vcpkg Macs" spreadsheet line for the machine with the new pool. + +[grab a PAT]: #getting-an-azure-pipelines-pat + +## Getting an Azure Pipelines PAT + +Personal Access Tokens are an important part of this process, +and they are fairly easy to generate. +On ADO, under the correct project (in vcpkg's case, "vcpkg"), +click on the "User Settings" icon, then go to "Personal access tokens". +It is the icon to the left of your user icon, in the top right corner. + +Then, create a new token, give it a name, make sure it expires quickly, +and give it a custom defined scope that includes the +"Agent pools: Read & manage" permission (you'll need to "Show all scopes" +to access this). +You can now copy this token and use it to allow machines to join. diff --git a/vcpkg/scripts/azure-pipelines/osx/azure-pipelines.yml b/vcpkg/scripts/azure-pipelines/osx/azure-pipelines.yml new file mode 100644 index 0000000..db31a00 --- /dev/null +++ b/vcpkg/scripts/azure-pipelines/osx/azure-pipelines.yml @@ -0,0 +1,109 @@ +# Copyright (c) Microsoft Corporation. +# SPDX-License-Identifier: MIT +# + +parameters: + - name: vcpkgToolSha + displayName: 'Custom SHA of vcpkg-tool to use rather than bootstrap' + type: string + default: 'use default' + - name: jobName + type: string + default: 'x64_osx' + - name: poolName + type: string + - name: tripletPattern + displayName: 'Enable the triplets which contain this substring' + type: string + default: '' + +jobs: +- job: ${{ parameters.jobName }} + condition: and(succeeded(), contains('^${{ replace(parameters.jobName, '_', '-') }}$', '${{ parameters.tripletPattern }}')) + pool: + name: ${{ parameters.poolName }} + workspace: + clean: resources + timeoutInMinutes: 2880 # 2 days + variables: + - name: WORKING_ROOT + value: /Users/vcpkg/Data + - name: VCPKG_DOWNLOADS + value: /Users/vcpkg/Data/downloads + steps: + - bash: | + sudo mdutil -ad || 0 + sudo mkdir ${{ variables.VCPKG_DOWNLOADS }} || 0 + sudo chmod 777 ${{ variables.VCPKG_DOWNLOADS }} || 0 + exit 0 + displayName: 'Create ${{ variables.VCPKG_DOWNLOADS }}' + - bash: ./bootstrap-vcpkg.sh + displayName: 'Bootstrap vcpkg' + condition: eq('use default', '${{ parameters.vcpkgToolSha }}') + - bash: ./scripts/azure-pipelines/bootstrap-from-source.sh ${{ parameters.vcpkgToolSha }} + displayName: "Build vcpkg with CMake" + condition: ne('use default', '${{ parameters.vcpkgToolSha }}') + - task: AzureCLI@2 + displayName: '*** Test Modified Ports' + inputs: + azureSubscription: 'vcpkg-pr-fleet-wus' + scriptType: 'pscore' + scriptLocation: 'inlineScript' + inlineScript: | + $current = Get-Date -AsUtc + $endDate = $current.AddDays(2) + $end = Get-Date -Date $endDate -UFormat '+%Y-%m-%dT%H:%MZ' + $assetSas = az storage container generate-sas --name cache --account-name vcpkgassetcachewus --as-user --auth-mode login --https-only --permissions rcl --expiry $end -o tsv | Out-String + $assetSas = $assetSas.Trim() + $binarySas = az storage container generate-sas --name cache --account-name vcpkgbinarycachewus --as-user --auth-mode login --https-only --permissions rclw --expiry $end -o tsv | Out-String + $binarySas = $binarySas.Trim() + # Persist the binary SAS as a secret pipeline variable for the owners-db step + Write-Host "##vso[task.setvariable variable=BCACHE_SAS_TOKEN;issecret=true]$binarySas" + $env:X_VCPKG_ASSET_SOURCES = "x-azurl,https://vcpkgassetcachewus.blob.core.windows.net/cache,$assetSas,readwrite" + & scripts/azure-pipelines/test-modified-ports.ps1 -Triplet ${{ replace(parameters.jobName, '_', '-') }} -BuildReason $(Build.Reason) -BinarySourceStub "x-azcopy-sas,https://vcpkgbinarycachewus.blob.core.windows.net/cache,$binarySas" -WorkingRoot $env:WORKING_ROOT -ArtifactStagingDirectory $(Build.ArtifactStagingDirectory) + - task: PublishPipelineArtifact@1 + displayName: "Publish Artifact: failure logs for ${{ replace(parameters.jobName, '_', '-') }}" + inputs: + targetPath: '$(Build.ArtifactStagingDirectory)/failure-logs' + artifact: "failure logs for ${{ replace(parameters.jobName, '_', '-') }}" + condition: ne(variables['FAILURE_LOGS_EMPTY'], 'True') + - task: PublishPipelineArtifact@1 + displayName: "Publish Artifact: azcopy logs for ${{ replace(parameters.jobName, '_', '-') }}" + inputs: + targetPath: '$(WORKING_ROOT)/azcopy-logs' + artifactName: "z azcopy logs for ${{ replace(parameters.jobName, '_', '-') }}" + condition: ne(variables['AZCOPY_LOGS_EMPTY'], 'True') + - task: UseNode@1 + displayName: 'Ensure Node.js is available' + inputs: + version: '22.x' + - bash: | + cd scripts/azure-pipelines/owners-db && npm ci || true + # Construct the blob base url using the secret SAS token set earlier + blob="https://vcpkgbinarycachewus.blob.core.windows.net/cache?${BCACHE_SAS_TOKEN}" + if [ "$(Build.Reason)" = "PullRequest" ]; then + echo "Running file_script_from_cache for PR" + npx --yes ts-node ./file_script_from_cache.ts --pr-hashes "$(Build.ArtifactStagingDirectory)/pr-hashes.json" --blob-base-url "$blob" --target-branch "origin/master" --out-dir ../../list_files + else + echo "Running file_script for CI" + npx --yes ts-node ./file_script.ts --info-dir /Users/vcpkg/Data/installed/vcpkg/info/ --out-dir ../../list_files + fi + displayName: 'Build a file list for all packages' + condition: always() + env: + BCACHE_SAS_TOKEN: $(BCACHE_SAS_TOKEN) + - task: PublishPipelineArtifact@1 + displayName: "Publish Artifact: file lists for ${{ replace(parameters.jobName, '_', '-') }}" + condition: always() + inputs: + targetPath: scripts/list_files + artifact: "file lists for ${{ replace(parameters.jobName, '_', '-') }}" + - task: PublishTestResults@2 + displayName: 'Publish Test Results' + condition: ne(variables['XML_RESULTS_FILE'], '') + inputs: + testRunTitle: ${{ replace(parameters.jobName, '_', '-') }} + testResultsFormat: xUnit + testResultsFiles: $(XML_RESULTS_FILE) + platform: ${{ replace(parameters.jobName, '_', '-') }} + configuration: static diff --git a/vcpkg/scripts/azure-pipelines/osx/register-guest.sh b/vcpkg/scripts/azure-pipelines/osx/register-guest.sh new file mode 100644 index 0000000..f714e45 --- /dev/null +++ b/vcpkg/scripts/azure-pipelines/osx/register-guest.sh @@ -0,0 +1,23 @@ +#!/bin/sh +if [ -z "$1" ]; then + echo "PAT missing" + exit 1 +fi +export AGENT=$(hostname | sed -nr 's/([^.]+).*/\1/p' | tr '[:lower:]' '[:upper:]') +echo "THIS IS AGENT: $AGENT" +if [ `uname -m` = 'arm64' ]; then +export POOL=`echo ~/Parallels/*/ | sed -nr 's/\/Users\/vcpkg\/Parallels\/vcpkg-osx-([0-9]{4}-[0-9]{2}-[0-9]{2})-arm64\/$/PrOsx-\1-arm64/p'` +# on arm64, DNS works +export SSH_COOKIE=vcpkg@vcpkgs-Virtual-Machine.local +else +export POOL=`echo ~/Parallels/*.pvm | sed -nr 's/\/Users\/vcpkg\/Parallels\/vcpkg-osx-([0-9]{4}-[0-9]{2}-[0-9]{2})\.pvm/PrOsx-\1/p'` +# on amd64, DNS does not work, but luckily for us prlctl does know the IP +export GUEST_IP=`prlctl list --full | sed -nr 's/^.*running *([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}).*/\1/p'` +export SSH_COOKIE=vcpkg@$GUEST_IP +fi +echo "POOL: $POOL" +echo "SSH_COOKIE: $SSH_COOKIE" +ssh $SSH_COOKIE -o "StrictHostKeyChecking=no" -i ~/Parallels/*/id_guest "~/myagent/config.sh --unattended --url https://dev.azure.com/vcpkg --work ~/Data/work --auth pat --token $1 --pool $POOL --agent $AGENT --replace --acceptTeeEula" +if [ `uname -m` = 'arm64' ]; then + ssh $SSH_COOKIE -o "StrictHostKeyChecking=no" -i ~/Parallels/*/id_guest "sudo shutdown -h now" +fi diff --git a/vcpkg/scripts/azure-pipelines/osx/setup-box.sh b/vcpkg/scripts/azure-pipelines/osx/setup-box.sh new file mode 100644 index 0000000..0dd81e7 --- /dev/null +++ b/vcpkg/scripts/azure-pipelines/osx/setup-box.sh @@ -0,0 +1,26 @@ +#!/bin/sh + +hdiutil attach clt.dmg -mountpoint /Volumes/setup-installer +sudo installer -pkg "/Volumes/setup-installer/Command Line Tools.pkg" -target / +hdiutil detach /Volumes/setup-installer +rm clt.dmg +sudo xcode-select -s /Applications/Xcode.app +/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" +if [ `uname -m` = 'arm64' ]; then + (echo; echo 'eval "$(/opt/homebrew/bin/brew shellenv)"') >> /Users/vcpkg/.zprofile + eval "$(/opt/homebrew/bin/brew shellenv)" +else + (echo; echo 'eval "$(/usr/local/bin/brew shellenv)"') >> /Users/vcpkg/.zprofile + eval "$(/usr/local/bin/brew shellenv)" +fi +brew install autoconf-archive autoconf automake azcopy azure-cli bison cmake gettext gfortran gnu-sed gperf gtk-doc libtool meson mono nasm ninja pkg-config powershell python-setuptools texinfo yasm +mkdir ~/Data +if [ `uname -m` = 'arm64' ]; then +curl -s -o ~/Downloads/azure-agent.tar.gz https://download.agent.dev.azure.com/agent/4.261.0/vsts-agent-osx-arm64-4.261.0.tar.gz +else +curl -s -o ~/Downloads/azure-agent.tar.gz https://download.agent.dev.azure.com/agent/4.261.0/vsts-agent-osx-x64-4.261.0.tar.gz +fi +mkdir ~/myagent +tar xf ~/Downloads/azure-agent.tar.gz -C ~/myagent +rm ~/Downloads/azure-agent.tar.gz +rm setup-box.sh diff --git a/vcpkg/scripts/azure-pipelines/osx/setup-guest.sh b/vcpkg/scripts/azure-pipelines/osx/setup-guest.sh new file mode 100644 index 0000000..8efd97d --- /dev/null +++ b/vcpkg/scripts/azure-pipelines/osx/setup-guest.sh @@ -0,0 +1,21 @@ +#!/bin/sh +export VM_DIRECTORY=`ls ~/Parallels | grep vcpkg-osx` +export SSH_KEY="$HOME/Parallels/$VM_DIRECTORY/id_guest" +export SSH_PUBLIC_KEY="$SSH_KEY.pub" +ssh-keygen -P '' -f "$SSH_KEY" +if [ `uname -m` = 'arm64' ]; then +# on arm64, prlctl does not know the IP address, but luckily for us DNS works +export SSH_COOKIE=vcpkg@vcpkgs-Virtual-Machine.local +else +# on amd64, DNS does not work, but luckily for us prlctl does know the IP +export GUEST_IP=`prlctl list --full | sed -nr 's/^.*running *([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}).*/\1/p'` +export SSH_COOKIE=vcpkg@$GUEST_IP +fi +echo Type 'vcpkg' and press enter +ssh-copy-id -i "$SSH_PUBLIC_KEY" $SSH_COOKIE +echo Keys deployed +ssh $SSH_COOKIE -i "$SSH_KEY" echo hello from \`hostname\` +scp -i "$SSH_KEY" ./clt.dmg $SSH_COOKIE:/Users/vcpkg/clt.dmg +scp -i "$SSH_KEY" ./setup-box.sh $SSH_COOKIE:/Users/vcpkg/setup-box.sh +ssh $SSH_COOKIE -i "$SSH_KEY" chmod +x /Users/vcpkg/setup-box.sh +ssh $SSH_COOKIE -i "$SSH_KEY" /Users/vcpkg/setup-box.sh diff --git a/vcpkg/scripts/azure-pipelines/osx/start-parallels-vm.zsh b/vcpkg/scripts/azure-pipelines/osx/start-parallels-vm.zsh new file mode 100644 index 0000000..5e60b2a --- /dev/null +++ b/vcpkg/scripts/azure-pipelines/osx/start-parallels-vm.zsh @@ -0,0 +1,47 @@ +#!/usr/bin/env zsh + +set -e # Exit on any error + +# Find the .pvmp file +PVMP_FILE=$(find ~/Parallels -name "*.pvmp" -type f | head -1) +if [[ -z "$PVMP_FILE" ]]; then + echo "Error: No .pvmp file found in ~/Parallels" + exit 1 +fi + +echo "Found PVMP file: $PVMP_FILE" + +# Attach the PVMP file +echo "Registering PVMP file..." +/usr/local/bin/prlctl register "$PVMP_FILE" + +VM_NAME=$(/usr/local/bin/prlctl list --all --output name --no-header | head -1) + +if [[ -z "$VM_NAME" ]]; then + echo "Error: Failed to register PVMP file or extract VM ID" + exit 1 +fi + +echo "VM registered with ID: $VM_NAME" + +# Unpack the VM +echo "Unpacking VM..." +/usr/local/bin/prlctl unpack "$VM_NAME" + +# Configure startup and shutdown settings +echo "Configuring VM startup and shutdown settings..." +/usr/local/bin/prlctl set "$VM_NAME" --startup-view headless +/usr/local/bin/prlctl set "$VM_NAME" --autostart start-host +/usr/local/bin/prlctl set "$VM_NAME" --autostop shutdown +/usr/local/bin/prlctl set "$VM_NAME" --on-shutdown close +/usr/local/bin/prlctl set "$VM_NAME" --on-window-close keep-running + +echo "VM startup and shutdown settings configured successfully" + +# Display current VM configuration for verification +echo "Current VM configuration:" +/usr/local/bin/prlctl list "$VM_NAME" --info | grep -E "(Autostart|Autostop|Startup view|On shutdown|On window close)" + +echo "Parallels VM setup completed successfully!" +echo "VM ID: $VM_NAME" + diff --git a/vcpkg/scripts/azure-pipelines/osx/stop-delete-all-vms.zsh b/vcpkg/scripts/azure-pipelines/osx/stop-delete-all-vms.zsh new file mode 100644 index 0000000..1eb6fef --- /dev/null +++ b/vcpkg/scripts/azure-pipelines/osx/stop-delete-all-vms.zsh @@ -0,0 +1,17 @@ +#!/usr/bin/env zsh + +vm_name=$(/usr/local/bin/prlctl list --all --output name --no-header 2>/dev/null | head -1) +if [[ -z "$vm_name" ]]; then + echo "No VM found on the system." + exit 0 +fi + +echo "Found VM: $vm_name" + +# Stop the VM if it's running +echo "Stopping VM..." +/usr/local/bin/prlctl stop "$vm_name" --kill 2>/dev/null || echo "VM was already stopped or failed to stop" + +# Delete the VM +echo "Deleting VM..." +/usr/local/bin/prlctl delete "$vm_name"
\ No newline at end of file |