Issue creating windows2022 image

[Jack Cooper]

I'm currently having issues building a new win2022 image, ubuntu images are being built fine.

The pipeline runs for ~6hours fine then errors out with the below (tested on 3 seperate runs)

Any ideas? Or a way I could dig into the issue further?

Thanks,

[Yannick Reekmans]

This project always checks out the latest version of the code from Microsoft, and tries to build it, but that might also be broken code on their end. When I started this, they did not have Releases but now they do. Ideally, this project would checkout the code from the latest Release of an image version because that's what they deemed stable and would build.

This step needs to be adjusted for it, so it checks out the tagged release instead of the bleeding edge of their repository:
https://github.com/YannickRe/azuredevops-buildagents/blob/7dcee5913853c2c964eccc6eb32756858dfeca73/buildagent-generation-template.yml#LL50-L52C47

Either you adjust it yourself, or you wait until their code is stable or until I find the time to make this improvement.
I'm sorry I can't be of more help currently.

[Erik de Bont]

@SH-JackCooper I ran into the same issue today. It seems related to the image from the pool your are using that builds the new image. I think I have a solution for this but I haven't fully tested it yet if it solves the issue.

I have created a dedicated Agent Pool in devops only for image building. It's connected to a scaleset based on a default
UbuntuLTS image. You can configure this manually or via this template script

When the VM Scale created. Select it in the Azure Portal, goto "Operations" and copy the contents of this script in the User data box. This script will install the minimal required tools (latest versions) and settings to be able to create a runner image.

After this you should create a new Agent host in Azure DevOps connected to this VM Scale Set, and have the build pipeline create the runner image via this Agent Pool.

Hope this helps!

[Erik de Bont]

@SH-JackCooper @YannickRe The above solution didn't solve the issue, although I still recommend configuring a dedicated Agent Pool to build the runner Images.

As far as I understand the issue seems related related Microsoft Defender for Endpoint already onboarded the build VM, when the runner VM is being build. This causes sysprep to fail. See actions/runner-images#6251 .

A workaround has been mentioned. I have implemented this workaround in the buildgeneration-template.yml to this message, and am currently testing it. (takes 7 hours). When it works I will make a pull request. If you want to test yourself add the following task under 'Clone the Github Repository Task' in buildgeneration-template.yml

  - task: PowerShell@2
    displayName: 'Workaround for issue: https://github.com/actions/runner-images/discussions/6251'
    inputs:
      targetType: 'inline'
      script: |        
        $TemplateFile = "./runner-images/images/win/${{ parameters.image_type }}.json"
        Write-Host 'Attempting to modify $TemplateFile with workaround'
        ((Get-Content -path $TemplateFile -Raw) -replace '"type": "azure-arm",','"type": "azure-arm","azure_tags": { "ExcludeMdeAutoProvisioning": "True" },') | Set-Content -Path $TemplateFile               
        Write-Host "$TemplateFile modified"
        Get-Content -path $TemplateFile -Raw

[Erik de Bont]

I have tested the solution, and it solved the problem.

[Erik de Bont]

@SH-JackCooper @YannickRe A workaround has been implemented in my fork https://github.com/erik-de-bont/azuredevops-buildagents and a pull request has been made for implementation in this repository,

[Jack Cooper]

@erik-de-bont Perfect, thanks for this! I've just tested this working also - was tearing my hair out thinking it was something enviroment specific.

[Erik de Bont]

@SH-JackCooper @YannickRe Unfortunately my pull request to implement this solution in the runner images repository has been rejected. Since my previous pull request was also rejected, it is getting more difficult to use the action/runner-images without modifications in a corporate environment. Therefore I have to keep the workaround steps in buildpipeline.

[Sam Conn]

I used the code from Erik's clone and was able to build a Win2022 image.

I did run into a strange error when using the "East US" location, but once I changed to "East US 2" everything worked.

[Yannick Reekmans]

The workaround is now part of the project, thanks to #33 from @erik-de-bont