clean up work dir on self hosted agent

[David Vescovi]

My pipeline build on my self hosted agent creates incremental folders in the _work directory (1, 2, 3 etc). These eventually fill up my hard disk as they are not cleaned after pipeline completion. I suspect I can add a final task to clean up but I can not find any documentation on how to do this.
Any suggestion, pointers or documentation?

[Alex Vincent]

Specifically you are looking for

workspace:
  clean: all

Yes it's all in the docs, https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=schema#job

jobs:
- job: string  # name of the job (A-Z, a-z, 0-9, and underscore)
  displayName: string  # friendly name to display in the UI
  workspace:
    clean: outputs | resources | all # what to clean up before the job runs
  steps: [ script | bash | pwsh | powershell | checkout | task | templateReference ]
...

[David Vescovi]

Yea, but according to the doc you mention "clean: all" cleans up "before" the job runs.
I need something that cleans up everything "after" the job is run to get rid of all the residuals.
I was thinking you could just add an additional job at the end but I think that would still just clean its workspace job, not all the previous ones.
maybe adding a script at the end that just does a "rm -fr ." would do it?

[stale]

In order to consolidate to fewer feedback channels, we've moved suggestions and issue reporting to Developer Community. Sorry for any confusion resulting from this move.

[Eugen Mayer]

Since running pre-cleanup-jobs (workspace.clean) or post-cleanup-jobs some step X to cleanup the agent workdir heavily relies on the pipeline-author of "some pipeline" to fix a potential hard to detect "workdir pollution" issue, we decided to re-implement the idea to guarantee workdir pollution can never happen.

I have created https://github.com/EugenMayer/azure-agent-self-hosted-toolkit which fixes this project on the agent-level, not requiring any changes to the pipelines nor relying on those.

To quote to project idea

The run-once mode is based on Microsoft's `./run.sh --once` which ensures that an agents only runs 1 job and then stops.
This is used to

 - cleanup the workdir in a safe manner after each job
 - ensures each job on an agent runs in a clean workdir
 - starts the agent right after cleanup up (few seconds) to be available for the next job

This repository also offers a toolkit to start / setup x-agents and maintain them using the original tools of Microsoft, but wrapped in convenient scripts.

If this helps anybody else, happy to share it.