microsoft / ga4gh-tes

C# implementation of the GA4GH TES API; provides distributed batch task execution on Microsoft Azure

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[#627 expired SAS tokens] Place runner only once rather than for each task on any given compute node

BMurri opened this issue · comments

Problem:
The task runner is downloaded for each and every task attempt. That wastes time, network bandwidth, and customer money.

Solution:

  1. Download the task runner using the pool's start task. Colocate it under batch's "shared" folder.
  2. Utilize the downloaded task runner (rather than redownloading it) in the batch_script generated by BatchNodeScriptBuilder.

Additional context
When #627 is fully implemented, this download should be the only time that TES will place a SAS token anywhere in any Azure Batch API/metadata.