Giters

geeklearningio / gl-vsts-tasks-yarn

Yarn Package Manager Visual Studio Team Services Build and Release Management extensions

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

New feature: retries

kachkaev opened this issue · comments

commented

Are you reporting a bug or a feature request ?

  • Bug
  • Feature request

Description

It'd be cool to see more resilience in the yarn install task. About one in our 20-50 pipeline runs fails because of some random network or registry issue while fetching packages. Here is an example:

==============================================================================
Task         : Yarn task
Description  : Executes Yarn
Version      : 2.8.1001
Author       : Geek Learning
Help         : [More Information](https://github.com/geeklearningio/gl-vsts-tasks-yarn/wiki/Yarn) (Version 2.8.1001).

Dear Angular and Ember CLI users, please check our [known issues](https://github.com/geeklearningio/gl-vsts-tasks-yarn/wiki/Known-Issues)
==============================================================================
SYSTEMVSSCONNECTION exists true
(node:3473) Warning: Use Cipheriv for counter mode of aes-256-ctr
(node:3473) Warning: Use Cipheriv for counter mode of aes-256-ctr
(node:3473) Warning: Use Cipheriv for counter mode of aes-256-ctr
(node:3473) Warning: Use Cipheriv for counter mode of aes-256-ctr
(node:3473) Warning: Use Cipheriv for counter mode of aes-256-ctr
(node:3473) Warning: Use Cipheriv for counter mode of aes-256-ctr
(node:3473) Warning: Use Cipheriv for counter mode of aes-256-ctr
SYSTEMVSSCONNECTION exists true
SYSTEMVSSCONNECTION exists true
[command]/opt/hostedtoolcache/yarn/1.21.1/x64/yarn-v1.21.1/bin/yarn
yarn install v1.21.1
[1/5] Validating package.json...
[2/5] Resolving packages...
warning Resolution field "espree@6.0.0" is incompatible with requested version "espree@^6.1.1"
[3/5] Fetching packages...
error An unexpected error occurred: "https://pkgs.dev.azure.com/acme/_packaging/xyz/npm/registry/write-file-atomic/-/write-file-atomic-2.4.3.tgz: Request failed \"503 Service Unavailable\"".
info If you think this is a bug, please open a bug report with the information provided in "/home/vsts/work/1/s/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
warning url-loader@1.1.2: Invalid bin field for "url-loader".
warning webpack-hot-client@4.1.2: Invalid bin field for "webpack-hot-client".
warning sha.js@2.4.11: Invalid bin entry for "sha.js" (in "sha.js").
##[error]Error: The process '/opt/hostedtoolcache/yarn/1.21.1/x64/yarn-v1.21.1/bin/yarn' failed with exit code 1

Expected behavior

I'd expect some extra failover from the task in such case, e.g. via an option to have extra attempts. Meanwhile, here's our workaround:

Azure Devops tasks: Install dependencies, Install dependencies (attempt 2), Install dependencies (attempt 3)

↑ All tasks are the same except that the first two have Control options → ☑️ Continue on error

It'd be cool if we could just use a new option for the number of retries.

System information

VSTS or TFS:

  • VSTS
  • TFS
  • dev.azure.com

Agent:

  • Hosted
  • Custom

OS:

  • Windows
  • Mac
  • Linux

Yarn version:
v1.21

Node version:
v12.16.1

commented

Have you tried to reach out to Azure devops support, it does not sound right that you get Service Unavailable errors?

If this is some kind of rate limiting that requires honoring retry-after header it should be implemented in yarn itself, there are already feature request you can comment on.

commented

IMHO retry could be done on the task level, i.e. the task will invoke yarn install with all flags, check the exit code and retry N times until it's zero. This can be entirely done in the scope of the task logic.

Service Unavailable is not necessarily Azure's fault – npmjs.com can be buggy due to current load etc.

commented

+1, we are seeing plenty of 503 errors too.
It would be nice to have this retry logic

commented

Ideally Microsoft/Yarn/npm would fix the registries, so that things do not time out, but if that doesn't happen, someone else needs to implement solutions.

Second best would be if Yarn could have functionality for that, but it does not seem to happen very quickly, even though there is a two year old issue: yarnpkg/yarn#5547

If none of those are happening, it would be wonderful to have some logic in this task itself. Would at least get rid of the issues we are seeing in CI, and prevent randomly failing builds to some extent 🤔