Giters

geeklearningio / gl-vsts-tasks-yarn

Yarn Package Manager Visual Studio Team Services Build and Release Management extensions

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Package install failing with Unexpected end of data and then timing out when accessing external npm feed

mitch726 opened this issue · comments

commented

Are you reporting a bug or a feature request ?

  • Bug
  • Feature request

Description

I'm experiencing an error when running the yarn install step as part of a build but it works fine on all local development machines.
error https://pkgs.dev.azure.com/uifabric/_packaging/iss/npm/registry/@cseo/styling/-/styling-2.2.3.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "Unexpected end of data"
After I see this error, the script continues running and I get
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
info There appears to be trouble with your network connection. Retrying...
info There appears to be trouble with your network connection. Retrying...
info There appears to be trouble with your network connection. Retrying...
info There appears to be trouble with your network connection. Retrying...
Until the script timesout on the next attempt and fails out.

Expected behavior

I would expect the build task to be able to successfully install the same packages that are able to be installed locally.

Reproduction

  1. Added new package to package.json sourced from external npm feed
  2. Update .npmrc with the registry entry for the new feed '@cseo:registry=https://pkgs.dev.azure.com/uifabric/_packaging/iss/npm/registry/'
  3. Create Service Connection in Azure Dev Ops and provide a PAT with Read permissions for Packaging
  4. Add the Service Connection to the build task for Yarn Install for 'Credentials for registries outside this account/collection'
  5. Run the build with npm authenticate step and then yarn install step

image
--Note: We have tried with versions 2.* and 3.* of the task but same result

##[debug]Evaluating condition for step: 'Yarn install'
##[debug]Evaluating: succeeded()
##[debug]Evaluating succeeded:
##[debug]=> True
##[debug]Result: True
##[section]Starting: Yarn install
==============================================================================
Task         : Yarn task
Description  : Executes Yarn
Version      : 3.0.1010
Author       : Geek Learning
Help         : [More Information](https://github.com/geeklearningio/gl-vsts-tasks-yarn/wiki/Yarn) (Version 3.0.1010).

Dear Angular and Ember CLI users, please check our [known issues](https://github.com/geeklearningio/gl-vsts-tasks-yarn/wiki/Known-Issues)
==============================================================================
##[debug]agent.TempDirectory=D:\a\_temp
##[debug]loading inputs and endpoints
##[debug]loading ENDPOINT_AUTH_b9631f61-c202-4b05-8fe0-5054f645033f
##[debug]loading ENDPOINT_AUTH_PARAMETER_b9631f61-c202-4b05-8fe0-5054f645033f_APITOKEN
##[debug]loading ENDPOINT_AUTH_PARAMETER_SYSTEMVSSCONNECTION_ACCESSTOKEN
##[debug]loading ENDPOINT_AUTH_SCHEME_b9631f61-c202-4b05-8fe0-5054f645033f
##[debug]loading ENDPOINT_AUTH_SCHEME_SYSTEMVSSCONNECTION
##[debug]loading ENDPOINT_AUTH_SYSTEMVSSCONNECTION
##[debug]loading INPUT_CUSTOMENDPOINT
##[debug]loading INPUT_CUSTOMREGISTRY
##[debug]loading INPUT_PRODUCTIONMODE
##[debug]loading INPUT_PROJECTDIRECTORY
##[debug]loaded 10
##[debug]Agent.ProxyUrl=undefined
##[debug]Agent.CAInfo=undefined
##[debug]Agent.ClientCert=undefined
##[debug]Agent.SkipCertValidation=undefined
##[debug]check path : D:\a\_tasks\Yarn_0888c9a9-d1b2-4567-aae2-beadcb2059b3\3.0.1010\task.json
##[debug]adding resource file: D:\a\_tasks\Yarn_0888c9a9-d1b2-4567-aae2-beadcb2059b3\3.0.1010\task.json
##[debug]system.culture=en-US
##[debug]which 'yarn'
##[debug]found: 'D:\a\_tool\yarn\1.12.3\x64\yarn-v1.12.3\bin\yarn.cmd'
##[debug]arguments=null
##[debug]projectDirectory=D:\a\1\s
##[debug]customRegistry=useNpmrc
##[debug]Getting Packaging service access points
##[debug]System.TeamFoundationCollectionUri=https://microsoftit.visualstudio.com/
##[debug]System.ServerType=Hosted
##[debug]Getting credentials for local feeds
SYSTEMVSSCONNECTION exists true
##[debug]SYSTEMVSSCONNECTION exists true
##[debug]Got auth token
##[debug]System.ServerType=Hosted
##[debug]Agent.ProxyUrl=undefined
##[debug]Getting URI for area ID 4C83CFC1-F33A-477E-A789-29D38FFCA52E from https://microsoftit.visualstudio.com/
##[debug]Getting credentials for local feeds
SYSTEMVSSCONNECTION exists true
##[debug]SYSTEMVSSCONNECTION exists true
##[debug]Got auth token
##[debug]Agent.ProxyUrl=undefined
##[debug]Acquiring Packaging endpoints from https://microsoftit.pkgs.visualstudio.com/
##[debug]Successfully acquired the connection data
##[debug]Acquired location
##[debug]{"PackagingUris":["https://microsoftit.visualstudio.com/","https://microsoftit.pkgs.visualstudio.com/","https://pkgsprodscussu1.pkgs.visualstudio.com/","https://microsoftit.pkgs.visualstudio.com/","https://microsoftit.pkgs.visualstudio.com/","https://pkgs.dev.azure.com/MicrosoftIT/"],"DefaultPackagingUri":"https://microsoftit.pkgs.visualstudio.com/"}
##[debug]D:\a\_tool\yarn\1.12.3\x64\yarn-v1.12.3\bin\yarn.cmd
##[debug]Build.BuildId=2620288
##[debug]Agent.BuildDirectory=D:\a\1
##[debug]testing directory 'D:\a\1\npm'
##[debug]testing directory 'D:\a\1'
##[debug]mkdir 'D:\a\1\npm'
##[debug]Found 3 registries in this account/collection
##[debug]Getting credentials for local feeds
SYSTEMVSSCONNECTION exists true
##[debug]SYSTEMVSSCONNECTION exists true
##[debug]Got auth token
##[debug]Getting credentials for local feeds
SYSTEMVSSCONNECTION exists true
##[debug]SYSTEMVSSCONNECTION exists true
##[debug]Got auth token
##[debug]Getting credentials for local feeds
SYSTEMVSSCONNECTION exists true
##[debug]SYSTEMVSSCONNECTION exists true
##[debug]Got auth token
##[debug]Using registries in .npmrc
##[debug]customEndpoint=b9631f61-c202-4b05-8fe0-5054f645033f
b9631f61-c202-4b05-8fe0-5054f645033f exists true
##[debug]b9631f61-c202-4b05-8fe0-5054f645033f exists true
##[debug]b9631f61-c202-4b05-8fe0-5054f645033f=https://pkgs.dev.azure.com/uifabric/_packaging/iss/npm/registry
##[debug]Agent.ProxyUrl=undefined
##[debug]AZURE_HTTP_USER_AGENT=VSTS_cdcc3dee-d62a-41ee-aded-daf587e1851b_build_20173_0
##[debug]Processed: ##vso[task.setsecret]***
##[debug]Adding auth for registry: https://microsoftit.pkgs.visualstudio.com/_packaging/MicrosoftIT/npm/registry/
##[debug]Adding auth for registry: https://microsoftit.pkgs.visualstudio.com/_packaging/Sales/npm/registry/
##[debug]Adding auth for registry: https://pkgs.dev.azure.com/uifabric/_packaging/iss/npm/registry/
##[debug]Adding auth for registry: https://pkgs.dev.azure.com/uifabric/_packaging/iss/npm/registry/
##[debug]which 'yarn'
##[debug]found: 'D:\a\_tool\yarn\1.12.3\x64\yarn-v1.12.3\bin\yarn.cmd'
##[debug]which 'yarn'
##[debug]found: 'D:\a\_tool\yarn\1.12.3\x64\yarn-v1.12.3\bin\yarn.cmd'
##[debug]ProductionMode=false
##[debug]exec tool: D:\a\_tool\yarn\1.12.3\x64\yarn-v1.12.3\bin\yarn.cmd
##[debug]arguments:
[command]C:\Windows\system32\cmd.exe /D /S /C "D:\a\_tool\yarn\1.12.3\x64\yarn-v1.12.3\bin\yarn.cmd"
yarn install v1.12.3
[1/4] Resolving packages...
warning Resolution field "js-yaml@3.13.1" is incompatible with requested version "js-yaml@~3.7.0"
[2/4] Fetching packages...
error https://pkgs.dev.azure.com/uifabric/_packaging/iss/npm/registry/@cseo/styling/-/styling-2.2.3.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "Unexpected end of data"
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
info There appears to be trouble with your network connection. Retrying...
info There appears to be trouble with your network connection. Retrying...
info There appears to be trouble with your network connection. Retrying...
info There appears to be trouble with your network connection. Retrying...
##[debug]Exit code 1 received from tool 'D:\a\_tool\yarn\1.12.3\x64\yarn-v1.12.3\bin\yarn.cmd'
##[debug]STDIO streams have closed for tool 'D:\a\_tool\yarn\1.12.3\x64\yarn-v1.12.3\bin\yarn.cmd'
##[debug]Error: The process 'D:\a\_tool\yarn\1.12.3\x64\yarn-v1.12.3\bin\yarn.cmd' failed with exit code 1
##[debug]Error: The process 'D:\a\_tool\yarn\1.12.3\x64\yarn-v1.12.3\bin\yarn.cmd' failed with exit code 1
    at ExecState._setResult (D:\a\_tasks\Yarn_0888c9a9-d1b2-4567-aae2-beadcb2059b3\3.0.1010\node_modules\azure-pipelines-task-lib\toolrunner.js:816:25)
    at ExecState.CheckComplete (D:\a\_tasks\Yarn_0888c9a9-d1b2-4567-aae2-beadcb2059b3\3.0.1010\node_modules\azure-pipelines-task-lib\toolrunner.js:799:18)
    at ChildProcess.<anonymous> (D:\a\_tasks\Yarn_0888c9a9-d1b2-4567-aae2-beadcb2059b3\3.0.1010\node_modules\azure-pipelines-task-lib\toolrunner.js:721:19)
    at emitTwo (events.js:106:13)
    at ChildProcess.emit (events.js:191:7)
    at maybeClose (internal/child_process.js:886:16)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:226:5)
##[debug]task result: Failed
##[error]Error: The process 'D:\a\_tool\yarn\1.12.3\x64\yarn-v1.12.3\bin\yarn.cmd' failed with exit code 1
##[debug]Processed: ##vso[task.issue type=error;]Error: The process 'D:\a\_tool\yarn\1.12.3\x64\yarn-v1.12.3\bin\yarn.cmd' failed with exit code 1
##[debug]Processed: ##vso[task.complete result=Failed;]Error: The process 'D:\a\_tool\yarn\1.12.3\x64\yarn-v1.12.3\bin\yarn.cmd' failed with exit code 1
##[debug]Agent.BuildDirectory=D:\a\1
##[debug]rm -rf D:\a\1\npm
##[debug]removing directory
##[section]Finishing: Yarn install

In order to allow us to quickly reproduce you issue please include a link to a reproduction repository

System information

VSTS or TFS:

  • VSTS
  • TFS

Agent:

  • Hosted
  • Custom

OS:

  • Windows
  • Mac
  • Linux

Yarn version:
1.12.3*

Node version:
10.15.3*

commented
commented

Let me give some of those options a try and I'll get back to you. Using NPM definitely gives a more verbose error. However, I'm not sure what to do with that information. I'll keep digging. I don't think the issue is with your component though.

2822 verbose stack Error: Unable to authenticate, need: Bearer authorization_uri=https://login.windows.net/72f988bf-86f1-41af-91ab-2d7cd011db47, Basic realm="https://pkgsprodcus1.pkgs.visualstudio.com/", TFS-Federated
2822 verbose stack at res.buffer.catch.then.body (D:\a_tool\node\10.16.0\x64\node_modules\npm\node_modules\npm-registry-fetch\check-response.js:94:17)
2822 verbose stack at process._tickCallback (internal/process/next_tick.js:68:7)

commented

We ended up figuring out how to fix this. We needed to set up a Service Connection in Azure DevOps to help bridge the authentication gap between ADO instances.

Here is the setting within the Yarn task we used. You can select your service connection here.
image

Clicking 'Manage' will take you to the Service Connections page in your instance to set up a new connection or to modify the existing ones.

We set up the service connection using the Personal Access Token approach and it worked great.
image
I believe the PAT needed the package read permissions but I don't recall exactly which ones we gave it when it was generated.

Another thing we found was an issue for us was that the URL used in the service connection didn't match the one in .npmrc file because of a recent change ADO did where they changed the structure of their urls from/to subdomains.