Yarn tool install get timeout when behind proxy

Question

Yarn tool install get timeout when behind proxy

DalpheSDC opened this issue 6 years ago · comments

DalpheSDC commented 6 years ago

Before opening an issue please read this.

Are you reporting a bug or a feature request ?

[x ] Bug
Feature request

Description

When using Yarn install task in VSTS Self hosted agent it failes with a timeout on Yarn install. I know that the HTTP_PROXY, HTTPS_PROXY enviroment veriable is set and the proxy file on the agent is set, but every time i trieds to run the task it get this teamout
2018-11-26T13:50:35.0516311Z ##[debug]Evaluating condition for step: 'Use Yarn 1.x'
2018-11-26T13:50:35.0517068Z ##[debug]Evaluating: succeeded()
2018-11-26T13:50:35.0517196Z ##[debug]Evaluating succeeded:
2018-11-26T13:50:35.0517392Z ##[debug]=> True
2018-11-26T13:50:35.0517578Z ##[debug]Result: True
2018-11-26T13:50:35.0517751Z ##[section]Starting: Use Yarn 1.x
2018-11-26T13:50:35.0522665Z ==============================================================================
2018-11-26T13:50:35.0522742Z Task : Yarn Tool Installer
2018-11-26T13:50:35.0522777Z Description : Installs a version of Yarn
2018-11-26T13:50:35.0522835Z Version : 2.7.1003
2018-11-26T13:50:35.0522874Z Author : Geek Learning
2018-11-26T13:50:35.0522906Z Help : More Information (Version 2.7.1003)
2018-11-26T13:50:35.0522944Z ==============================================================================
2018-11-26T13:50:38.5930039Z ##[debug]agent.TempDirectory=D:\VSTS\A2\w_temp
2018-11-26T13:50:38.5974688Z ##[debug]loading inputs and endpoints
2018-11-26T13:50:38.5975278Z ##[debug]loading ENDPOINT_AUTH_PARAMETER_SYSTEMVSSCONNECTION_ACCESSTOKEN
2018-11-26T13:50:38.6153826Z ##[debug]loading ENDPOINT_AUTH_SCHEME_SYSTEMVSSCONNECTION
2018-11-26T13:50:38.6154375Z ##[debug]loading ENDPOINT_AUTH_SYSTEMVSSCONNECTION
2018-11-26T13:50:38.6528406Z ##[debug]loading INPUT_CHECKLATEST
2018-11-26T13:50:38.6528939Z ##[debug]loading INPUT_INCLUDEPRERELEASE
2018-11-26T13:50:38.6989762Z ##[debug]loading INPUT_VERSIONSPEC
2018-11-26T13:50:38.6990291Z ##[debug]loaded 6
2018-11-26T13:50:38.7104110Z ##[debug]Agent.ProxyUrl="Proxy setting remove"
2018-11-26T13:50:38.7262481Z ##[debug]Agent.ProxyUsername=undefined
2018-11-26T13:50:38.7263004Z ##[debug]Agent.ProxyPassword=undefined
2018-11-26T13:50:38.7402052Z ##[debug]Agent.ProxyBypassList=undefined
2018-11-26T13:50:38.7403169Z ##[debug]expose agent proxy configuration.
2018-11-26T13:50:38.7403499Z ##[debug]Agent.CAInfo=undefined
2018-11-26T13:50:38.7403743Z ##[debug]Agent.ClientCert=undefined
2018-11-26T13:50:38.7404298Z ##[debug]Agent.ProxyUrl="Proxy setting remove"
2018-11-26T13:50:38.7404490Z ##[debug]Agent.ProxyUsername=undefined
2018-11-26T13:50:38.7404705Z ##[debug]Agent.ProxyPassword=undefined
2018-11-26T13:50:38.7404898Z ##[debug]Agent.ProxyBypassList=undefined
2018-11-26T13:50:38.7405151Z ##[debug]check path : D:\VSTS\A2\w_tasks\YarnInstaller_2ec1147b-5c33-41df-94b5-676758d87c1b\2.7.1003\node_modules\vsts-task-tool-lib\lib.json
2018-11-26T13:50:38.7405392Z ##[debug]set resource file to: D:\VSTS\A2\w_tasks\YarnInstaller_2ec1147b-5c33-41df-94b5-676758d87c1b\2.7.1003\node_modules\vsts-task-tool-lib\lib.json
2018-11-26T13:50:38.7405597Z ##[debug]system.culture=en-US
2018-11-26T13:50:38.8881683Z ##[debug]Agent.BuildDirectory=D:\VSTS\A2\w\12
2018-11-26T13:50:38.8985873Z ##[debug]testing directory 'D:\VSTS\A2\w\12\yarn'
2018-11-26T13:50:38.9078563Z ##[debug]testing directory 'D:\VSTS\A2\w\12'
2018-11-26T13:50:38.9141633Z ##[debug]mkdir 'D:\VSTS\A2\w\12\yarn'
2018-11-26T13:50:38.9180408Z ##[debug]versionSpec=1.x
2018-11-26T13:50:38.9181078Z ##[debug]checkLatest=false
2018-11-26T13:50:38.9181392Z ##[debug]includePrerelease=false
2018-11-26T13:50:38.9181655Z ##[debug]isExplicit: null
2018-11-26T13:50:38.9181909Z ##[debug]explicit? false
2018-11-26T13:50:38.9182162Z ##[debug]isExplicit: null
2018-11-26T13:50:38.9182400Z ##[debug]explicit? false
2018-11-26T13:50:38.9182621Z ##[debug]Agent.Version=2.141.1
2018-11-26T13:50:38.9182865Z ##[debug]Agent.ToolsDirectory=D:\VSTS\A2\w_tool
2018-11-26T13:50:38.9183060Z ##[debug]evaluating 0 versions
2018-11-26T13:50:38.9183314Z ##[debug]match not found
2018-11-26T13:50:38.9184315Z ##[debug]isExplicit: null
2018-11-26T13:50:38.9184591Z ##[debug]explicit? false
2018-11-26T13:51:00.1962453Z ##[debug]task result: Failed
2018-11-26T13:51:00.2044172Z ##[error]connect ETIMEDOUT 52.239.140.202:443
2018-11-26T13:51:00.2062408Z ##[debug]Processed: ##vso[task.issue type=error;]connect ETIMEDOUT 52.239.140.202:443
2018-11-26T13:51:00.2197013Z ##[debug]Processed: ##vso[task.complete result=Failed;]connect ETIMEDOUT 52.239.140.202:443
2018-11-26T13:51:00.2766613Z ##[section]Finishing: Use Yarn 1.x

Expected behavior

Describe the expected behavior

Reproduction

Insert reproduction steps here

MANDATORY FOR BUGS: Insert debug trace

In order to allow us to quickly reproduce you issue please include a link to a reproduction repository

System information

VSTS or TFS:

[x ] VSTS
TFS

Agent:

Hosted
Custom

OS:

[x ] Windows
Mac
Linux

Yarn version: 1.x
Yarn version* 1.x

Node version:
Node version*

Sandor · Answer 1 · Sat Dec 15 2018 21:39:44 GMT+0800 (China Standard Time)

you need to set environements variables proxy and https-proxy or add a .yarnrc with those values.

see https://github.com/yarnpkg/yarn/blob/master/src/config.js#L341

Sandor · Answer 2 · Tue Jan 15 2019 09:07:11 GMT+0800 (China Standard Time)

@DalpheSDC did you give it a try?

Sandor · Answer 3 · Sun Jun 02 2019 16:58:17 GMT+0800 (China Standard Time)

Closing, feel free to reopen if you need.

Anders Thorsen · Answer 4 · Wed Jun 12 2019 04:25:26 GMT+0800 (China Standard Time)

Hi!
I hope it's ok to re-open this issue as I'm facing the same problem.

Your response points to yarn itself, but it's actually the yarn installer that fails to download yarn.

I've gotten as far as finding that aparently the node http client which is used by follow-redirects aparently does not honor the standard HTTP_PROXY and HTTPS_PROXY environement variables that I've set in addition to PROXY and HTTPS-PROXY as you suggest above.

Would it be possible to use tl.getHttpProxyConfiguration(); to get the agent's proxy config and then pass that on to the options of the .get method in downloadFile?

Sandor · Answer 5 · Wed Jun 12 2019 08:40:54 GMT+0800 (China Standard Time)

A PR would be appreciated :)

…

On Wed 12 Jun 2019 at 06:25, Anders Thorsen ***@***.***> wrote: Hi! I hope it's ok to re-open this issue as I'm facing the same problem. Your response points to yarn itself, but it's actually the yarn installer that fails to download yarn. I've gotten as far as finding that aparently the node http client which is used by follow-redirects aparently does not honor the standard HTTP_PROXY and HTTPS_PROXY environement variables that I've set in addition to PROXY and HTTPS-PROXY as you suggest above. Would it be possible to use tl.getHttpProxyConfiguration(); to get the agent's proxy config and then pass that on to the options of the .get method in downloadFile? — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#59?email_source=notifications&email_token=AAUJPJR5AFZHPPTYLKMJPBDP2AC3PA5CNFSM4GGMKXL2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXOMU5I#issuecomment-501009013>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAUJPJSS2CYBZHGOBPNFDS3P2AC3PANCNFSM4GGMKXLQ> .

Anders Thorsen · Answer 6 · Thu Jun 13 2019 20:46:37 GMT+0800 (China Standard Time)

I'll see what I can do. The JavaScript changes should be doable, but testing it probably requires me to create a private marketplace deployment so I can test it inside DevOps I guess.

Sandor · Answer 7 · Sat Jun 15 2019 13:25:37 GMT+0800 (China Standard Time)

@andersthorsen, I removed the dependency to follow redirect which does not support proxies😕. Can you give it a try (use the 3.x preview version)

Anders Thorsen · Answer 8 · Tue Jun 18 2019 18:58:07 GMT+0800 (China Standard Time)

Thanks!

Unfortunately I get the same problem, except I get a bit more error messages:
##[debug]isExplicit: null
53##[debug]explicit? false
54##[debug]downloading: https://publicblobs.geeklearning.io/yarn/tarballsV2.json
55##[debug]task result: Failed
56##[error]connect ETIMEDOUT 104.27.154.154:443
57##[debug]Processed: ##vso[task.issue type=error;]connect ETIMEDOUT 104.27.154.154:443
58##[debug]Processed: ##vso[task.complete result=Failed;]connect ETIMEDOUT 104.27.154.154:443

I cloned the repository and reproduced the issue with "yarn test"

I'm creating a pull requests with the changes I needed to be able to run yarn test successfully behind a proxy.

Sandor · Answer 9 · Wed Jun 19 2019 19:03:36 GMT+0800 (China Standard Time)

I published your PR, let me know how it goes.

Anders Thorsen · Answer 10 · Wed Jun 19 2019 19:57:23 GMT+0800 (China Standard Time)

It works! Thanks for your help!

For future references: I've defined HTTPS_PROXY and HTTP_PROXY as system variables pointing to http(s)://:port.

This makes yarn work behind a proxy, and with the change done the HTTPS_PROXY variable is now also used successfully by the Yarn installer Task.