ERROR Unable to establish one or more of the specified browser connections. This can be caused by network issues or remote device failure.

Question

ERROR Unable to establish one or more of the specified browser connections. This can be caused by network issues or remote device failure.

SebastianChece2389 opened this issue 4 years ago · comments

Hello,

What happens

After starting test with command line above (or a different browser/OS combination), then the test shows a running icon for 6 minutes, and then I get the message:
ERROR Unable to establish one or more of the specified browser connections. This can be caused by network issues or remote device failure.
Nothing more.

This is a similar problem described in another ticket, which is closed:
#10
ERROR Unable to establish one or more of the specified browser connections.

I read everything, but I do not get it done. In this other ticket are also advises how to start tunnel-logging. But files in node_modules/* have changes. Not only line count, also content. I don't get a tunnel-Log active or do not find logging content.

My setup

I am in a linux VM (Mint) on a Windows 10 host, behind a router (as most people today will be)
Node-js Project of course
"testcafe-browser-provider-saucelabs": installed this week, should be latest version
tried start very small test example with
"testcafe "saucelabs:Chrome" src/test/client/js/e2e/sctest.ts" , but tried also other starts.
I tried to start a tunnel manually with the saucelabs tunnels for linux, and it works fine:
bin/sc -u <SAUCE_USERNAME> -k <SAUCE_ACCESS_KEY> -x https://eu-central-1.saucelabs.com/rest/v1
shows me
Started scproxy on port 45829. 27 May 09:54:12 - Please wait for 'you may start your tests' to start your tests. 27 May 09:54:28 - Secure remote tunnel VM provisioned. 27 May 09:54:28 - Tunnel ID: <ID> 27 May 09:54:28 - Using no proxy for connecting to tunnel VM. 27 May 09:54:28 - Starting Selenium listener... 27 May 09:54:28 - Establishing secure TLS connection to tunnel... 27 May 09:54:28 - Selenium listener started on port 4445. **27 May 09:54:29 - Sauce Connect is up, you may start your test**s. This tells me (I hope) that there are no Internet or Firewall problems. Because manual tunnel could be opened fine. And also my SAUCE_USERNAME and SAUCE_ACCESS_KEY must be correct. When I open this tunnel, I also see the tunnel active on my Saucelabs online dashboard.
But as I understand, this manual start of a tunnel is not needed and "testcafe-browser-provider-saucelabs" start it's own tunnel.
Of course USERNAME and SAUCE_ACCESS_KEY are exported as linux env variables.
test runs fine locally without saucelabs, it's a simple test, nothing wired.

need help

I read and tried much, but I don't get a solution.
I am new to nodejs, saucelabs, etc.
Would be very kind, when someone can help, perhaps by starting to get more logfile.

thanks a lot
Sebastian

Artem Lavrov · Answer 1 · Thu May 28 2020 18:50:17 GMT+0800 (China Standard Time)

Could you please check a new tunnel and browser in the SauceLabs Dashboard when you run TestCafe? There you will see a video and screenshots of the browser. Could you show them? If the SauceLabs Dashboard is empty, do these steps:

find the node_modules/saucelabs_connector/index.js file
change the 104 line

- logfile: tunnelLogging ? 'sc_' + this.tunnelIdentifier + '.log' : null
+ logfile: '/path/to/logfile'

run TestCafe
send us the /path/to/logfile file.

SebastianC · Answer 2 · Thu May 28 2020 21:58:53 GMT+0800 (China Standard Time)

Thank you. There was no video in saucelabs. The message says, the test in saucelabs has not started. So, the error is some steps before you will find saucelabs test videos.

In the meantime I got luck and get a hint of members of another project which are also using testcafe and saucelabs.

The Problem is

Saucelabs has different APIs active. Your plugin https://github.com/DevExpress/saucelabs-connector/ which is called by "testcafe-browser-provider-saucelabs" is just handling the V1 API but not the newer V1.2 API.

"saucelabs-connector" just used only API V1:
https://github.com/DevExpress/saucelabs-connector/blob/master/src/index.js

async _getFreeMachineCount () {
        var params = {
            method: 'GET',
            url:    [`https://${SAUCE_API_HOST}/rest/v1/users`, this.username, 'concurrency'].join('/'),
            auth:   { user: this.username, pass: this.accessKey }
        };

That was correct before, but is not anymore for new saucelabs customer/user, because saucelabs also added new API 1.2 in saucelabs environments.
url: ['https://' + _sauceHost.SAUCE_API_HOST + '/rest/v1.2/users', this.username, 'concurrency'].join('/'),

The quick and dirty local fix

Here is a quick and dirty fix for local "node_modules".

Go to your local "node_modules/saucelabs-connector/lib"
edit index.js

delete or comment existing code block:

// SaucelabsConnector.prototype._getFreeMachineCount = function _getFreeMachineCount() {
 //     var params, response;
 //     return _regeneratorRuntime.async(function _getFreeMachineCount$(context$2$0) {
 //         while (1) switch (context$2$0.prev = context$2$0.next) {
 //             case 0:
 //                 params = {
 //                     method: 'GET',
 //                     url: ['https://' + _sauceHost.SAUCE_API_HOST + '/rest/v1/users', this.username, 'concurrency'].join('/'),
 //                     auth: { user: this.username, pass: this.accessKey }
 //                 };
 //                 context$2$0.next = 3;
 //                 return _regeneratorRuntime.awrap(requestPromised(params));
 //
 //             case 3:
 //                 response = context$2$0.sent;
 //                 return context$2$0.abrupt('return', JSON.parse(response.body).concurrency[this.username].remaining.overall);
 //
 //             case 5:
 //             case 'end':
 //                 return context$2$0.stop();
 //         }
 //     }, null, this);
 // };

replace or add new code block as fix

SaucelabsConnector.prototype._getFreeMachineCount = function _getFreeMachineCount() {
        var params, response, organization, allowed, current;
        return _regeneratorRuntime.async(function _getFreeMachineCount$(context$2$0) {
            while (1) switch (context$2$0.prev = context$2$0.next) {
                case 0:
                    params = {
                        method: 'GET',
                        url: ['https://' + _sauceHost.SAUCE_API_HOST + '/rest/v1.2/users', this.username, 'concurrency'].join('/'),
                        auth: { user: this.username, pass: this.accessKey }
                    };
                    context$2$0.next = 3;
                    return _regeneratorRuntime.awrap(requestPromised(params));

                case 3:
                    response = context$2$0.sent;
                    organization = JSON.parse(response.body).concurrency.organization;
                    allowed = organization.allowed.vms;
                    current = organization.current.vms;
                    return context$2$0.abrupt('return', allowed - current);

                case 8:
                case 'end':
                    return context$2$0.stop();
            }
        }, null, this);
    };

Please notice the changes in the URL: "/rest/v1/users" -> "/rest/v1.2/users"
And also notice the changes in "case 3"
This works for me and fixed my problems.

Better and global solution would be ...

Change "testcafe-browser-provider-saucela" and "saucelabs-connector"
Add l new possibility to testcafe-browser-provider-saucelabs to select saucelabs API Version as example by adding a environment configuration to select used API Version:
-- Example
export SAUCE_Connector_API_Version="V1"
export SAUCE_Connector_API_Version="V1.2"
pass API Version number to "saucelabs-connector"
Edit Github "saucelabs-connector" also and implement to make possible to use "V1" and "V1.2" regarding to Version selected at testcafe-browser-provider-saucelabs call
Much better solution would be, change "saucelabs-connector" so it can check which API is used by the saucelabs account and then use the API which is assigned to the saucelabs account. But I am not sure how to check which API on a saucelabs account must be used. Perhaps saucelabs can tell more.

I am not really a great JS programmer. My skill was good enough to implement the dirty and quick fix locally, but not to implement the global solution to both github projects.
But in the next weeks and months, more customer/user of testcafe + saucelabs will get the same error and the same problem as me, because more (new) saucelabs user will have an saucelabs environment with API V1.2. If someone can fix this in both github projects, this would be great!
If not, I will try to get it done by myself in some days and make a pull request.

Best regards and thanks a lot
Sebastian

AlexKamaev · Answer 3 · Mon Jun 01 2020 22:50:10 GMT+0800 (China Standard Time)

@SebastianChece2389
Thank you for this information. We really appreciate it. I think we definitely should support new SauceLabs API. It would be the best solution. However, I could not reproduce the issue. I created a new user and everything works as expected with V1 API.

Could you please run your tests without your changes but set the SAUCE_API_HOST environment variable to eu-central-1.saucelabs.com?

As for your workaround, we are happy to hear that you solved the issue. We would like to integrate your solution into our code base, but at the moment, the documentation of the 1.2 version is confusing.

The response in your workaround contains the organization field with a nested allowed field, etc. However, the example in the docs does not contain such a field. It's great that the workaround works, however, we need to find documented approval for this.
Could you please clarify where your colleagues read about the response format they are using.

SebastianC · Answer 4 · Wed Jun 03 2020 16:41:46 GMT+0800 (China Standard Time)

@AlexKamaev :
I forgot to mention that I already use EU Central Host, because we stay in EU. Without it, it won't work anyway.

$ echo $SAUCE_API_HOST
eu-central-1.saucelabs.com

I was not involved in the email traffic with saucelabs, but have got some information. Without quoting directly, saucelabs answered after my my colleagues telling them the problem with "testcafe-browser-provider-saucelabs":

All new accounts are created on new team management, hence saucelabs created new endpoints and the same changes in the endpoint are being published here:
https://wiki.saucelabs.com/display/DOCS/Test+Activity+and+Usage+Methods
Since TestCafe is not maintained by saucelabs, they we are not aware of the endpoints "testcafe-browser-provider-saucelabs" use internally.
As of now, no accounts on new team management will be migrated at saucelabs. Hence Ideally saucelabs expect newly updated endpoints to be used since old endpoints are deprecated.

Fix for "saucelabs-connector" to use API 1.2 should be:

async _getFreeMachineCount () {
var params = {
method: 'GET',
url: [`https://${SAUCE_API_HOST}/rest/v1.2/users`, this.username, 'concurrency'].join('/'),
auth: { user: this.username, pass: this.accessKey }
};
var response = await requestPromised(params);
var organization = JSON.parse(response.body).concurrency.organization;
var allowed = organization.allowed.vms;
var current = organization.current.vms;
return allowed - current;
}

But you are right anyway, on https://wiki.saucelabs.com/display/DOCS/Test+Activity+and+Usage+Methods there is nothing about that nested organization field.

What About this idea? I do a
curl -u USERNAME:API_KEY https://saucelabs.com/rest/v1.2/users/USERNAME/concurrency
as shown in the saucelabs documentation and copy and paste the repsonse. I think that's what my colleagues did. Because it seems saucelabs documentation is lacking or incorrect here.

SebastianC · Answer 5 · Wed Jun 03 2020 17:06:18 GMT+0800 (China Standard Time)

Some additional information:

In the Email Traffic with saucelabs I see a example response, I think there my colleagues found all information directly:

The correct endpoint:
https://eu-central-1.saucelabs.com/rest/v1.2/users/<USERNAME>/concurrency

Example response from saucelabs handed my colleagues over:
{"timestamp": <TIMESTAMP>, "concurrency": {"organization": {"current": {"vms": <current_used>, "rds": 0, "mac_vms": 0}, "id": "<ID>", "allowed": {"vms": <Allowed_VMs>, "rds": 0, "mac_vms": <AllowedOS_VMs>}}, "team": {}}}
This example shows the nested "organization" field.

Here is my own test from today:
$ curl -u <User>:<Auth_key> https://eu-central-1.saucelabs.com/rest/v1.2/users/<User>/concurrency
Actuall Response from today:
{"timestamp": <TIMESTAMP>, "concurrency": {"organization": {"current": {"vms": <current_used>, "rds": 0, "mac_vms": 0}, "id": "<ID>", "allowed": {"vms": <Allowed_VMs>, "rds": 0, "mac_vms": <AllowedOS_VMs>}}, "team": {}}}
Again by testing the endpoint today, you see the nested organization which documentation is lacking.

Would be nice to see this endpoint implemented as option in both of your github projects. Thanks a lot!

Arseniy Rubtsov · Answer 6 · Thu Jun 04 2020 17:24:19 GMT+0800 (China Standard Time)

@SebastianChece2389
Thank you for the information. We will research possible solutions and contact SauceLabs regarding the new API format. Please stay tuned for the updates.

SebastianC · Answer 7 · Thu Jun 04 2020 21:01:51 GMT+0800 (China Standard Time)

Nice to hear. I don't want to hurry, but just would be nice to know: Do you think your global solution needs "days" or more "weeks"?
Just good to know because have to implement in our CI pipeline and have to decide to use an own fork or your global solution.
Thank you very much for your support!

aleks-pro · Answer 8 · Fri Jun 05 2020 20:21:35 GMT+0800 (China Standard Time)

I cannot give you any personal estimate as they may be misleading. We will update this thread as soon as we have any results.

JackSinclairT · Answer 9 · Mon Sep 13 2021 15:54:54 GMT+0800 (China Standard Time)

Whats the status of this? Is this at a standstill?

github-actions · Answer 10 · Mon Sep 13 2021 16:35:31 GMT+0800 (China Standard Time)

No updates yet. Once we get any results, we will post them in this thread.

github-actions · Answer 11 · Sat Dec 10 2022 09:59:25 GMT+0800 (China Standard Time)

This issue has been automatically marked as stale because it has not had any activity for a long period. It will be closed and archived if no further activity occurs. However, we may return to this issue in the future. If it still affects you or you have any additional information regarding it, please leave a comment and we will keep it open.

github-actions · Answer 12 · Wed Dec 21 2022 09:56:23 GMT+0800 (China Standard Time)

We're closing this issue after a prolonged period of inactivity. If it still affects you, please add a comment to this issue with up-to-date information. Thank you.