test script only checks for `.vscode-test` folder but not actual Code executable

[Danny Tuppeny]

In the test script, the check for whether to download Code is just based on the folder existing:

var testRunFolder = '.vscode-test';
var testRunFolderAbsolute = path.join(process.cwd(), testRunFolder);

/// ...

fs.exists(testRunFolderAbsolute, function (exists) {
	if (exists) {
		runTests();
	} else {
		downloadExecutableAndRunTests();
	}
});

This means that if you want to run your tests against both stable and insiders you can't. eg. I have a wrapper that spawns the test script after setting env vars like this:

runTests("flutter");
runTests("flutter", "insiders");

The insiders run fails, because the first run downloads stable into .vscode-test and then the insider run assumes the insiders executable is in there.

I think the fix may relatively simple, moving the var executable decleration up outside of the runTests function and then change the exists to use that instead of the folder.

[Danny Tuppeny]

@bpasero Just noticed that on Windows if you want to use this for repeated builds (where you do not clear the .vscode-test folder) you might still hit issues because there are overlapping filenames. On Mac everything goes inside a .app folder but on Windows there are a ton of DLLs etc. that go directly into the .vscode-test folder. So, if you were to run stable tests, then insiders, then stable - the last invocation will run code.exe but many of the DLLs are overwritten from insiders.

(Note: this doesn't break anything that wasn't already broken - nobody today could be running stable+insiders tests without deleting this folder because of the original issue).

I think the best fix might be to just ensure that stable/insiders always into their own folders (eg. .vscode-test\stable and .vscode-test\insiders). What do you think? I presume these paths are isolated to this script so it shouldn't be a complicated change?

[Benjamin Pasero]

@DanTup sounds good, I am open for another PR 👍

[Danny Tuppeny]

@bpasero Great! I've sent #97.

[Benjamin Pasero]

@DanTup I published 1.1.12, maybe you could check its ok now.

[Danny Tuppeny]

@bpasero LGTM! Thanks for the quick turnaround! 👍

Running tests from 'general' in workspace 'hello_world' using version * of Code and PATH: null
### VS Code Extension Test Run ###
Current working directory: M:\Coding\Applications\Dart-Code
Downloading VS Code into "M:\Coding\Applications\Dart-Code\.vscode-test\stable" from: https://vscode-update.azurewebsites.net/1.21.0/win32-archive/stable
Running extension tests: M:\Coding\Applications\Dart-Code\.vscode-test\stable\Code M:\Coding\Applications\Dart-Code\test\test_projects\hello_world --extensionDevelopmentPath=M:\Coding\Applications\Dart-Code --extensionTestsPath=M:\Coding\Applications\Dart-Code\out\test\general


Running tests from 'general' in workspace 'hello_world' using version insiders of Code and PATH: null
### VS Code Extension Test Run ###
Current working directory: M:\Coding\Applications\Dart-Code
Downloading VS Code into "M:\Coding\Applications\Dart-Code\.vscode-test\insiders" from: https://vscode-update.azurewebsites.net/1.22.0-insider/win32-archive/insider
Running extension tests: M:\Coding\Applications\Dart-Code\.vscode-test\insiders\Code - Insiders M:\Coding\Applications\Dart-Code\test\test_projects\hello_world --extensionDevelopmentPath=M:\Coding\Applications\Dart-Code --extensionTestsPath=M:\Coding\Applications\Dart-Code\out\test\general

[Benjamin Pasero]

@DanTup awesome