Allow customize binary & binary security

[h4x3rotab]

Currently the bin files (i.e. phala-node, pruntime, and pherry) are shipped with the npm package, which is precompiled. Ideally it would be nice to support customized binaries for a few reasons:

1. The precompiled binary may not be compatible with all the OSes. Some developers may need to compile their own suites.
2. In the Phala development env, we may want to test some unreleased features locally. So allowing customized binaries would be very helpful.
3. The precompiled binaries also introduce binary security problem (after all, it's just binary). An easier way is to use the official docker image, or let the official team to publish some binaries built by Github CI/CD pipeline.

In the short term, I suggest to allow the user to define some environmental variables to specify the binary paths. The script can prefer to use the custom binary, and fall back to the default binaries.

In the long term, it can download the official binaries when initializing the project, and allow the user to change the binary in the config.

[100k]

Absolutely agreed.
Current solution is not ideal - also because it makes package large (45M packed). However as there is no public (as far as I know) repository of dev stack I didn't implement it yet.

Ideally it would be to make it configurable via project config file. Just picking version in config - also good for testing different versions of stack.
For now it is good idea to add configuration option to use custom binaries.

[100k]

Documentation updated. Check config section.
Now it is also allowed to pick custom binary.

Short term solution. However I really like idea of repository of stacks in any version.

[h4x3rotab]

Thanks for the updates. The config is very useful.

There's still a problem for me. I usually create a ./tmp directory under my project root, and put the binaries and generated files inside. I have a script to create ${PROJECT}/tmp/phala-dev-stack/bin dir, and put the binaries inside. However, although there's #DEVPHASE# placeholder in the config file to point to the devphase installation dir root, it seems no easy way to specify a path relative to the project root. I tried to write tmp/phala-dev-stack/bin/... without any luck.

I can specify the absolute path, but that makes the config stick to my local dev environment. So I cannot commit the config to the git repo.

Can you introduce another placeholder (e.g. #PROJECT#) or just make the path relative to the project root by default?

My current workaround:

import { join } from 'path';
function rel(p: string): string {
    return join(process.cwd(), p);
}
// ...
const config : ConfigOption = {
    stack: {
        node: {
            binary: rel('tmp/phala-dev-stack/bin/node'),
// ...

[h4x3rotab]

When it comes to the e2e test, I found that the blockchain and pruntime ports are hardcoded here:

devphase/src/service/api/DevPhase.ts

Lines 74 to 96 in 21f4f79

 options = { 
     nodeUrl: 'ws://localhost:9944', 
     nodeApiOptions: { 
         types: { 
             ...KhalaTypes, 
             ...PhalaSDKTypes, 
         } 
     }, 
     workerUrl: 'http://localhost:8000', 
     accountsMnemonic: '', 
     accountsPaths: { 
         alice: '//Alice', 
         bob: '//Bob', 
         charlie: '//Charlie', 
         dave: '//Dave', 
         eve: '//Eve', 
         ferdie: '//Ferdie', 
     }, 
     sudoAccount: 'alice', 
     ss58Prefix: 30, 
     clusterId: undefined, 
     ...options, 
 }; 

I can track back to etc/mocha.global.ts where it calles setup() without any config. Maybe we can just read the config from devphase.config.ts?

[100k]

Fixed in ^0.0.9.
Docs updated.

[100k]

In ^0.0.11
it is also possible to defined custom environment setup procedure
DevPhase exposes few methods to help preparing environment

prepareWorker(workerUrl : string)
waitForGatekeeper()
createCluster()
waitForClusterReady()