Trying to rebuild aws-crt-nodejs

[Jake He]

Hi,

I stuck at rebuilding https://github.com/awslabs/aws-crt-nodejs on windows

I noticed https://github.com/awslabs/aws-crt-nodejs/blob/5b0fea4f4180e9b2c72721d05c64708a91c0e00b/.github/workflows/ci.yml#L84

But I can't download build.pyz

How do I download it?

Sorry, it might should a trivial question.

This is what I did so far

% python -m zipapp builder
%  python builder.pyz build -p aws-crt-nodejs downstream
Traceback (most recent call last):
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.7_3.7.2544.0_x64__qbz5n2kfra8p0\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.7_3.7.2544.0_x64__qbz5n2kfra8p0\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "builder.pyz\__main__.py", line 1, in <module>
ImportError: attempted relative import with no known parent package

[Bret Ambrose]

The line above it does the download.

[Jake He]

I realised that. But I get HTTP Error 403: Forbidden.

https://d19elf31gohf1l.cloudfront.net/releases/v0.8.11/build.pyz

Is this the wrong URL?

PS C:\Code\aws-crt-builder> python -c "from urllib.request import urlretrieve; urlretrieve('https://d19elf31gohf1l.cloudfront.net/releases/v0.8.11/build.pyz?run=1-1', 'builder.pyz')"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.7_3.7.2544.0_x64__qbz5n2kfra8p0\lib\urllib\request.py", line 247, in urlretrieve
    with contextlib.closing(urlopen(url, data)) as fp:
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.7_3.7.2544.0_x64__qbz5n2kfra8p0\lib\urllib\request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.7_3.7.2544.0_x64__qbz5n2kfra8p0\lib\urllib\request.py", line 531, in open
    response = meth(req, response)
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.7_3.7.2544.0_x64__qbz5n2kfra8p0\lib\urllib\request.py", line 641, in http_response
    'http', request, response, code, msg, hdrs)
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.7_3.7.2544.0_x64__qbz5n2kfra8p0\lib\urllib\request.py", line 569, in error
    return self._call_chain(*args)
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.7_3.7.2544.0_x64__qbz5n2kfra8p0\lib\urllib\request.py", line 503, in _call_chain
    result = func(*args)
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.7_3.7.2544.0_x64__qbz5n2kfra8p0\lib\urllib\request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden

[Bret Ambrose]

Alternatively, you can use the builder source directly via https://github.com/awslabs/aws-crt-builder

[Jake He]

This is what I did. Can I skip the tests?

PS C:\Code\aws-crt-builder\builder> python main.py build -p aws-crt-nodejs downstream

Now I got test failure.

> aws-crt@1.0.0-dev test:native C:\Code\aws-crt-builder\builder\aws-crt-nodejs
> npx jest --runInBand --verbose --config test/native/jest.config.js --forceExit

FAIL lib/common/mqtt.spec.ts (25.906 s)
   MQTT Connect/Disconnect (5022 ms)
   MQTT Pub/Sub (3591 ms)
   MQTT Will (5037 ms)
   MQTT On Any Publish (3492 ms)
   MQTT payload types (5013 ms)

   MQTT Pub/Sub

    CredentialsError: Missing credentials in config, if using AWS_CONFIG_FILE, set AWS_SDK_LOAD_CONFIG=1

      at Timeout.connectTimeout [as _onTimeout] (node_modules/aws-sdk/lib/http/node.js:69:15)

   MQTT On Any Publish

    CredentialsError: Missing credentials in config, if using AWS_CONFIG_FILE, set AWS_SDK_LOAD_CONFIG=1

      at Timeout.connectTimeout [as _onTimeout] (node_modules/aws-sdk/lib/http/node.js:69:15)

PASS lib/common/http.spec.ts
   HTTP Headers
   HTTP Request without body (16 ms)
   HTTP Request with body
   HTTP Connection Create/Destroy (251 ms)
   HTTPS Connection Create/Destroy (891 ms)
   HTTP Stream GET (454 ms)
   HTTPS Stream GET (968 ms)
   HTTP Stream UnActivated (142 ms)
   HTTP Connection Manager create/destroy
   HTTP Connection Manager acquire/release (250 ms)
   HTTP Connection Manager acquire/stream/release (328 ms)

PASS lib/native/auth.spec.ts
   AWS Signer SigV4 Headers (17 ms)
   AWS Signer SigV4 Request with body
   AWS Signer SigV4A Headers

PASS lib/native/mqtt.spec.ts (9.007 s)
   MQTT Native Websocket Connect/Disconnect (5017 ms)
   MQTT Native Websocket Connect/Disconnect with TLS Context Options (3615 ms)

PASS lib/native/crypto.spec.ts
   md5 multi-part matches
   md5 one-shot matches
   SHA256 multi-part matches
   SHA256 one-shot matches
   hmac-256 multi-part matches
   hmac-256 one-shot matches

PASS lib/native/io.spec.ts
   Error Resolve
   ALPN availability

PASS lib/native/crt.spec.ts
   Native Memory

PASS lib/common/platform.spec.ts
   platform.is_nodejs is correct
   platform.is_browser is correct

Test Suites: 1 failed, 1 skipped, 7 passed, 8 of 9 total
Tests:       2 failed, 14 skipped, 30 passed, 46 total
Snapshots:   0 total
Time:        42.538 s
Ran all test suites.
Force exiting Jest: Have you considered using `--detectOpenHandles` to detect async operations that kept running after all tests finished?
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! aws-crt@1.0.0-dev test:native: `npx jest --runInBand --verbose --config test/native/jest.config.js --forceExit`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the aws-crt@1.0.0-dev test:native script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\Users\jake\AppData\Roaming\npm-cache\_logs\2021-06-15T23_00_12_650Z-debug.log
Failed to run npm run test:native: Command exited with code 1
Command failed, exiting

[Jake He]

I guess I need to set

AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY

[Jake He]

Don't worry about it. I managed to build aws-crt-nodejs from source.