AWS typescript SDK for deno typescript runtime.
PaulThompson opened this issue · comments
Is your feature request related to a problem? Please describe.
Typescript code generated to a file tree on github, in a manner that can be imported and used by deno.
So that users of deno can use the AWS typescript SDK in future.
Describe the solution you'd like
Typescript code generated to a file tree on github, in a manner that can be imported by deno.
A simple copy of the generated typescript transformed with a find & replace to make imports use a ".ts" suffix on files eg:
https://github.com/aws/aws-sdk-js-v3/blob/master/clients/client-s3/S3.ts#L1
would read "import { S3Client } from "./S3Client.ts";
would be a start.
Deno aims to be very browser compatible including http fetch API.
Describe alternatives you've considered
An alternative process could be to create a separate repo that watches this repo and transforms the typescript sources as required.
Another alternative is to address this request upstream to Smithy for typescript code-gen https://github.com/awslabs/smithy-typescript.
Additional context
https://deno.land/
Attempted the following:
According to https://www.reddit.com/r/Deno/comments/hcdtrx/deno_faq_for_this_subreddit/
jspm.dev operates a rewriting service that transforms NPM modules.
Therefore tried this deno typescript source:
import S3 from 'https://jspm.dev/@aws-sdk/client-s3';
async function test() {
const s3 = new S3({region:'ap-southeast-2'});
const buckets = await s3.listBuckets({});
console.log(buckets);
}
test();
Unfortunately jspm doesn't implement node's http2 library:
error: Import 'https://jspm.dev/npm:@jspm/core@2/nodelibs/http2' failed: 404 Not Found
Imported from "https://jspm.dev/npm:@aws-sdk/node-http-handler@1!cjs:8"
Apparantly there is a tool https://www.npmjs.com/package/denoify to transform .ts sources.
Working on this in https://github.com/PaulThompson/aws-sdk-js-v3/tree/master/deno
Denoify was too generic for the task - particularly the fact that all the @aws-sdk/* dependencies are right here in the repo.
Had an attempt at doing the various find/replaces to get this to work.
See https://github.com/PaulThompson/aws-sdk-js-v3/tree/master/deno
Latest error on that is:
error: TS1192 [ERROR]: Module '"https://raw.githubusercontent.com/PaulThompson/aws-sdk-js-v3/master/deno/client-s3/mod"' has no default export.
import S3 from 'https://raw.githubusercontent.com/PaulThompson/aws-sdk-js-v3/master/deno/client-s3/mod.ts';
~~
at file:///tmp/test_deno_s3.ts:1:8
TS2552 [ERROR]: Cannot find name 'FileReader'. Did you mean 'fileReader'?
const fileReader = new FileReader();
~~~~~~~~~~
at https://raw.githubusercontent.com/PaulThompson/aws-sdk-js-v3/master/deno/chunked-blob-reader/mod.ts:7:28
'fileReader' is declared here.
const fileReader = new FileReader();
~~~~~~~~~~
at https://raw.githubusercontent.com/PaulThompson/aws-sdk-js-v3/master/deno/chunked-blob-reader/mod.ts:7:11
Which is unfortunately a deno missing API.
denoland/deno#5249
The fetch-http-handler should work with Deno:
I tried loading the EventBridge client via the pika.dev service to no avail:
https://www.pika.dev/npm/@aws-sdk/client-eventbridge
Deno support would be awesome for scripts that are easy to pass around without having to pass a whole project directory and without requiring colleagues to have deep knowledge of e.g., pip (Python), bundler (Ruby), npm/yarn, etc.
It's not pretty but, for anyone who comes looking, I was able to piece together a request that seems to mostly* work:
// To Run: deno run --allow-net /path/to/this/file.ts
import FetchHttpHandler from 'https://jspm.dev/@aws-sdk/fetch-http-handler';
import AwsTypes from 'https://jspm.dev/@aws-sdk/types'
import SignatureV4 from 'https://jspm.dev/@aws-sdk/signature-v4';
import Sha256 from "https://jspm.dev/@aws-crypto/sha256-js";
import moment from 'https://cdn.skypack.dev/moment';
const signer = new SignatureV4.SignatureV4({
credentials: {
accessKeyId: 'SomeAccessKey',
secretAccessKey: 'SomeSecretKey',
sessionToken: 'SomeSessionToken'
},
region: "eu-west-2",
service: "events",
sha256: Sha256.Sha256
});
var request = {
method: "POST",
headers: {
"X-Amz-Target": "AWSEvents.PutEvents",
"Content-Type": "application/x-amz-json-1.1"
},
body: JSON.stringify({
"Entries": [
{
"DetailType": "Scheduled Event",
"Source": "aws.events",
"Resources": [ "arn:aws:events:eu-west-2:SomeAccountId:rule/SomeRuleName" ],
"Account": "SomeAccountId",
"Time": moment().valueOf(),
"Region": "eu-west-2",
"Detail": "{}"
}
]
}),
protocol: "https:",
hostname: "events.eu-west-2.amazonaws.com",
port: 443,
path: "/",
query: null,
}
request = await signer.sign(request);
const fetcher = new FetchHttpHandler.FetchHttpHandler();
const response = await fetcher.handle(request);
const parsedResponse = await response.body.text();
console.log('response', parsedResponse);
* The Deno code above does seem to reach the right backend logic at AWS but the particular action I was trying to invoke is apparently not allowed by AWS. When I ran the code with expired credentials, I got the following response: {"__type":"ExpiredTokenException","message":"The security token included in the request is expired"}. After I fixed that and another minor issue about the format of the timestamp in the body of my request, I got the following response: {"Entries":[{"ErrorCode":"NotAuthorizedForSourceException","ErrorMessage":"Not authorized for the source."}],"FailedEntryCount":1} which means that my use-case of manually testing a Cron/Schedule rule won't work (even using a more mature client) because my user account, which did seem to authenticate, is not authorized to impersonate the EventBridge service itself.
deno FileReader API has been merged to master: denoland/deno#5249 🍾
I forked the work from @PaulThompson and added a few more fixes. We can now make some basic API requests:
import { S3 } from 'https://raw.githubusercontent.com/christophgysin/aws-sdk-js-v3/deno/deno/client-s3/mod.ts'
const s3 = new S3({
region: Deno.env.get('AWS_REGION'),
credentials: {
accessKeyId: Deno.env.get('AWS_ACCESS_KEY_ID')!,
secretAccessKey: Deno.env.get('AWS_SECRET_ACCESS_KEY')!,
},
})
const { Buckets = [] } = await s3.listBuckets({})
Buckets.forEach(bucket => console.log(bucket.Name))Thanks @christophgysin
Yes and now deno 1.3.0 has the FileReader landed.
someone has been working on DynamoDB client https://github.com/chiefbiiko/dynamodb
I noticed SecretsManager > listSecrets request params are missing 'Filter' param.
const listSecretsRequest: ListSecretsRequest = {
// this errors out
Filters: [{
Key: 'name',
Values: ['someSecretName],
}],
};
const existingSecretsResponse: ListSecretsResponse = await secretsManager.listSecrets(
listSecretsRequest,
);
Technically, listSecrets() function wants ListSecretsCommandInput, but this is just a type that is set to ListSecretsRequest.
I'd love to help, but not sure where to start!
@mrowles Could you open an issue on https://github.com/christophgysin/aws-sdk-js-v3/issues? Please provide a full example to reproduce the problem.
@christophgysin Done, cheers - christophgysin#3
Straight question as the outcome of this thread is not clear ..... does the AWS JavaScript SDK fully support Deno?
@bootrino No, it does not support deno at all. I maintain a fork that I try to keep in sync which supports deno, and is published on deno.land.
That's a pity, anyone know if Amazon plans to officially support deno? Is posting in this thread making such a request, or does such a request need to be posted elsewhere?
Is posting in this thread making such a request, or does such a request need to be posted elsewhere?
This is maintainer of AWS SDK for JavaScript.
Tagging recent commenters @bootrino @christophgysin @mrowles
Do upvote on the first post on this issue #1289 (comment), so that it appears in one of the most voted issues.
Some ways this request can get prioritized is:
- Use existing community supported forks of AWS SDK for JavaScript to use Deno on AWS, like https://github.com/christophgysin/aws-sdk-js-v3
- Use Deno Runtime on AWS Lambda https://github.com/hayd/deno-lambda. If Deno usage increases on AWS, Lambda team is likely to prioritize providing official support and SDK would likely follow.
- Work with Deno community to request release schedule which is backward compatible. This would increase trust among Deno community. Deno currently releases new minor version every six weeks, and I couldn't find documentation on when
v2.0will release or what would happen tov1.0when next major version is released. - Write tweets or blog posts about how support for Deno would be helpful for AWS SDK for JavaScript users. For example, a blog post about benefits of using Deno in the cloud vs other runtimes like Node.js.
Thanks for the encouragement @trivikr and the work @christophgysin on https://github.com/christophgysin/aws-sdk-js-v3 https://deno.land/x/aws_sdk
Update: The private Deno company was just announced https://deno.com/blog/the-deno-company
Rest assured that Deno will remain MIT licensed. For Deno to grow and be maximally useful, it must remain permissively free. We don’t believe the “open core” business model is right for a programming platform like Deno. We do not want to find ourselves in the unfortunate position where we have to decide if certain features are for paid customers only. If you watch our conference talks, you will find we've been hinting at commercial applications of this infrastructure for years. We are bullish about the technology stack we've built and intend to pursue those commercial applications ourselves. Our business will build on the open source project, not attempt to monetize it directly.
@trivikr As of time of writing, this issue has the highest number of 👍s by a fairly large margin: https://github.com/aws/aws-sdk-js-v3/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%2B1-desc
Great work deno community! 🎉
I can't speak to how much progress has been made on the other points, but for my use case, Deno's number 1 advantage compared to node is its excellent support for HTTP2 out of the box with its native fetch for the client side and built-in http server APIs for the server side.
From my limited research so far, the HTTP2 client situation in node is especially dire:
- node-fetch doesn't support http2 and has no plans to add it: node-fetch/node-fetch#560
- axios also doesn't support http2: axios/axios#1175
- fetch-h2 is working for some people but still has bugs to iron out: grantila/fetch-h2#116
- got has been the most usable of the bunch for me so far, but specifically doesn't support node 14 (latest version available for lambda) support due to its buggy http2 implementation: https://github.com/sindresorhus/got/blob/main/documentation/2-options.md#http2
Many of the most widely used AWS services (dynamo, s3, lambda, etc) would benefit greatly from a wider availability of high-quality HTTP2-capable clients (fewer tcp connections -> lower latency for clients and lower load on servers). In fact, this library itself can significantly reduce its maintenance burden in the long term by relying on a robust native fetch implementation in deno over maintaining its own http2 client built on top of the low-level APIs provided by node (and hope node eventually catches up with a native fetch implementation of its own).
Deno v1.25 now has experimental support for npm.
If you test AWS SDK for JavaScript (v3) with Deno v1.25, do post your summary in this issue.
If any Node.js modules are not supported on Deno, report those problems on Deno issue tracker.
Deno v1.25 now has experimental support for npm.
It's just not ready yet, their node polyfill in std didn't implement Hmac. Despite there is a Hmac module available 3rd party, the npm: resolution mechanism has no way to override that via import map.
Given the following script:
import { S3 } from "npm:@aws-sdk/client-s3";
await new S3().listBuckets({});Deno will fetch and cache all the node dependencies, and then throw this error in runtime:
error: Uncaught Error: Not implemented: crypto.Hmac
throw new Error(message);
^
at notImplemented (https://deno.land/std@0.155.0/node/_utils.ts:23:9)
at new Hmac (https://deno.land/std@0.155.0/node/internal/crypto/hash.ts:136:5)
at createHmac (https://deno.land/std@0.155.0/node/crypto.ts:264:10)
at new Hash (file:///Users/vicary/Library/Caches/deno/npm/registry.npmjs.org/@aws-sdk/hash-node/3.171.0/dist-cjs/index.js:9:54)
at hmac (file:///Users/vicary/Library/Caches/deno/npm/registry.npmjs.org/@aws-sdk/signature-v4/3.171.0/dist-cjs/credentialDerivation.js:35:18)
at getSigningKey (file:///Users/vicary/Library/Caches/deno/npm/registry.npmjs.org/@aws-sdk/signature-v4/3.171.0/dist-cjs/credentialDerivation.js:11:29)
at SignatureV4.getSigningKey (file:///Users/vicary/Library/Caches/deno/npm/registry.npmjs.org/@aws-sdk/signature-v4/3.171.0/dist-cjs/SignatureV4.js:156:57)
at SignatureV4.signRequest (file:///Users/vicary/Library/Caches/deno/npm/registry.npmjs.org/@aws-sdk/signature-v4/3.171.0/dist-cjs/SignatureV4.js:101:73)
at async file:///Users/vicary/Library/Caches/deno/npm/registry.npmjs.org/@aws-sdk/middleware-signing/3.171.0/dist-cjs/middleware.js:13:18
at async StandardRetryStrategy.retry (file:///Users/vicary/Library/Caches/deno/npm/registry.npmjs.org/@aws-sdk/middleware-retry/3.171.0/dist-cjs/StandardRetryStrategy.js:51:46)
After publishing this post, someone from Deno pointed me to aws-api.deno.dev, which turned out to provide the only DynamoDB client for Deno that worked for me right off the bat. Maybe worth considering as an alternative until we can use the official SDK.
someone from Deno pointed me to aws-api.deno.dev, which turned out to provide the only DynamoDB client for Deno that worked for me right off the bat. Maybe worth considering as an alternative until we can use the official SDK.
Thanks for the shoutout! My aws-api.deno.dev project is intended to serve as a minimal Deno-first API client for all of the various AWS services. It certainly lacks various higher-level routines, notably DynamoDB document helpers (as shown in the linked tweet) and S3 pre-signing.
Anyway, aws-api.deno.dev should work for most API-invoking usecases, and I still use it extensively,
but I hope to see it made irrelevant by first-class SDK support from AWS ❤️
in case it helps someone: got the following to work in the current Deno version:
import { PutObjectCommand, S3Client } from "https://esm.sh/@aws-sdk/client-s3@3.245.0"
// trying to load the credentials provider through esm.sh as well didn't work
import { fromSSO } from 'npm:@aws-sdk/credential-providers'
const s3 = new S3Client({ credentials, region: 'eu-central-1'})
await s3.send(new PutObjectCommand({
Bucket: "myBucket",
Key: '/my/key',
Body: 'Body'
}))The same works for the Lambda client.
@skorfmann This did not work with aws-sdk 3.264.0, with either esm.sh or the deno npm specifier, using fromContainerMetadata or fromInstanceMetadata.
Using npm specifier in deno, it is unable to import an internal package hash-node. Here's the error:
error: Uncaught TypeError: Expected a JavaScript or TypeScript module, but identified a Unknown module. Importing these types of modules is currently not supported.
Specifier: https://jspm.dev/npm:@aws-sdk/hash-node@3.267.0
at https://jspm.dev/@aws-sdk/hash-node:7:15
const exports = await import(path.join(file.path, file.name))
hi @christophgysin I tried to import_map
"aws-ses-client" : "https://deno.land/x/aws_sdk@v3.6.0.0/client-ses?source",
and then
in my edge function
import { SendEmailCommand, SESClient } from "aws-ses-client";
but I am getting this error:
Error: Module not found "https://deno.land/x/aws_sdk@v3.6.0.0/client-ses?source".
in case it helps someone: got the following to work in the current Deno version:
import { PutObjectCommand, S3Client } from "https://esm.sh/@aws-sdk/client-s3@3.245.0" // trying to load the credentials provider through esm.sh as well didn't work import { fromSSO } from 'npm:@aws-sdk/credential-providers' const s3 = new S3Client({ credentials, region: 'eu-central-1'}) await s3.send(new PutObjectCommand({ Bucket: "myBucket", Key: '/my/key', Body: 'Body' }))The same works for the Lambda client.
hi can I ask you how did you find this esm .sh link?
I want to know if there is a search engine where I can search for different esm modules somewhere?
Also thanks allot for sharing it :)
For the version you mean?
If yes, just visit https://esm.sh/@aws-sdk/client-s3 will bring you to the latest version https://esm.sh/@aws-sdk/client-s3@3.400.0.
If you are looking for other ESM bundlers, here are a few (but they are not all equal):
https://esm.sh/
https://www.jsdelivr.com/
https://www.skypack.dev/
https://jspm.org/
https://cdnjs.com/
https://unpkg.com/
https://esb.deno.dev/
https://denopkg.com/
hi @christophgysin I tried to import_map
"aws-ses-client" : "https://deno.land/x/aws_sdk@v3.6.0.0/client-ses?source",and then
in my edge function
import { SendEmailCommand, SESClient } from "aws-ses-client";but I am getting this error:
Error: Module not found "https://deno.land/x/aws_sdk@v3.6.0.0/client-ses?source".
Did you find a solution?
Updates on my last comment. As of Deno 1.39.0, npm support already allows the use of AWS SDK.
The following snippet already works.
import { S3 } from "npm:@aws-sdk/client-s3";
await new S3({}).listBuckets({});
Hi everyone on the thread.
This feature request was never meant for the AWS SDK to address, the AWS SDK for JavaScript only officially supports the Node runtime, therefore it was up to Deno, to make their runtime compatible with Node's.
Based on @vicary 's last comment I assume this is now the case, but I haven't actually looked into it.
Because of those factors, I'm going to go ahead and close this issue. Since there is a lot of comments and upvotes on this thread, if anything is left unresolved, please feel free to create a separate issue so we can address it on a per-customer cadence.
Thanks again,
Ran~
Understood, @RanVaknin.
To note though, the AWS SDK does actually have separate support for node, browser and react-native (see all runtimeConfig*.ts files).
At the time of the original post was looking like deno would be a variant needing minor differences that could be accommodated in a similar manner.
Over time deno has added support for importing npm modules as well as supporting more of node's own internal APIs to the point where it now works.
test.ts:
import { S3 } from "npm:@aws-sdk/client-s3";
import { SecretsManager } from "npm:@aws-sdk/client-secrets-manager";
console.log(
await new S3({
region:"ap-southeast-2"
}).listBuckets({}));
console.log(
await new SecretsManager({
region:"ap-southeast-2"
}).listSecrets({}));deno run --allow-sys --allow-env --allow-net --allow-read test.ts
This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.