Deno fails to start
kyeotic opened this issue · comments
I'm bundling my lambda with the recommended instructions
# Compile the handler (and cache dependencies and compile artifacts into DENO_DIR).
DENO_DIR=.deno_dir deno cache --import-map=import_map.json src/lambda.ts
# This is the "remapping" step:
cp -R .deno_dir/gen/file/$PWD/ .deno_dir/LAMBDA_TASK_ROOT
# Note: We do the inversew of this operation in bootstrap.
# First zip deno deps up
zip build/lambda.zip -x '.deno_dir/gen/file/*' -r .deno_dir
One interesting thing to note is that deno cache is producing files with double extensions
I've tried setting HANDLER_EXT = "ts.js, but I still encounter the following errors
2022-05-23T20:30:23.956-07:00 | error: Unable to load 'file:///var/task/import_map.json' import map
-- | --
| 2022-05-23T20:30:23.956-07:00 | Caused by:
| 2022-05-23T20:30:23.956-07:00 | No such file or directory (os error 2)
| 2022-05-23T20:30:24.113-07:00Copyerror: missing expected handler file 'src/lambda.ts.js' | error: missing expected handler file 'src/lambda.ts.js'
The failure to transform DENO_IMPORTMAP = "./import_map.json" the way $LAMBDA_TASK_ROOT/$HANDLER_FILE is possibly a bug, but I don't think it is related.
Hmm, aside I'm not 100% sure the .deno_dir works as well as it once did (the hashes of local files might not map correctly)
Are you including import_map.json and src/lambda.ts in the zip ?
Basically you want to include the original source files (src/lambda.ts), import_map.json and the .deno_dir e.g. https://github.com/hayd/deno-lambda/blob/b2e02b0158282b200cbc5e2bd238403cb22b1ed6/runtime/artifacts#L20
I think the .ts.js is just how deno compiles the output down but you don't need to reference that.
I do not want to include the original source files, because I dont want the lambda to have to transpile them to JS. I like that deno cache converts everything to JS; the lambda should just execute the code and not have to typecheck or compile.
I think deno cache prevents the transpile, even though you (must) include the source.
Perhaps what you want is deno bundle...? (thinking about it this is what should recommended in the docs!)
I could go with bundle I guess, but I would prefer to use cache. Cache is definitely transpiling though, I even clear the .deno_dir and re-ran cache and got what you see in the screenshot.
I tried bundle with the following
deno bundle --import-map=import_map.json src/lambda.ts build/lambda.js
# First zip deno deps up
(cd build && zip lambda.zip lambda.js)
and now the container logs this error when starting, then times out
| 2022-05-25T13:15:42.457-07:00 | warn: unable to import '.deno_dir/' as DENO_DIR
I'm not sure how to produce a usable build with deno bundle either. With the current deno cache instructions not working, I don't know how to deploy a usable deno lambda.
Trying to provide as much context as possible without the source code. This is a screenshot of the file tree for the unzipped lambda package, which I got by download it from AWS Lambda (so it should match the runtime exactly). Note the prescence of a .deno_dir
Here are all the ENV VARs
And here is the error. IT fails to find .deno_dir, even though its there. Then it loads lambda.ts.js correctly, but fails to locate the workflow import correctly, and produces a compile error.
2022-05-25T14:12:35.140-07:00 | warn: unable to import '.deno_dir/' as DENO_DIR
-- | --
| 2022-05-25T14:12:35.461-07:00 | error: Module not found "file:///var/task/dist/workflow.ts".
| 2022-05-25T14:12:35.461-07:00 | at file:///var/task/dist/lambda.ts.js:2:37
| 2022-05-25T14:12:35.488-07:00Copyerror: unable to compile dist/lambda.ts.js | error: unable to compile dist/lambda.ts.js
This might be an issue with deno cache, since it seems to be producing bad output files. Here is the lambda.ts.js file
export { handler as api } from './api.ts';
export { handler as workflow } from './workflow.ts';
//# source map removed from brevity
I doesn't change the file extensions on the imports, even though it changed the files extensions on the actual files.
This is with the 1.22.0 version for deno-lambda and the cache was produced with deno cache
EDIT: just to see if I could get it working, I did as the README said an included the typescript source
DENO_DIR=.deno_dir deno cache --unstable src/lambda.ts
cp -R .deno_dir/gen/file/$PWD/ .deno_dir/LAMBDA_TASK_ROOT
zip build/lambda.zip -x '.deno_dir/gen/file/*' -r .deno_dir src
This lambda times out without logging anything, even a console log at the first line of the handler.
@hayd So what I was doing is officially wrong and including the source TS files along with the output from deno cache does bypass type checking and compilation and use the cached files.
However, when I tried that it was still failing and timing out. If the structure of deno cache output has changed, it seems like the current version of deno and deno-lambda are no longer compatible. Perhaps the "reverse mapping" needs to be changed?
Perhaps the "reverse mapping" needs to be changed?
I think so too. 😢
Well... $%^& me. I produced a minimal repro to try to start narrowing down the issue, and of course it worked.
So something in my app code is causing the "time out before the first line of code" problem. I assume it will be one of the rather large dependencies... I just need to figure out which one
@hayd Well that didn't take long. You're going to love this.
Its the official AWS V3 Step Functions client.
https://deno.land/x/aws_sdk@v3.32.0-1/client-sfn/mod.ts
So (apologies I'd missed this before), you definitely want to include/cp the .deno_dir/deps/ directory in your zip, since this includes the downloaded and compiled dependencies.
That's getting included, and everything works if I take that package out. It does use a ton of memory though, more than it seems like it should. Averaging 226mb per execution. I think copying the deps from .deno_dir to /tmp on startup is pretty high cost to be paying. Is it really required?