Cannot import package as ES6 module using AWS SAM CLI

[Joseph Blair]

Expected Behavior

My project uses the AWS SAM transform on a template that declares Layers for this repo, to make the datadog-lambda-js package available for import in the function code. For local development, we use the AWS SAM CLI. We currently import the package using a require() call, and expect to be able to also use the package using an ES6 import.

Actual Behavior

When switching from const { sendDistributionMetric } = require('datadog-lambda-js') to the ES6-style import (import { sendDistributionMetric } from 'datadog-lambda-js'), attempting to invoke the function results in an error:

Cannot find package 'datadog-lambda-js' imported from /var/task/src/index.js
Did you mean to import datadog-lambda-js/index.js?

Trying the suggestion of appending /index.js to the import path does not change the result. Because this issue causes our tests not to pass, I do not want to merge the updates into a live environment, so I'm not sure if this issue is limited to just the SAM CLI.

Steps to Reproduce the Problem

1. Create SAM template with Layers array containing arn:aws:lambda:us-east-1:464622532012:layer:Datadog-Extension:47 and arn:aws:lambda:us-east-1:464622532012:layer:Datadog-Node16-x:96
2. Declare function code as a module by including "type": "module" in package.json
3. Import the library as an ES6 module: import { sendDistributionMetric } from 'datadog-lambda-js'
4. Attempt to run the application locally using the SAM CLI: sam build && sam local invoke --parameter-overrides Region=us-east-1 --env-vars env.local -e ./tests/dev.json

Specifications

• Datadog Lambda Layer version: arn:aws:lambda:us-east-1:464622532012:layer:Datadog-Extension:47
• Node version: arn:aws:lambda:us-east-1:464622532012:layer:Datadog-Node16-x:96

Stacktrace

Error [ERR_MODULE_NOT_FOUND]: Cannot find package 'datadog-lambda-js' imported from /var/task/src/index.js
Did you mean to import datadog-lambda-js/index.js?
  at new NodeError (node:internal/errors:387:5)
  at packageResolve (node:internal/modules/esm/resolve:852:9)
  at moduleResolve (node:internal/modules/esm/resolve:901:20)
  at defaultResolve (node:internal/modules/esm/resolve:1115:11)
  at nextResolve (node:internal/modules/esm/loader:163:28)
  at ESMLoader.resolve (node:internal/modules/esm/loader:841:30)
  at ESMLoader.getModuleJob (node:internal/modules/esm/loader:424:18)
  at ModuleWrap.<anonymous> (node:internal/modules/esm/module_job:76:40)
  at link (node:internal/modules/esm/module_job:75:36)

[Siddhitha Bhoopathy]

Hey thank you for reaching out!
Can you please go ahead and open a support ticket and provide us with the ticket number?
Also please enable debug log by setting DD_LOG_LEVEL to debug and send us a flare following the instructions here: https://docs.datadoghq.com/serverless/aws_lambda/troubleshooting/?tab=serverlessflare#get-help

Once we have the zendesk ticket, our team will be able to take a look and investigate this further. Thanks!

[AJ Stuyvenberg]

Hi @jpblair, thanks for reaching out. I think this is a limitation within Lambda.

I've attempted to replicate this without layers (just locally, with a very simple esm project), but it seems to be working as intended:

"use strict";

import { sendDistributionMetricWithDate } from "datadog-lambda-js";

export async function hello(event, context) {
  sendDistributionMetricWithDate('foo', 1);
  const res = { statusCode: 200, body: JSON.stringify({ message: "hello, demo people!" }) }
  return res;
}

When executed:

> const { hello} = await import("./handler.mjs")
undefined
> hello()
{"status":"error","message":"datadog:handler not initialized"}
Promise {
  { statusCode: 200, body: '{"message":"hello, demo people!"}' },
  [Symbol(async_id_symbol)]: 807,
  [Symbol(trigger_async_id_symbol)]: 6
}

The handler not initialized is what I'd expect here (as I didn't wrap or initialize the handler), but it verifies that the export is correct and the right method is called without issues.

So I followed up by looking at the SAM CLI repo and found some interesting results, the most intriguing is this: aws/aws-sam-cli#3600
aws/aws-sdk-js-v3#3386

Apparently esm does not support NODE_PATH env var, which means that es modules in Lambda layers are not automatically visible to the lambda runtime.

There are some options people recommend, including creating symlinks. You may also be able to import the full path, which would be /opt/nodejs/node_modules/datadog-lambda-js, or maybe /opt/nodejs/node18/node_modules (I'm not sure).

I would recommend either using these options, or installing this library + ddtrace via npm instead of lambda layers, which is documented here

Please let me know if you have any questions, I'm going to close it for now as I do not think we can solve this problem in this library (Lambda must solve it).

Thanks!