lambci / docker-lambda

Im trying to get jest tests running with github actions. Locally it works fine but on github it errors. My github action file looks like this:

# This is a basic workflow to help you get started with Actions

name: CI

# Controls when the action will run. 
on:
  push:
  pull_request:

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
  # This workflow contains a single job called "build"
  build:
    # The type of runner that the job will run on
    runs-on: ubuntu-latest
    container: 
      image: lambci/lambda:build-nodejs12.x

    # Steps represent a sequence of tasks that will be executed as part of the job
    steps:
      # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
      - uses: actions/checkout@v2

      # Runs a set of commands using the runners shell
      - name: Build
        run: |
          curl --silent --location https://dl.yarnpkg.com/rpm/yarn.repo | tee /etc/yum.repos.d/yarn.repo
          curl --silent --location https://rpm.nodesource.com/setup_12.x | bash -
          yum install yarn -y
          for d in $GITHUB_WORKSPACE/lambda/*; do (cd "$d" && yarn --frozen-lockfile); done
          cd $GITHUB_WORKSPACE
          yarn --frozen-lockfile
          yarn build
          ls -la $GITHUB_WORKSPACE/lambda/hooks
          yarn test
    env:
      AWS_DEFAULT_REGION: "us-east-1"

$ tsc
Done in 10.06s.
total 72
drwxr-xr-x  3 root root  4096 Nov 28 11:51 .
drwxr-xr-x  6 root root  4096 Nov 28 11:50 ..
-rw-r--r--  1 root root    11 Nov 28 11:51 hookHandler.d.ts
-rw-r--r--  1 root root  4172 Nov 28 11:51 hookHandler.js
-rw-r--r--  1 root root  1087 Nov 28 11:50 hookHandler.ts
drwxr-xr-x 93 root root  4096 Nov 28 11:51 node_modules
-rw-r--r--  1 root root   281 Nov 28 11:50 package.json
-rw-r--r--  1 root root 40128 Nov 28 11:50 yarn.lock
yarn run v1.22.5
$ jest
FAIL test/lambda.test.ts (18.96 s)
  ✕ Test Empty event results in 500 (8621 ms)

  ● Test Empty event results in 500


    {"errorType":"Runtime.ImportModuleError","errorMessage":"Error: Cannot find module 'hookHandler'\nRequire stack:\n- /var/runtime/UserFunction.js\n- /var/runtime/index.js"}

      3 | 
      4 | test('Test Empty event results in 500', () => {
    > 5 |     var lambdaCallbackResult = dockerLambda({
        |                                ^
      6 |         taskDir: path.resolve(__dirname + "/../lambda/hooks/"),
      7 |         handler: "hookHandler.handler",
      8 |         dockerImage: 'lambci/lambda:nodejs12.x'

      at runSync (node_modules/docker-lambda/index.js:49:13)
      at Object.<anonymous> (test/lambda.test.ts:5:32)

Test Suites: 1 failed, 1 total
Tests:       1 failed, 1 total
Snapshots:   0 total
Time:        19.635 s

My test code:

var dockerLambda = require('docker-lambda')
import path from "path"

test('Test Empty event results in 500', () => {
    var lambdaCallbackResult = dockerLambda({
        taskDir: path.resolve(__dirname + "/../lambda/hooks/"),
        handler: "hookHandler.handler",
        dockerImage: 'lambci/lambda:nodejs12.x'
    })

    expect(lambdaCallbackResult.statusCode).toEqual(500);
});

Not really sure how to replicate as its working fine locally. I am probably missing some dependancy on the github runner image?

Are you sure the ../lambda/hooks/ directory exists when you're running it in GitHub Actions?

Are you sure the ../lambda/hooks/ directory exists when you're running it in GitHub Actions?

@mhart Thats actually why I have the ls -la $GITHUB_WORKSPACE/lambda/hooks up above to validate in the output that it exists in the action as I was suspecting the same thing.

my directory structure is:

 - lambda
    - hooks <!--- this is the ls -la dir from above
- test
   - lambda.test.ts

Not sure then I'm afraid. You could try outputting more debugging info from the Jest test – like ensuring that path.resolve(__dirname + "/../lambda/hooks/") is the directory you think it is, and listing that directory from your Jest test.

If that all still looks fine, then I can't really understand why the Lambda runtime can't find the file – possibly a permissions issue? All of the module-loading code is native to the runtime, not written/managed by docker-lambda

Thanks for the suggestion. Have verified inside jest test that the path does indeed contain the files and also chmod 777 everything inside of it to test if its a permission issue with no luck.

Will report back if I make any progress.

You could also just try executing the docker run command manually. That's all that dockerLambda does:

docker-lambda/index.js

Lines 33 to 40 in 4e1b563

    
           var args = ['run'] 
        
             .concat(taskDir ? ['-v', taskDir + ':/var/task'] : []) 
        
             .concat(cleanUp ? ['--rm'] : []) 
        
             .concat(addEnvVars ? ENV_ARGS : []) 
        
             .concat(dockerArgs) 
        
             .concat([dockerImage, handler, JSON.stringify(event)]) 
        
           var spawnResult = spawnSync('docker', args, spawnOptions)

That might give you an indication as to where the problem is.

It might be actually that GitHub Action's volume mounting is causing problems:

https://github.community/t/how-to-bind-repository-to-docker-container/17869

I wonder if you use:

taskDir: "/github/workflow/lambda/hooks",

Whether that will work (I realize that won't work locally, but you can create a conditional to check if you're in CI or not)

I think your suspicion about volume mounting seems to be the right direction. Even when I manually directly run

docker run --rm -v "/github/workflow/lambda/hooks":/var/task:ro,delegated lambci/lambda:nodejs12.x hookHandler.handler

or

docker run --rm -v "$GITHUB_WORKSPACE/lambda/hooks":/var/task:ro,delegated lambci/lambda:nodejs12.x hookHandler.handler

in the action I get the same error. so its nothing to do with the node side of things but the docker side / mounting

Going to close this issue as it has nothing to do with lambci but rather its a volume mounting issue in github actions as suggested above. Even with docker run -v /any_path:/var/task bash ls -lah /var/task I cant see anything I put into any_path.

I have opened a discussion here: https://github.community/t/docker-run-v-inside-container/145921

Will edit this post if I manage to solve it.

Edit: Got it working! Managed to get the contents: https://github.community/t/docker-run-v-inside-container/145921/5?u=mattvb91

For anyone else looking into the same issue here is my complete workflow:

name: CI

on:
  push:
  pull_request:

jobs:
  build:
    runs-on: ubuntu-latest
    container: 
      image: lambci/lambda:build-nodejs12.x
      options: -v /volume_mount:/volume_mount
        
    steps:
      - uses: actions/checkout@v2

      - name: Build
        run: |
          export MOUNT_PATH=/volume_mount/platform
          cp -R /__w/platform/platform /volume_mount
          curl --silent --location https://dl.yarnpkg.com/rpm/yarn.repo | tee /etc/yum.repos.d/yarn.repo
          curl --silent --location https://rpm.nodesource.com/setup_12.x | bash -
          yum install yarn -y
          for d in $MOUNT_PATH/lambda/*; do (cd "$d" && yarn --frozen-lockfile); done
          cd $MOUNT_PATH
          yarn --frozen-lockfile
          yarn build
          yarn test
        env:
          MOUNT_DIR: "/volume_mount/platform/"

the important parts i was missing:

      options: -v /volume_mount:/volume_mount

and then copying the files into that directory. Still a bit rough around the edges but I just needed it green once to work off!

and in my tests:

var dockerLambda = require('docker-lambda')
import path from "path"

//If we are inside Github Action we need to go to the mounted directory
const taskDir = path.resolve((process.env.MOUNT_DIR ?? __dirname + "/..") + "/lambda/hooks")

test('Test Empty event results in 404', () => {
    var lambdaCallbackResult = dockerLambda({
        taskDir,
        handler: "hookHandler.handler",
        dockerImage: 'lambci/lambda:nodejs12.x'
    })

    expect(lambdaCallbackResult.statusCode).toEqual(404);
});

thank you for the help @mhart appreciated!

	var args = ['run']
	.concat(taskDir ? ['-v', taskDir + ':/var/task'] : [])
	.concat(cleanUp ? ['--rm'] : [])
	.concat(addEnvVars ? ENV_ARGS : [])
	.concat(dockerArgs)
	.concat([dockerImage, handler, JSON.stringify(event)])

	var spawnResult = spawnSync('docker', args, spawnOptions)

Runtime.ImportModuleError jest