Issues with lambda receive function

Question

Issues with lambda receive function

bagitte opened this issue 6 years ago · comments

I've setup Fraudmarc-ce on 6/23 in AWS and i'm getting the following invocation errors for the fraudmarc-ce-receive lamda function:

START RequestId: f2f63600-7779-11e8-b538-5fcd7d03aece Version: $LATEST
fork/exec /var/task/receive: no such file or directory: PathError null
END RequestId: f2f63600-7779-11e8-b538-5fcd7d03aece

I ran the Fraudmarc-ce-install Docker file to automate the setup of the lambda functions in AWS. Previously, I attempted the advance installation but encountered issues with 'apex deploy' where it indicates that there's no GO files.

This appears to be the same issue mentioned above.
https://stackoverflow.com/questions/49000980/deploying-aws-lambda-with-go-cloudformation

Jeff Kim · Answer 1 · Tue Jun 26 2018 08:17:19 GMT+0800 (China Standard Time)

Hi @bagitte ,
I will look into it tomorrow morning and write back to you!

Jeff Kim · Answer 2 · Tue Jun 26 2018 22:50:45 GMT+0800 (China Standard Time)

@bagitte for clarification, are you encountering the issue for both the advanced and simple installation? Or did the simple install work but the advanced install fail when you ran apex deploy?

bagitte · Answer 3 · Tue Jun 26 2018 23:03:36 GMT+0800 (China Standard Time)

@kimkb2011 I only encountered the apex deploy issue with the advanced installation whereby there's no GO files hence the lambda functions were never created. The simple installation completed w/out incident; however, the receive lambda function has invocation errors that it can't find a file or directory. Sorry for the confusion.

Jeff Kim · Answer 4 · Tue Jun 26 2018 23:40:25 GMT+0800 (China Standard Time)

@bagitte for the advanced install, did you run apex deploy in the project directory?

bagitte · Answer 5 · Tue Jun 26 2018 23:50:34 GMT+0800 (China Standard Time)

@kimkb2011 yep, i ran it in the project directory where project.json is located.

Jeff Kim · Answer 6 · Wed Jun 27 2018 00:20:32 GMT+0800 (China Standard Time)

@bagitte can you post the output of pwd && ls -l && apex deploy from your project directory?

bagitte · Answer 7 · Wed Jun 27 2018 00:24:18 GMT+0800 (China Standard Time)

@kimkb2011 here's the output:

[ fraudmarc-ce]$ pwd && ls -l && apex deploy
/home/ec2-user/go/src/github.com/fraudmarc/fraudmarc-ce
total 3964
-rw-rw-r-- 1 ec2-user ec2-user 10613 Jun 23 22:57 25Fraudmarc-CE-Logo-on-Light.png
drwxrwxr-x 4 ec2-user ec2-user 4096 Jun 23 22:57 backend
drwxrwxr-x 2 ec2-user ec2-user 4096 Jun 23 22:57 database
-rw-rw-r-- 1 ec2-user ec2-user 50408 Jun 23 22:57 diagram2.png
-rw-rw-r-- 1 ec2-user ec2-user 1248 Jun 23 22:57 Dockerfile
-rw-rw-r-- 1 ec2-user ec2-user 296 Jun 24 05:14 env.list
-rwxrwxr-x 1 ec2-user ec2-user 211 Jun 23 22:57 env.list.old
-rw-rw-r-- 1 ec2-user ec2-user 6646 Jun 23 22:57 fraudmarcce
drwxrwxr-x 4 ec2-user ec2-user 4096 Jun 23 22:57 frontend
drwxrwxr-x 4 ec2-user ec2-user 4096 Jun 23 22:57 functions
-rw-rw-r-- 1 ec2-user ec2-user 10243 Jun 23 22:57 INSTALL-ADVANCED.md
drwxrwxr-x 2 ec2-user ec2-user 4096 Jun 23 22:57 installer
-rw-rw-r-- 1 ec2-user ec2-user 10853 Jun 23 22:57 INSTALL.md
-rw-rw-r-- 1 ec2-user ec2-user 11357 Jun 23 22:57 LICENSE
-rw-rw-r-- 1 ec2-user ec2-user 8369 Jun 23 22:57 logo.png
-rw-rw-r-- 1 ec2-user ec2-user 3886844 Jun 23 22:57 newgif.gif
-rw-rw-r-- 1 ec2-user ec2-user 842 Jun 23 20:13 project.json
-rw-rw-r-- 1 ec2-user ec2-user 747 Jun 23 22:57 project.json.old
-rw-rw-r-- 1 ec2-user ec2-user 1269 Jun 23 22:57 README.md
-rw-rw-r-- 1 ec2-user ec2-user 373 Jun 23 22:57 SES-setup.json
⨯ Error: function process: build hook: ../../../../../gopkg.in/mgutz/dat.v1/init.go:7:2: no Go files in /home/ec2-user/go/src/github.com/mgutz/logxi

Jeff Kim · Answer 8 · Wed Jun 27 2018 01:22:36 GMT+0800 (China Standard Time)

I believe the build failed because you did not install the dependencies correctly before deploying the Lambdas. Could you tell me if running this works (the patch should resolve the build issue)?

go get github.com/fraudmarc/fraudmarc-ce/backend/lib \
       github.com/fraudmarc/fraudmarc-ce/database
(go get -d gopkg.in/mgutz/dat.v1 ; exit 0)
cd $GOPATH/src/gopkg.in/mgutz/dat.v1
patch -p1 < $GOPATH/src/github.com/fraudmarc/fraudmarc-ce/database/dat.patch
cd $GOPATH/src/github.com/fraudmarc/fraudmarc-ce
go get ./...

bagitte · Answer 9 · Wed Jun 27 2018 01:32:29 GMT+0800 (China Standard Time)

@kimkb2011 Here's the output:

[ec2-user@ip ~]$ go get github.com/fraudmarc/fraudmarc-ce/backend/lib
github.com/fraudmarc/fraudmarc-ce/database
patch -p1 < $GOPATH/src/github.com/fraudmarc/fraudmarc-ce/database/dat.patch
cd $GOPATH/src/github.com/fraudmarc/fraudmarc-ce
go get ./...go/src/gopkg.in/mgutz/dat.v1/init.go:7:2: no Go files in /home/ec2-user/go/src/github.com/mgutz/logxi
[ec2-user@ip ~]$ (go get -d gopkg.in/mgutz/dat.v1 ; exit 0)
go/src/gopkg.in/mgutz/dat.v1/init.go:7:2: no Go files in /home/ec2-user/go/src/github.com/mgutz/logxi
[ec2-user@ip ~]$ cd $GOPATH/src/gopkg.in/mgutz/dat.v1
[ec2-user@ip dat.v1]$ patch -p1 < $GOPATH/src/github.com/fraudmarc/fraudmarc-ce/database/dat.patch
[ec2-user@ip dat.v1]$ cd $GOPATH/src/github.com/fraudmarc/fraudmarc-ce
[ec2-user@ip fraudmarc-ce]$ go get ./...
../../../gopkg.in/mgutz/dat.v1/init.go:7:2: no Go files in /home/ec2-user/go/src/github.com/mgutz/logxi
[ec2-user@ip fraudmarc-ce]$ ^C
[ec2-user@ip fraudmarc-ce]$ apex deploy
⨯ Error: function receive: build hook: ../../../../../gopkg.in/mgutz/dat.v1/init.go:7:2: no Go files in /home/ec2-user/go/src/github.com/mgutz/logxi

Jeff Kim · Answer 10 · Wed Jun 27 2018 01:36:38 GMT+0800 (China Standard Time)

Does running apex deploy from the root directory still not work?

bagitte · Answer 11 · Wed Jun 27 2018 01:40:05 GMT+0800 (China Standard Time)

Correct, I get the same message: ⨯ Error: function receive: build hook: ../../../../../gopkg.in/mgutz/dat.v1/init.go:7:2: no Go files in /home/ec2-user/go/src/github.com/mgutz/logxi

bagitte · Answer 12 · Wed Jun 27 2018 01:40:49 GMT+0800 (China Standard Time)

Also, is this related to the simple installation because that went through fine and the lambda functions were created. The only issue is that error I referenced above.

Jeff Kim · Answer 13 · Wed Jun 27 2018 01:50:49 GMT+0800 (China Standard Time)

@bagitte it seems the patch isn't being applied correctly. Maybe starting from a completely clean slate of golang will fix this issue.

Regarding the simple install, we have resolved the zip issue with lambda and I will merge it to master branch. Since you already have deployed your Lambdas, RDS, etc, don't forget to delete all of the fraudmarc-ce related AWS services before re-running the simple install as it will exit if there's an existing instance with the same name.

bagitte · Answer 14 · Wed Jun 27 2018 01:52:18 GMT+0800 (China Standard Time)

@kimkb2011 sounds goods. Thanks.

Jeff Kim · Answer 15 · Wed Jun 27 2018 01:54:27 GMT+0800 (China Standard Time)

@bagitte merged.

bagitte · Answer 16 · Wed Jun 27 2018 01:59:08 GMT+0800 (China Standard Time)

@kimkb2011 will let you know in a few how that went. Thanks again.

bagitte · Answer 17 · Wed Jun 27 2018 02:55:30 GMT+0800 (China Standard Time)

@kimkb2011 so I removed the lambda role, rds, lambda functions and re-ran.

Here's the output:

**Your Database has been launched! Check the AWS RDS Console -> Instances tab (fraudmarcce)
Your IAM Role has been created! Check the AWS IAM Console (FraudmarcCE)

An error occurred (InvalidParameterValueException) when calling the CreateFunction operation: The role defined for the function cannot be assumed by Lambda.
Your Lambda functions have been created! Check the AWS Lambda Console (fraudmarc-ce-receive/process)

An error occurred (MalformedPolicyDocument) when calling the PutRolePolicy operation: Syntax errors in policy.**

The lambda role that was created looks fine. Not sure what happened there. In any event, I'm going to
remove and try again.

bagitte · Answer 18 · Wed Jun 27 2018 03:10:16 GMT+0800 (China Standard Time)

@kimkb2011 re-ran and got the same error. Yeah, it wasn't able to create the inline policy for that role. # #

bagitte · Answer 19 · Wed Jun 27 2018 03:21:42 GMT+0800 (China Standard Time)

@kimkb2011 I replaced the inline-policy and lamdba-assume-policy json with the previous copies and this time it went through successfully:

Your Database has been launched! Check the AWS RDS Console -> Instances tab (fraudmarcce)
Your IAM Role has been created! Check the AWS IAM Console (FraudmarcCE)
Your Lambda functions have been created! Check the AWS Lambda Console (fraudmarc-ce-receive/process)
Your inline-policy has been added to the FraudmarcCE role! Check the IAM Console
Your AWS Role, RDS, Lambdas has been setup. Fraudmarc CE installation complete.

Jeff Kim · Answer 20 · Wed Jun 27 2018 03:31:56 GMT+0800 (China Standard Time)

@bagitte Nice! The Dockerfile inside the installer directory looks for the string "ARN" and replaces it with your receive lambda ARN. Since you already have changed the string, the error occurred since it couldn't find the ARN it was looking for.

bagitte · Answer 21 · Wed Jun 27 2018 03:37:19 GMT+0800 (China Standard Time)

@kimkb2011 ah ok. Thanks.

bagitte · Answer 22 · Wed Jun 27 2018 04:05:49 GMT+0800 (China Standard Time)

@kimkb2011 Looks like the issue remains. Here's the output:

fork/exec /var/task/receive: no such file or directory: PathError
null

Jeff Kim · Answer 23 · Wed Jun 27 2018 04:43:14 GMT+0800 (China Standard Time)

When does this exactly occur? When you send an email to your SES recipient?

bagitte · Answer 24 · Wed Jun 27 2018 05:00:22 GMT+0800 (China Standard Time)

Yeah, after the email(s) is in the S3 bucket and then the lambda function is triggered.

Jeff Kim · Answer 25 · Wed Jun 27 2018 05:17:04 GMT+0800 (China Standard Time)

@bagitte That issue should have been fixed with the previous merge.
I have made some other changes to the /installer/Dockerfile file (resolved InvokeFunction) to the master branch. Could you try cloning the repo and go through the steps?

Also, I've made some changes in the INSTALL.md file (Database Configuration section). Check the diff and see how you can connect the Lambdas to the DB instance.

bagitte · Answer 26 · Wed Jun 27 2018 05:30:31 GMT+0800 (China Standard Time)

@kimkb2011 Ok, will do. Thanks.

bagitte · Answer 27 · Wed Jun 27 2018 08:36:11 GMT+0800 (China Standard Time)

@kimkb2011 looks like the lambda receive function is working properly now. The only issue now is that it's timing out connecting to the database. I'll need to configure the function to be able to communicate with RDS. Thanks for fixing the file/directory issue.

Keith Coleman · Answer 28 · Wed Jun 27 2018 08:55:42 GMT+0800 (China Standard Time)

@bagitte Glad to know it's working for you. Next is simply a matter of configuring your environment or DB instance. Use the DMARC forum or Fraudmarc Support for help with any of that and reserve this tracker for code issues.

ps. Did you know we provide a free service tier where it only takes ~1 minute to install DMARC for your domain? There's even an automated tool to protect your domain with p=reject.

bagitte · Answer 29 · Wed Jun 27 2018 09:22:44 GMT+0800 (China Standard Time)

@kcolemangt ok, will do. Will take a look at the free service tier. Thanks.

Jeff Kim · Answer 30 · Sat Jun 30 2018 04:58:58 GMT+0800 (China Standard Time)

Closing issue.