Zip file is not uploaded in S3

Question

Zip file is not uploaded in S3

chandantiwari opened this issue 9 years ago · comments

Hello,

I am able to properly deploy and build the packages successfully. However I am not able to download the files. I also looked into S3 bucket as well but nothing is uploaded there.
Command line says that the build is successful and results at the given s3 bucket. But nothing is there on S3.
Am I doing something wrong here ? Is anyone else facing the same issue.

Thanks,
Chandan

John Titus · Answer 1 · Sun Oct 11 2015 19:45:49 GMT+0800 (China Standard Time)

Hi Chandan,
Can you check your CloudWatch - Logs to see what the logs say?

Chandan Tiwari · Answer 2 · Sun Oct 11 2015 21:10:18 GMT+0800 (China Standard Time)

Hi John,

Following is my cloud watch logs for the latest call. It is going to some module named 'are-we-there-yet'....dont know what it is ...

{
"errorMessage": "Command failed: \nmodule.js:340\n throw err;\n ^\nError: Cannot find module 'are-we-there-yet'\n at Function.Module._resolveFilename (module.js:338:15)\n at Function.Module._load (module.js:280:25)\n at Module.require (module.js:364:17)\n at require (module.js:380:17)\n at Object. (/tmp/thaumaturgy/node_modules/npm/node_modules/npmlog/log.js:2:16)\n at Module._compile (module.js:456:26)\n at Object.Module._extensions..js (module.js:474:10)\n at Module.load (module.js:356:32)\n at Function.Module._load (module.js:312:12)\n at Module.require (module.js:364:17)\n",
"errorType": "Error",
"stackTrace": [
"module.js:340",
" throw err;",
" ^",
"Error: Cannot find module 'are-we-there-yet'",
"Function.Module._resolveFilename (module.js:338:15)",
"Function.Module._load (module.js:280:25)",
"Module.require (module.js:364:17)",
"require (module.js:380:17)",
"Object. (/tmp/thaumaturgy/node_modules/npm/node_modules/npmlog/log.js:2:16)",
"Module._compile (module.js:456:26)",
"Object.Module._extensions..js (module.js:474:10)",
"Module.load (module.js:356:32)",
"Function.Module._load (module.js:312:12)",
"Module.require (module.js:364:17)",
"",
"ChildProcess.exithandler (child_process.js:658:15)",
"ChildProcess.emit (events.js:98:17)",
"maybeClose (child_process.js:766:16)",
"Process.ChildProcess._handle.onexit (child_process.js:833:5)"
]
}

Thanks.

John Titus · Answer 3 · Sun Oct 11 2015 22:21:17 GMT+0800 (China Standard Time)

Strange. It's not part of thaumaturgy. Can you cut & paste the build command you're running? And are you running NPM 2 or 3?

https://www.npmjs.com/package/are-we-there-yet

Chandan Tiwari · Answer 4 · Sun Oct 11 2015 22:46:20 GMT+0800 (China Standard Time)

Earlier one tried using NPM 3.

Now changed to NPM2 and can see following error in cloudwatch .. now a new -
{
"errorMessage": "Command failed: \nmodule.js:340\n throw err;\n ^\nError: Cannot find module 'osenv'\n at Function.Module._resolveFilename (module.js:338:15)\n at Function.Module._load (module.js:280:25)\n at Module.require (module.js:364:17)\n at require (module.js:380:17)\n at Object. (/tmp/thaumaturgy/node_modules/npm/lib/config/defaults.js:11:13)\n at Module._compile (module.js:456:26)\n at Object.Module._extensions..js (module.js:474:10)\n at Module.load (module.js:356:32)\n at Function.Module._load (module.js:312:12)\n at Module.require (module.js:364:17)\n",
"errorType": "Error",
"stackTrace": [
"module.js:340",
" throw err;",
" ^",
"Error: Cannot find module 'osenv'",
"Function.Module._resolveFilename (module.js:338:15)",
"Function.Module._load (module.js:280:25)",
"Module.require (module.js:364:17)",
"require (module.js:380:17)",
"Object. (/tmp/thaumaturgy/node_modules/npm/lib/config/defaults.js:11:13)",
"Module._compile (module.js:456:26)",
"Object.Module._extensions..js (module.js:474:10)",
"Module.load (module.js:356:32)",
"Function.Module._load (module.js:312:12)",
"Module.require (module.js:364:17)",
"",
"ChildProcess.exithandler (child_process.js:658:15)",
"ChildProcess.emit (events.js:98:17)",
"maybeClose (child_process.js:766:16)",
"Process.ChildProcess._handle.onexit (child_process.js:833:5)"
]
}

Chandan Tiwari · Answer 5 · Sun Oct 11 2015 22:46:49 GMT+0800 (China Standard Time)

I am using following build command - thaumaturgy build phantomjs:1.9.18

John Titus · Answer 6 · Mon Oct 12 2015 01:10:00 GMT+0800 (China Standard Time)

Good news - I can recreate the problem when building phantomjs.1.9.18
Bad news - I have no idea what's causing it yet. I suspect it may be because of one of the Troubleshooting issues.

Chandan Tiwari · Answer 7 · Thu Oct 15 2015 02:18:49 GMT+0800 (China Standard Time)

This is also happening with swisseph npm package. I am still wondering what's causing this issue.
However, I went ahead and spun an EC2 instance and compiled the packages. :)

D. C. Rockwell · Answer 8 · Thu Nov 19 2015 12:47:00 GMT+0800 (China Standard Time)

Also having the same problem with:

$ thaumaturgy build temp:0.8.3
Invoking Thaumaturgy Lambda in AWS with packages temp:0.8.3.
Build completed successfully.
Results at: s3://fam.thaumaturgy/thaumaturgy.zip

There is no file in the bucket, and cloudwatch comes up as:

START RequestId: 6af663d4-8e78-11e5-8bef-b90eb8b859e7 Version: $LATEST 
Unable to import module 'index': Error at Function.Module._resolveFilename (module.js:338:15) at Function.Module._load (module.js:280:25) at Module.require (module.js:364:17) at require (module.js:380:17) at Object.<anonymous> (/var/task/node_modules/archiver/lib/core.js:8:13) at Module._compile (module.js:456:26) at Object.Module._extensions..js (module.js:474:10) at Module.load (module.js:356:32) at Function.Module._load (module.js:312:12) at Module.require (module.js:364:17) 
END RequestId: 6af663d4-8e78-11e5-8bef-b90eb8b859e7 
REPORT RequestId: 6af663d4-8e78-11e5-8bef-b90eb8b859e7  Duration: 0.43 ms   Billed Duration: 100 ms Memory Size: 512 MB Max Memory Used: 28 MB

nico · Answer 9 · Tue Dec 08 2015 04:22:44 GMT+0800 (China Standard Time)

I could get this to work by following this steps:

thaumaturgy configure (as usual)
thaumaturgy deploy (as usual)
go to your thaumaturgy lambda function on AWS
select your lambda and actions -> download source code as a zip
uncompress the zip
go inside node_modules/npm
npm i osenv mkdirp
go to the lambda root again
compress it again as a zip
upload it to the thaumaturgy lambda
thaumaturgy build mysql2:0.15.5 --download mysql2.zip

Everything worked and the file is on S3 and also downloaded.
So it seems to just a couple of missing dependencies on the npm inside of it.
@johntitus any hint about how to fix this so I may create a PR for the repo?

nico · Answer 10 · Wed Dec 16 2015 07:06:30 GMT+0800 (China Standard Time)

@johntitus I fixed this is my fork. Let me know if you want a PR.
It was a version conflict... thaumaturgy is using osenv so the cp -r didn't copy that into the lambda context, because it was installed in a upper level.
I also made some other changes and removed npm 3 support (I just didn't make the package yet).

Michael Wills · Answer 11 · Fri Dec 25 2015 04:06:32 GMT+0800 (China Standard Time)

I just tried your fork @nicosommi and I still have the same issue of no zip. I'll try to get in to see what's happening on my end.

D. C. Rockwell · Answer 12 · Tue Mar 08 2016 11:15:30 GMT+0800 (China Standard Time)

@johntitus @chandantiwari @michaelwills @nicosommi,

After much work.. I've finally completed a working alpha of a library that will compile npm packages with native extensions: https://github.com/FreeAllMedia/akiro

I would be very grateful if you could take a look and submit any issues!