Demo troubles
ludovicc opened this issue · comments
Hello,
I've been trying gg, and found those issues:
- What are the access control settings that should be used for the S3 bucket? I've been using at first 'Block public access', but got a forbidden on upload_files() operation. Then I made the bucket fully writable for the world (bad bad practice), and I could get around this issue.
gg force --jobs 100 --engine lambda src/frontend/mosh-server
→ Loading the thunks... done (16 ms).
↗ Uploading 489 files (36.4 MiB)... terminate called after throwing an instance of 'std::runtime_error'
what(): HTTP failure in S3Client::upload_files(): HTTP/1.1 403 Forbidden
Abandon (core dumped)
- Compilation of mosh using gg failed with the following message:
gg force --jobs 100 --engine lambda src/frontend/mosh-server
→ Loading the thunks... done (3 ms).
↗ Uploading 489 files (36.4 MiB)... done (2355 ms).
ld: cannot find Scrt1.o: No such file or directory
ld: cannot find crti.o: No such file or directory
ld: cannot find crtbeginS.o: No such file or directory
ld: cannot find -ltinfo
ld: cannot find -lprotobuf
ld: cannot find -lssl
ld: cannot find -lcrypto
ld: cannot find -lutil
ld: cannot find -lz
ld: cannot find -lutempter
ld: cannot find -lstdc++
ld: cannot find -lm
ld: cannot find -lgcc_s
ld: cannot find -lgcc
ld: cannot find -lpthread
ld: cannot find -lc
ld: cannot find -lgcc_s
ld: cannot find -lgcc
ld: cannot find crtendS.o: No such file or directory
ld: cannot find crtn.o: No such file or directory
rmdir /tmp/thunk-execute.fb0WSr: Directory not empty
std::exception
`TZJokLWuLw23YLba.mIh.m26Qoc.AU8BG0qEvx2DyFAk00000903': process exited with failure status 1
gg-force: execution failed: TZJokLWuLw23YLba.mIh.m26Qoc.AU8BG0qEvx2DyFAk00000903
Thanks, Ludovic
Hi Ludovic,
-
It's not necessary to make the bucket fully writable -- just make sure that the IAM user (the one associated with your
AWS_ACCESS_KEY_ID
), has AmazonS3FullAccess permission. -
Could you please run
gg describe TZJokLWuLw23YLba.mIh.m26Qoc.AU8BG0qEvx2DyFAk00000903
and post the output here, so I can take a look at the thunk that fails?
Thank you,
Sadjad
gg describe TZJokLWuLw23YLba.mIh.m26Qoc.AU8BG0qEvx2DyFAk00000903
{
"function": {
"hash": "VYA7BN_Oi7TEF.SFqo2yJu2fVpJOGPyeu5ThcID2g86400123508",
"args": [
"/__gg__/g++",
"-L/usr/lib/gcc/x86_64-linux-gnu/7",
"-L/usr/lib/x86_64-linux-gnu",
"-L/usr/lib",
"-L/lib/x86_64-linux-gnu",
"-L/lib",
"-L/usr/lib/x86_64-linux-gnu",
"-L/usr/lib",
"-L/usr/lib",
"-L/lib",
"-L/usr/lib",
"-Wall",
"-fno-strict-overflow",
"-D_FORTIFY_SOURCE=2",
"-fstack-protector-all",
"-Wstack-protector",
"--param",
"ssp-buffer-size=1",
"-fPIE",
"-fno-default-inline",
"-pipe",
"-g",
"-O2",
"-pie",
"-Wl,-z,relro",
"-Wl,-z,now",
"mosh-server.o",
"../crypto/libmoshcrypto.a",
"../network/libmoshnetwork.a",
"../statesync/libmoshstatesync.a",
"../terminal/libmoshterminal.a",
"../util/libmoshutil.a",
"../protobufs/libmoshprotos.a",
"-lm",
"-ltinfo",
"-lprotobuf",
"-pthread",
"-lssl",
"-lcrypto",
"-lutil",
"-lz",
"-lutempter",
"-o",
"mosh-server",
"-B/usr/lib/gcc/x86_64-linux-gnu/7",
"-Wl,-rpath-link,/usr/local/lib/x86_64-linux-gnu",
"-Wl,-rpath-link,/lib/x86_64-linux-gnu",
"-Wl,-rpath-link,/usr/lib/x86_64-linux-gnu",
"-Wl,-rpath-link,/usr/lib/x86_64-linux-gnu64",
"-Wl,-rpath-link,/usr/local/lib64",
"-Wl,-rpath-link,/lib64",
"-Wl,-rpath-link,/usr/lib64",
"-Wl,-rpath-link,/usr/local/lib",
"-Wl,-rpath-link,/lib",
"-Wl,-rpath-link,/usr/lib",
"-Wl,-rpath-link,/usr/x86_64-linux-gnu/lib64",
"-Wl,-rpath-link,/usr/x86_64-linux-gnu/lib"
],
"envars": [
"PATH=/__gg__",
"GG_MANIFEST=@{GGHASH:VpwFAs0E9NnK6ue6EZs101prO_E0b2khFREMzHvw6eW0000001dd}"
]
},
"values": [
"V0QgOTbGhxmrvSY.nijd4Qv8dsQTYlbRrO5RVZjLa1J4000b8844=../network/libmoshnetwork.a",
"VCz5TNruI5cahljRJ3Vn7XpgTJCtxDOV_mAWbiQ3njb00001e0ea=../util/libmoshutil.a",
"VLRlryGMuUAMW.g4MPo9A0OfahTrpQIBQgLNJLOA7e1E00029bd0=/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2",
"VMBZ2UStz1OTCOwHbRr_GRN3zjGl9Bc9AZLdIKttYLds003e1fb6=../terminal/libmoshterminal.a",
"VQ.jIFc5GDDsyDGF7nhfBCULpw9mM5p2u3P54o8c6Tf80012a4ec=../protobufs/libmoshprotos.a",
"VTXmesrNEseLNA39uYrG0KuvIKH.29iPT14Yycdf1Xs80003c4b4=../crypto/libmoshcrypto.a",
"VY8uGTaz1V.7gBy_LkiMDGdYQZbFfnLBrKdLQ4ckVMsQ000038e0=/lib/x86_64-linux-gnu/libdl.so.2",
"VhOaEkqywgV4LiDRCVVTD2pUqVvlrYHPxbNb2tVvNGuc0019e030=mosh-server.o",
"VpwFAs0E9NnK6ue6EZs101prO_E0b2khFREMzHvw6eW0000001dd",
"Vva5lVMtscMO9jmybxF9W0FVsAsHAt2QSXcDIkxaNKI4000a4ef4=../statesync/libmoshstatesync.a"
],
"thunks": [],
"executables": [
"VYA7BN_Oi7TEF.SFqo2yJu2fVpJOGPyeu5ThcID2g86400123508=/__gg__/g++",
"VeDb5H5mtTk1vFTGivc2k7K_In2JD5Mbw_Z6VI8ftisg0022b5f8=/__gg__/ld",
"VwfFvtNgbE1OEXyGh3w0L.U6FWKKg2hHtWrX3DR.IPKw000bca88=/__gg__/collect2"
],
"outputs": [
"output"
],
"timeout": 5000
}
Hi @sadjad ,
same issue here: The user as well as the GG_LAMBDA_ROLE have "AmazonS3FullAccess". Only if I make the bucket public, will the demo (building mosh) work. Otherwise, I get a 403 error as reported above.
Setup: Ubuntu 18.04 instance created on AWS, zone us-west-1.
Many thanks for this awesome project!
@ludovicc , I can restrict it a little bit. If I only disable "Block public access to buckets and objects granted through new access control lists (ACLs)", then it works fine.
This did not help, but provides more details: https://aws.amazon.com/premiumsupport/knowledge-center/lambda-execution-role-s3-bucket/
You don't need to make the bucket public. The IAM policy that I assigned to the role used by gg if it helps:
{
"Version": "<version_date>",
"Statement": [
{
"Effect": "Allow",
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::<bucket_name>",
"arn:aws:s3:::<bucket_name>/*"
]
},
{
"Effect": "Allow",
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": [
"arn:aws:logs:<region>:<id>:log-group:/aws/lambda/gg-lambda-function",
"arn:aws:logs:<region>:<id>:log-group:/aws/lambda/gg-lambda-function:*"
]
}
]
}
I believe you're getting the 403 because you have to give the List Action permission on the bucket itself, not just bucket/*.