No cache data - Error: unexpected EOF
Techbrunch opened this issue · comments
Description of Bug
When running: cloudfox aws instances -c I get the following error:
[🦊 cloudfox v1.12.2 🦊 ] AWS Caller Identity: arn:aws:iam::XXXXXXXXXXXX:user/pentest-user-2023
[🦊 cloudfox v1.12.2 🦊 ] No cache data for XXXXXXXXXXXX. Error: unexpected EOF
[🦊 cloudfox v1.12.2 🦊 ] Account is not part of an Organization
Here is the error file:
❯ cat /Users/XXX/.cloudfox/cloudfox-error.log
time="2023-09-19T11:30:07+02:00" level=error msg="Could not decode the following file: /Users/XXX/.cloudfox/cached-data/aws/XXXXXXXXXXXX/XXXXXXXXXXXX-apigatewayv2-GetApis-ap-northeast-1.json"
time="2023-09-19T11:30:08+02:00" level=error msg="Failed to describe organization: operation error Organizations: DescribeOrganization, https response error StatusCode: 400, RequestID: 8b38ad4f-db5f-4404-809a-b4c232829860, AWSOrganizationsNotInUseException: Your account is not a member of an organization."
❯ cat /Users/XXX/.cloudfox/cached-data/aws/XXXXXXXXXXXX/XXXXXXXXXXXX-apigatewayv2-GetApis-ap-northeast-1.json
{
"Value": null,
"Exp": 0
}
What should the expected behavior be
It should list the instances from cache.
Note: The error is the same for all commands
Good catch, thank you @Techbrunch! I had to make some changes in v1.12.1&2 affecting the way cloudfox writes to the filesystem in order to pass the brew tests in the brew test sandbox, and i bet that messed up the cache functionality -- I'll look into this!
Ok. I think it was the json files (as you showed in the output below) rather than the directory updates for brew. Can you checkout the seth-dev branch, and try again? You can either build it from seth-dev first and run the executable, or from that branch you can do a go run main.go aws buckets -c and see if the error is fixed.
Thanks!
It works now but for some reasons it takes 33seconds to run: go run main.go aws buckets -c -v2, is that normal ?
When you do go run it is building it behind the scenes and downloading all of the imported modules. if you run it additional times without making any code changes it should be faster... at least thats my understanding.
It still takes around 30 seconds after building it.
I have the following errors in cloudfox-error.log:
1 error
Failed to describe organization: operation error Organizations: DescribeOrganization, https response error StatusCode: 400, RequestID: 9f044575-a29e-4d52-8894-1fe6f3602d94, AWSOrganizationsNotInUseException: Your account is not a member of an organization."
50 errors
time="2023-09-20T20:32:01+02:00" level=error msg="operation error S3: GetBucketPolicy, https response error StatusCode: 404, RequestID: TS597FN0BT3312PR, HostID: hHGWE3OPa4e0/LG5muoPPc2HfTxVcD2gzMWHwhjzmqz93QNLUrbA7Fs/JAEwCEf/QPN3VluRGNI=, api error NoSuchBucketPolicy: The bucket policy does not exist" module=buckets
time="2023-09-20T20:32:01+02:00" level=error msg="parsing bucket access policy () as JSON: unmarshal JSON: unexpected end of JSON input" module=buckets
So clearly cloudfox is doing a lot of HTTP requests instead of using the cache.
Also when using -c I would remove the call to AWS Caller Identity, it looks like this adds at least 2 HTTP requests.
@Techbrunch - Pretty sure I fixed the issue, thanks for digging into it. Basically for buckets that don't have a policy, it was not creating a stubbed cached entry, which means next run even in cached mode it would make the sdk call again. i changed hte logic to create an empty cache entry file so it skips the bucket policy lookup when run in -c mode.
It's in seth-dev right now if you want to test it to confirm.
Sorry but the engagement is over so I won't be able to test it in real condition until the next one.