Suddenly the --incognito flag seems not to work anymore with cnquery scan

Question

Suddenly the --incognito flag seems not to work anymore with cnquery scan

michaelkrieg opened this issue 3 months ago · comments

Describe the bug
cnquery scan aws does not work anymore, with or without specifying a querypack.

To Reproduce
Steps to reproduce the behavior:

For AWS, gain AWS credentials first. Then, with or without an optional querypack:
cnquery scan aws --incognito --querypack mondoo-incident-response-aws

Expected behavior
The output should equal to the same command executed without the --incognito flag.

Screenshots or CLI Output

→ found a new version for 'aws' provider installed=10.2.7 latest=10.2.8
→ successfully installed aws provider path=/Users/hidden/.config/mondoo/providers/aws version=10.2.8
→ loaded configuration from /Users/hidden/.config/mondoo/mondoo.yml using source default
→ discover related assets for 1 asset(s)

 AWS Account hidden (123456789012) ───────────────────────────────────────────────────────────────────────────────────────────────────────────────    X


Summary (1 assets)
==================

Target:     AWS Account hidden (123456789012)
error: cannot find any policy for this search

Desktop (please complete the following information):

OS: macOS
OS Version: 14.2.1
cnquery version
cnquery 10.3.4 (f380756, 2024-02-17T08:12:07Z)

Additional context
This has been working until recently. Even cnquery scan local --incognito results in the same error as shown above.

Michael Krieg · Answer 1 · Wed Mar 13 2024 20:48:57 GMT+0800 (China Standard Time)

still the error occurs after the recent update.

cnquery scan aws --incognito --querypack mondoo-incident-response-aws --verbose
DBG using provider aws with connector aws
DBG Started a new runtime (1 total)
DBG performing request method=GET url={"ForceQuery":false,"Fragment":"","Host":"releases.mondoo.com","OmitHost":false,"Opaque":"","Path":"/providers/latest.json","RawFragment":"","RawPath":"","RawQuery":"","Scheme":"https","User":null}
→ found a new version for 'aws' provider installed=10.3.0 latest=10.3.2
DBG installing provider from URL url=https://releases.mondoo.com/providers/aws/10.3.2/aws_10.3.2_darwin_arm64.tar.xz
DBG performing request method=GET url={"ForceQuery":false,"Fragment":"","Host":"releases.mondoo.com","OmitHost":false,"Opaque":"","Path":"/providers/aws/10.3.2/aws_10.3.2_darwin_arm64.tar.xz","RawFragment":"","RawPath":"","RawQuery":"","Scheme":"https","User":null}
DBG create temp directory to unpack providers
DBG unpacking providers path=/Users/MYUSEER/.config/mondoo/providers/.providers-unpack58817101
DBG unpacking file dest=/Users/MYUSEER/.config/mondoo/providers/.providers-unpack58817101/aws name=aws
DBG unpacking file dest=/Users/MYUSEER/.config/mondoo/providers/.providers-unpack58817101/aws.json name=aws.json
DBG unpacking file dest=/Users/MYUSEER/.config/mondoo/providers/.providers-unpack58817101/aws.resources.json name=aws.resources.json
DBG move provider to destination
DBG move provider binary dst=/Users/MYUSEER/.config/mondoo/providers/aws/aws src=/Users/MYUSEER/.config/mondoo/providers/.providers-unpack58817101/aws
DBG loading providers
→ successfully installed aws provider path=/Users/MYUSEER/.config/mondoo/providers/aws version=10.3.2
DBG Log level set to debug
DBG Started a new runtime (2 total)
→ no Mondoo configuration file provided, using defaults
! No credentials provided. Switching to --incognito mode.
→ discover related assets for 1 asset(s)
DBG Started a new runtime (3 total)
DBG new aws connection
DBG Started a new runtime (4 total)
DBG new aws connection
DBG connecting to asset AWS Account 123456789012 (AWS Account)
DBG searching providers in path path=/Users/MYUSEER/.config/mondoo/providers
DBG starting query execution qrid=HIDDEN
DBG finished query execution qrid=HIDDEN
DBG HIDDEN finished
DBG graph has received all datapoints
DBG non-ok http request body="cannot find any policy for this search" status=500
DBG could not scan asset error="rpc error: code = Unknown desc = cannot find any policy for this search" asset="AWS Account 123456789012"
DBG shutting down unused provider aws
Summary (1 assets)
==================

Target:     AWS Account 123456789012
error: cannot find any policy for this search

Even if I move my default mondoo.yml configuration file somewhere else, the error remains the same.

Christoph Hartmann · Answer 2 · Fri Mar 15 2024 04:17:33 GMT+0800 (China Standard Time)

We are going to roll out a complete new backend for the public content registry. This is going to happen within the next week. Until then I recommend to clone https://github.com/mondoohq/cnquery-packs and run the policies from there.

Christoph Hartmann · Answer 3 · Mon Apr 29 2024 16:29:02 GMT+0800 (China Standard Time)

We completely updated the server-side fetching of the policies. I expect everything to be back to normal now. Please re-open if you still encounter issues.

Michael Krieg · Answer 4 · Mon Apr 29 2024 20:09:12 GMT+0800 (China Standard Time)

Thanks again for your effort!

But this still does not work, even without specifying a querypack.

cnquery scan aws --incognito --querypack mondoo-incident-response-aws --verbose
## redacted. ###
FTL failed to run scan error="all available packs filtered out. nothing to do"

cnquery scan aws --incognito  --verbose


DBG using provider aws with connector aws
DBG Started a new runtime (1 total)
DBG no need to update provider last-refresh=3m24.509994s provider=aws
DBG Log level set to debug
DBG Started a new runtime (2 total)
→ loaded configuration from /Users/michaelkrieg/.config/mondoo/mondoo.yml using source default
→ using service account credentials
→ discover related assets for 1 asset(s)
DBG Started a new runtime (3 total)
DBG new aws connection
DBG performing request method=POST url={"ForceQuery":false,"Fragment":"","Host":"sts.eu-central-1.amazonaws.com","OmitHost":false,"Opaque":"","Path":"/","RawFragment":"","RawPath":"/","RawQuery":"","Scheme":"https","User":null}
DBG performing request method=POST url={"ForceQuery":false,"Fragment":"","Host":"iam.amazonaws.com","OmitHost":false,"Opaque":"","Path":"/","RawFragment":"","RawPath":"/","RawQuery":"","Scheme":"https","User":null}
DBG Started a new runtime (4 total)
DBG new aws connection
DBG performing request method=POST url={"ForceQuery":false,"Fragment":"","Host":"sts.eu-central-1.amazonaws.com","OmitHost":false,"Opaque":"","Path":"/","RawFragment":"","RawPath":"/","RawQuery":"","Scheme":"https","User":null}
DBG initialize client authentication issuer=mondoo/ams kid=//agents.api.mondoo.app/spaces/eu-##REDACTED##-jackson-##REDACTED##/serviceaccounts/##REDACTED##
DBG connecting to asset AWS Account ##REDACTED## (AWS Account)
DBG could not scan asset error="failed to compile filters for query //policy.api.mondoo.app/queries/asset-overview-azure-creator\nfailed to compile filters for query //policy.api.mondoo.app/queries/asset-overview-azure-owner\nfailed to compile filters for query //policy.api.mondoo.app/queries/gcp-compute-instances-configured-use-default-service-account-full-access-all-cloud-single\nfailed to compile filters for query //policy.api.mondoo.app/queries/gcp-compute-instances-configured-use-default-service-account-single\nfailed to compile filters for query //policy.api.mondoo.app/queries/mondoo-azure-security-sql-server-tde-on-single" asset="AWS Account ##REDACTED##"
DBG shutting down unused provider aws
Data (1 assets)
===============

Asset: AWS Account ##REDACTED##
-------------------------------

error: failed to compile filters for query //policy.api.mondoo.app/queries/asset-overview-azure-creator
failed to compile filters for query //policy.api.mondoo.app/queries/asset-overview-azure-owner
failed to compile filters for query //policy.api.mondoo.app/queries/gcp-compute-instances-configured-use-default-service-account-full-access-all-cloud-single
failed to compile filters for query //policy.api.mondoo.app/queries/gcp-compute-instances-configured-use-default-service-account-single
failed to compile filters for query //policy.api.mondoo.app/queries/mondoo-azure-security-sql-server-tde-on-single

Summary (1 assets)
==================

Target:     AWS Account ##REDACTED##
error: failed to compile filters for query //policy.api.mondoo.app/queries/asset-overview-azure-creator
failed to compile filters for query //policy.api.mondoo.app/queries/asset-overview-azure-owner
failed to compile filters for query //policy.api.mondoo.app/queries/gcp-compute-instances-configured-use-default-service-account-full-access-all-cloud-single
failed to compile filters for query //policy.api.mondoo.app/queries/gcp-compute-instances-configured-use-default-service-account-single
failed to compile filters for query //policy.api.mondoo.app/queries/mondoo-azure-security-sql-server-tde-on-single

Also, I am curious about mentioning "azure" here in the verbose output?!