Clair CRDA configuration
adnan-drina opened this issue · comments
Description of Problem / Feature Request
Not able to configure Clair CRDA following documentation https://access.redhat.com/documentation/en-us/red_hat_quay/3.7/html-single/manage_red_hat_quay/index#clair-crda-configuration
Expected Outcome
Java vulnerability scanning enabled
Actual Outcome
Clair throws errors
{"level":"info","component":"crda/MatcherFactory.Configure","key":"9e7da76708fe374d8c10fa7xxxxxxxxx","time":"2022-05-27T06:49:09Z","message":"configured API key"}
{"level":"info","component":"crda/MatcherFactory.Matcher","time":"2022-05-27T06:49:09Z","message":"using default ecosystems"}
{"level":"info","component":"libvuln/New","matchers":[{"name":"debian-matcher","docs":"https://pkg.go.dev/github.com/quay/claircore/debian"},{"name":"python","docs":"https://pkg.go.dev/github.com/quay/claircore/python"},{"name":"rhel","docs":"https://pkg.go.dev/github.com/quay/claircore/rhel"},{"name":"alpine-matcher","docs":"https://pkg.go.dev/github.com/quay/claircore/alpine"},{"name":"aws-matcher","docs":"https://pkg.go.dev/github.com/quay/claircore/aws"},{"name":"suse","docs":"https://pkg.go.dev/github.com/quay/claircore/suse"},{"name":"ubuntu-matcher","docs":"https://pkg.go.dev/github.com/quay/claircore/ubuntu"},{"name":"photon","docs":"https://pkg.go.dev/github.com/quay/claircore/photon"},{"name":"crda-pypi","docs":"https://pkg.go.dev/github.com/quay/claircore/crda"},{"name":"crda-maven","docs":"https://pkg.go.dev/github.com/quay/claircore/crda"},{"name":"oracle","docs":"https://pkg.go.dev/github.com/quay/claircore/oracle"}],"time":"2022-05-27T06:49:09Z","message":"matchers created"}
{"level":"error","component":"crda/Matcher.QueryRemoteMatcher","matcher":"crda-pypi","error":"invalid character 'A' looking for beginning of value","time":"2022-05-27T06:49:33Z","message":"remote api call failure"}
{"level":"error","matcher":"crda-maven","component":"crda/Matcher.QueryRemoteMatcher","error":"invalid character 'A' looking for beginning of value","time":"2022-05-27T06:49:33Z","message":"remote api call failure"}
{"level":"error","matcher":"crda-maven","component":"crda/Matcher.QueryRemoteMatcher","error":"invalid character 'A' looking for beginning of value","time":"2022-05-27T06:49:33Z","message":"remote api call failure"}
{"level":"error","matcher":"crda-maven","component":"crda/Matcher.QueryRemoteMatcher","error":"invalid character 'A' looking for beginning of value","time":"2022-05-27T06:49:33Z","message":"remote api call failure"}
{"level":"error","matcher":"crda-maven","component":"crda/Matcher.QueryRemoteMatcher","error":"invalid character 'A' looking for beginning of value","time":"2022-05-27T06:49:33Z","message":"remote api call failure"}
{"level":"error","matcher":"crda-maven","component":"crda/Matcher.QueryRemoteMatcher","error":"invalid character 'A' looking for beginning of value","time":"2022-05-27T06:49:33Z","message":"remote api call failure"}
{"level":"error","component":"crda/Matcher.QueryRemoteMatcher","matcher":"crda-maven","error":"invalid character 'A' looking for beginning of value","time":"2022-05-27T06:49:33Z","message":"remote api call failure"}
{"level":"error","matcher":"crda-maven","component":"crda/Matcher.QueryRemoteMatcher","error":"invalid character 'A' looking for beginning of value","time":"2022-05-27T06:49:33Z","message":"remote api call failure"}
{"level":"error","matcher":"crda-maven","component":"crda/Matcher.QueryRemoteMatcher","error":"invalid character 'A' looking for beginning of value","time":"2022-05-27T06:49:33Z","message":"remote api call failure"}
{"level":"error","matcher":"crda-maven","component":"crda/Matcher.QueryRemoteMatcher","error":"invalid character 'A' looking for beginning of value","time":"2022-05-27T06:49:33Z","message":"remote api call failure"}
Environment
OpenShift 4.10.14
Quay Operator 3.7.0
registry.redhat.io/quay/clair-rhel8@sha256:86d45aaf6f783f119d9ed9acf9fc962b037564b32ba2b68e9075282f7e1d6e5b
clair-config.yaml
...
matchers:
config:
crda:
url: https://gw.api.openshift.io/api/v2/
source: quay.io
key: 10fcd9b01603d57e6887a4349xxxxxxx
...
I'm presuming the A is coming from the CRDA response body Authentication failed, this likely means that the key is incorrect, or for some reason there was a CRDA error when creating the key after filling out the form. Can you double check the key then we can discount that?
I have double-checked my user_key, and with the curl command API returns a response. So, the user_key doesn't seem to be a problem. I think that the problem is rather the API request format coming from Clair.
curl --location --request POST 'https://gw.api.openshift.io:443/api/v2/vulnerability-analysis?user_key=10fcd9b01603d57e6887a4349xxxxxxx' \
--header 'Content-Type: application/json' \
--data-raw '{
"ecosystem": "maven",
"package_versions": [
{"package": "com.netflix.ribbon:ribbon-eureka", "version": "2.3.0"},
{"package": "io.undertow:undertow-core", "version": "2.2.2.Final"},
{"package": "org.apache.xmlbeans:xmlbeans", "version": "3.0.1"},
{"package": "com.google.code.gson:gson", "version": "2.8.6"},
{"package": "commons-logging:commons-logging", "version": "1.1.1"},
{"package": "org.jfree:jcommon", "version": "1.0.23"},
{"package": "com.ongres.scram:client", "version": "2.1"},
{"package": "org.springframework.cloud:spring-cloud-starter-netflix-archaius", "version": "2.2.2.RELEASE"},
{"package": "org.ow2.asm:asm", "version": "5.0.4"},
{"package": "javax.xml.bind:jaxb-api", "version": "2.3.1"}
]
}'
[
{
"name": "com.netflix.ribbon:ribbon-eureka",
"version": "2.3.0",
"vulnerabilities": []
},
{
"name": "io.undertow:undertow-core",
"version": "2.2.2.Final",
"vulnerabilities": [
{
"fixed_in": [
"2.2.8.Final"
],
"id": "SNYK-JAVA-IOUNDERTOW-1304915",
"severity": "medium",
"title": "Denial of Service (DoS)",
"url": "https://snyk.io/vuln/SNYK-JAVA-IOUNDERTOW-1304915"
},
{
"fixed_in": [
"2.2.15.Final"
],
"id": "SNYK-JAVA-IOUNDERTOW-2391283",
"severity": "high",
"title": "Denial of Service (DoS)",
"url": "https://snyk.io/vuln/SNYK-JAVA-IOUNDERTOW-2391283"
}
]
},
{
"name": "org.apache.xmlbeans:xmlbeans",
"version": "3.0.1",
"vulnerabilities": []
},
{
"name": "com.google.code.gson:gson",
"version": "2.8.6",
"vulnerabilities": [
{
"fixed_in": [
"2.8.9"
],
"id": "SNYK-JAVA-COMGOOGLECODEGSON-1730327",
"severity": "high",
"title": "Deserialization of Untrusted Data",
"url": "https://snyk.io/vuln/SNYK-JAVA-COMGOOGLECODEGSON-1730327"
}
]
},
{
"name": "commons-logging:commons-logging",
"version": "1.1.1",
"vulnerabilities": []
},
{
"name": "org.jfree:jcommon",
"version": "1.0.23",
"vulnerabilities": []
},
{
"name": "com.ongres.scram:client",
"version": "2.1",
"vulnerabilities": []
},
{
"name": "org.springframework.cloud:spring-cloud-starter-netflix-archaius",
"version": "2.2.2.RELEASE",
"vulnerabilities": []
},
{
"name": "org.ow2.asm:asm",
"version": "5.0.4",
"vulnerabilities": []
},
{
"name": "javax.xml.bind:jaxb-api",
"version": "2.3.1",
"vulnerabilities": []
}
]
currently, my clair config is as follows:
auth:
psk:
iss:
- quay
- clairctl
key: OVAyQmNYWmlEYUM1SURJa0RCYXoza0tTOTYyNUphc2s=
http_listen_addr: :8080
indexer:
connstring: host=demo-registry-clair-postgres port=5432 dbname=postgres user=postgres password=postgres sslmode=disable
layer_scan_concurrency: 5
migrations: true
scanlock_retry: 10
log_level: info
matcher:
connstring: host=demo-registry-clair-postgres port=5432 dbname=postgres user=postgres password=postgres sslmode=disable
max_conn_pool: 100
migrations: true
matchers:
config:
crda:
key: 10fcd9b01603d57e6887a4349xxxxxx
source: quay.io
url: https://gw.api.openshift.io/api/v2/
metrics:
name: prometheus
notifier:
connstring: host=demo-registry-clair-postgres port=5432 dbname=postgres user=postgres password=postgres sslmode=disable
delivery_interval: 1m0s
migrations: true
poll_interval: 5m0s
webhook:
callback: http://demo-registry-clair-app/notifier/api/v1/notifications
target: https://demo-registry-quay-quay.apps.cluster-wx7z7.wx7z7.sandbox1084.opentlc.com/secscan/notification
and the error message has changes slightly:
{"level":"error","matcher":"crda-maven","component":"crda/Matcher.QueryRemoteMatcher","error":"invalid character '<' looking for beginning of value","time":"2022-05-27T14:08:40Z","message":"remote api call failure"}
{"level":"error","matcher":"crda-maven","component":"crda/Matcher.QueryRemoteMatcher","error":"invalid character '<' looking for beginning of value","time":"2022-05-27T14:08:40Z","message":"remote api call failure"}
{"level":"error","matcher":"crda-maven","component":"crda/Matcher.QueryRemoteMatcher","error":"invalid character '<' looking for beginning of value","time":"2022-05-27T14:08:40Z","message":"remote api call failure"}
{"level":"error","matcher":"crda-maven","component":"crda/Matcher.QueryRemoteMatcher","error":"invalid character '<' looking for beginning of value","time":"2022-05-27T14:08:40Z","message":"remote api call failure"}
{"level":"error","matcher":"crda-maven","component":"crda/Matcher.QueryRemoteMatcher","error":"invalid character '<' looking for beginning of value","time":"2022-05-27T14:08:40Z","message":"remote api call failure"}
{"level":"error","matcher":"crda-maven","component":"crda/Matcher.QueryRemoteMatcher","error":"invalid character '<' looking for beginning of value","time":"2022-05-27T14:08:40Z","message":"remote api call failure"}
{"level":"error","matcher":"crda-maven","component":"crda/Matcher.QueryRemoteMatcher","error":"invalid character '<' looking for beginning of value","time":"2022-05-27T14:08:40Z","message":"remote api call failure"}
{"level":"error","matcher":"crda-maven","component":"crda/Matcher.QueryRemoteMatcher","error":"invalid character '<' looking for beginning of value","time":"2022-05-27T14:08:40Z","message":"remote api call failure"}
{"level":"error","matcher":"crda-maven","component":"crda/Matcher.QueryRemoteMatcher","error":"invalid character '<' looking for beginning of value","time":"2022-05-27T14:08:40Z","message":"remote api call failure"}
{"level":"error","matcher":"crda-pypi","component":"crda/Matcher.QueryRemoteMatcher","error":"invalid character '<' looking for beginning of value","time":"2022-05-27T14:08:40Z","message":"remote api call failure"}
{"level":"error","matcher":"crda-maven","component":"crda/Matcher.QueryRemoteMatcher","error":"invalid character '<' looking for beginning of value","time":"2022-05-27T14:08:40Z","message":"remote api call failure"}
You can try rebuilding clair with claircore@main (i.e. including c854b78) to see why the CRDA API is complaining.
Please run 4.4.3 and see what sort of errors are being reported.
after changing Clair version to 4.4.3 I'm getting indeed a new error message
{"level":"error","component":"crda/Matcher.QueryRemoteMatcher","matcher":"crda-maven","error":"reported error: \"405 Method Not Allowed\" (body: \"<!DOCTYPE HTML PUBLIC \\\"-//W3C//DTD HTML 3.2 Final//EN\\\">\\n<title>405 Method Not Allowed</title>\\n<h1>Method Not Allowed</h1>\\n<p>The method is not allowed for the requested URL.</p>\\n\")","time":"2022-06-08T09:12:35Z","message":"remote api call failure"}
{"level":"error","matcher":"crda-pypi","component":"crda/Matcher.QueryRemoteMatcher","error":"reported error: \"405 Method Not Allowed\" (body: \"<!DOCTYPE HTML PUBLIC \\\"-//W3C//DTD HTML 3.2 Final//EN\\\">\\n<title>405 Method Not Allowed</title>\\n<h1>Method Not Allowed</h1>\\n<p>The method is not allowed for the requested URL.</p>\\n\")","time":"2022-06-08T09:12:35Z","message":"remote api call failure"}
{"level":"error","matcher":"crda-maven","component":"crda/Matcher.QueryRemoteMatcher","error":"reported error: \"405 Method Not Allowed\" (body: \"<!DOCTYPE HTML PUBLIC \\\"-//W3C//DTD HTML 3.2 Final//EN\\\">\\n<title>405 Method Not Allowed</title>\\n<h1>Method Not Allowed</h1>\\n<p>The method is not allowed for the requested URL.</p>\\n\")","time":"2022-06-08T09:12:35Z","message":"remote api call failure"}
{"level":"error","matcher":"crda-maven","component":"crda/Matcher.QueryRemoteMatcher","error":"reported error: \"405 Method Not Allowed\" (body: \"<!DOCTYPE HTML PUBLIC \\\"-//W3C//DTD HTML 3.2 Final//EN\\\">\\n<title>405 Method Not Allowed</title>\\n<h1>Method Not Allowed</h1>\\n<p>The method is not allowed for the requested URL.</p>\\n\")","time":"2022-06-08T09:12:35Z","message":"remote api call failure"}
{"level":"error","matcher":"crda-maven","component":"crda/Matcher.QueryRemoteMatcher","error":"reported error: \"405 Method Not Allowed\" (body: \"<!DOCTYPE HTML PUBLIC \\\"-//W3C//DTD HTML 3.2 Final//EN\\\">\\n<title>405 Method Not Allowed</title>\\n<h1>Method Not Allowed</h1>\\n<p>The method is not allowed for the requested URL.</p>\\n\")","time":"2022-06-08T09:12:35Z","message":"remote api call failure"}
{"level":"error","matcher":"crda-maven","component":"crda/Matcher.QueryRemoteMatcher","error":"reported error: \"405 Method Not Allowed\" (body: \"<!DOCTYPE HTML PUBLIC \\\"-//W3C//DTD HTML 3.2 Final//EN\\\">\\n<title>405 Method Not Allowed</title>\\n<h1>Method Not Allowed</h1>\\n<p>The method is not allowed for the requested URL.</p>\\n\")","time":"2022-06-08T09:12:35Z","message":"remote api call failure"}
{"level":"error","matcher":"crda-maven","component":"crda/Matcher.QueryRemoteMatcher","error":"reported error: \"405 Method Not Allowed\" (body: \"<!DOCTYPE HTML PUBLIC \\\"-//W3C//DTD HTML 3.2 Final//EN\\\">\\n<title>405 Method Not Allowed</title>\\n<h1>Method Not Allowed</h1>\\n<p>The method is not allowed for the requested URL.</p>\\n\")","time":"2022-06-08T09:12:35Z","message":"remote api call failure"}
{"level":"error","matcher":"crda-maven","component":"crda/Matcher.QueryRemoteMatcher","error":"reported error: \"405 Method Not Allowed\" (body: \"<!DOCTYPE HTML PUBLIC \\\"-//W3C//DTD HTML 3.2 Final//EN\\\">\\n<title>405 Method Not Allowed</title>\\n<h1>Method Not Allowed</h1>\\n<p>The method is not allowed for the requested URL.</p>\\n\")","time":"2022-06-08T09:12:35Z","message":"remote api call failure"}
I don't think there's much more we can do, here. I think you'll need to contact the CRDA team/operators for any additional troubleshooting.
We're using the correct method (see here) so I suspect the API is upset for some other reason and not reporting that correctly.
Thanks a lot for pointing me to the correct method. After omitting URL and source values from my Clair config everything works as expected.
My final Clair config looks like this:
matchers:
config:
crda:
key: 10fcd9b01603d57e6887a4349xxxxxxx