[v6] Test OSV data to vers conversion
prabhu opened this issue · comments
Once the CVE schema branch is merged, we need to test the vers conversion logic against the OSV data to reduce any precision loss in transformation.
https://github.com/AppThreat/vulnerability-db/tree/feature/store5
Test scenarios
- If the type is
maven, namespace should not be NULL - If the type is nuget or golang, namespace should be NULL
- If the type is cargo, gem, pypi, npm or composer, namespace can be NULL
- If the name starts with github.com, type must be golang
- If the namespace starts with github.com. type can be generic or swift but not golang
- All values must be lowercase, except for nuget type
Queries to test each condition:
- SELECT COUNT(*) FROM main.cve_data WHERE type == 'maven' and namespace IS NULL
- SELECT COUNT(*) FROM main.cve_data WHERE (type == 'nuget' OR type == 'golang') and namespace IS NOT NULL
- No test is needed as NULL is allowed as a value...?
- SELECT COUNT(*) FROM main.cve_data WHERE name LIKE 'github.com%' AND type != 'go'
- SELECT COUNT(*) FROM main.cve_data WHERE namespace LIKE 'github.com%' AND type != 'swift' AND type != 'generic'
- SELECT COUNT(*) FROM main.cve_data WHERE type LIKE '%[A-Z]%' OR namespace LIKE '%[A-Z]%' OR name LIKE '%[A-Z]%' or source_data LIKE '%[A-Z]%'
In order to pass the test, we would want zero records to be returned for all of these. Unfortunately, that was not the case for any of them.
@cerrussell are you building the database with --cache --only-osv?
@prabhu I built with --cache. If I do --cache-os, it takes an hour to build.
This is good. Will take a look at the issues tomorrow and get them to pass at least for osv.
@prabhu I will try osv only...
@cerrussell 4 is fine since it has a typo. Must be != 'golang'
6 is also fine since the source_data json has some camel cases and upper cases which cannot be changed.