Not detecting NPM dependencies
atelic opened this issue · comments
I'm trying to run Hub Detect on a project with bothnpm-shrinkwrap.json and package.json files present in the root of the directory. Instead of the usual logging on successful scans, it runs, detects the OS, then closes out.
Interestingly, when running the scanner on a project with just a package.json, it detects the correct configuration and successfully scans. Any idea why this would be happening?
Logs:
[10:18:16]Step 2/2: Run Black Duck Hub Detect (Command Line) (4s)
[10:18:16][Step 2/2] Starting: /home/buildagent/buildAgent/temp/agentTmp/custom_script6823763653112255195
[10:18:16][Step 2/2] in directory: /home/buildagent/buildAgent/work/d3f4ccac47328f58
[10:18:16][Step 2/2] % Total % Received % Xferd Average Speed Time Time Time Current
[10:18:16][Step 2/2] Dload Upload Total Spent Left Speed
[10:18:16][Step 2/2] 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
[10:18:16][Step 2/2] 100 41 100 41 0 0 545 0 --:--:-- --:--:-- --:--:-- 546
[10:18:16][Step 2/2] will look for release: hub-detect-0.0.5.jar
[10:18:16][Step 2/2] You have already downloaded the latest file, so the local file will be used.
[10:18:16][Step 2/2] running detect: java -jar /tmp/hub-detect-0.0.5.jar
[10:18:17][Step 2/2]
[10:18:17][Step 2/2] . ____ _ __ _ _
[10:18:17][Step 2/2] /\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
[10:18:17][Step 2/2] ( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
[10:18:17][Step 2/2] \\/ ___)| |_)| | | | | || (_| | ) ) ) )
[10:18:17][Step 2/2] ' |____| .__|_| |_|_| |_\__, | / / / /
[10:18:17][Step 2/2] =========|_|==============|___/=/_/_/_/
[10:18:17][Step 2/2] :: Spring Boot :: (v1.5.2.RELEASE)
[10:18:17][Step 2/2]
[10:18:18][Step 2/2] 2017-07-06 10:18:05.238 INFO 32298 --- [ main] s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext@2aaf7cc2: startup date [Thu Jul 06 10:18:05 EDT 2017]; root of context hierarchy
[10:18:19][Step 2/2] 2017-07-06 10:18:06.299 INFO 32298 --- [ main] c.b.i.h.d.u.e.ExecutableManager : You seem to be running in a LINUX operating system.
[10:18:19][Step 2/2] 2017-07-06 10:18:07.184 INFO 32298 --- [ main] c.b.integration.hub.detect.Application : Configuration processed completely.
[10:18:19][Step 2/2]
[10:18:19][Step 2/2] Current property values:
[10:18:19][Step 2/2] ------------------------------------------------------------
[10:18:19][Step 2/2] cleanupBdioFiles = true
[10:18:19][Step 2/2] cleanupBomToolFiles = true
[10:18:19][Step 2/2] createVirtualEnv = true
[10:18:19][Step 2/2] dockerInspectorVersion = 0.0.4
[10:18:19][Step 2/2] gradleBuildCommand = dependencies
[10:18:19][Step 2/2] gradleCleanupBuildBlackduckDirectory = true
[10:18:19][Step 2/2] gradleInspectorVersion = 0.0.7
[10:18:19][Step 2/2] hubPassword = *************
[10:18:19][Step 2/2] hubTimeout = 120
[10:18:19][Step 2/2] hubUrl = **********
[10:18:19][Step 2/2] hubUsername = ci_scanner
[10:18:19][Step 2/2] loggingLevel = INFO
[10:18:19][Step 2/2] mavenAggregateBom = true
[10:18:19][Step 2/2] nugetInspectorPackageName = IntegrationNugetInspector
[10:18:19][Step 2/2] nugetInspectorPackageVersion = 0.0.3-alpha
[10:18:19][Step 2/2] outputDirectoryPath = blackduck
[10:18:19][Step 2/2] policyCheck = true
[10:18:19][Step 2/2] policyCheckTimeout = 300000
[10:18:19][Step 2/2] projectName = $PROJECT_NAME
[10:18:19][Step 2/2] projectVersionName = In_Development
[10:18:19][Step 2/2] searchDepth = 10
[10:18:19][Step 2/2] sourcePaths = /home/buildagent/buildAgent/work/d3f4ccac47328f58
[10:18:19][Step 2/2] ------------------------------------------------------------
[10:18:19][Step 2/2]
[10:18:20][Step 2/2] 2017-07-06 10:18:07.308 INFO 32298 --- [ main] o.s.j.e.a.AnnotationMBeanExporter : Registering beans for JMX exposure on startup
[10:18:20][Step 2/2] 2017-07-06 10:18:07.323 INFO 32298 --- [ Thread-2] s.c.a.AnnotationConfigApplicationContext : Closing org.springframework.context.annotation.AnnotationConfigApplicationContext@2aaf7cc2: startup date [Thu Jul 06 10:18:05 EDT 2017]; root of context hierarchy
[10:18:20][Step 2/2] 2017-07-06 10:18:07.328 INFO 32298 --- [ Thread-2] o.s.j.e.a.AnnotationMBeanExporter : Unregistering JMX-exposed beans on shutdown
[10:18:20][Step 2/2] Process exited with code 0
Spring Application JSON Environment variable:
(Formatted for better legibility)
{
"detect.hub.url": "$BLACKDUCK_URL",
"detect.hub.username": "$BLACKDUCK_USERNAME",
"detect.hub.password": "$BLACKDUCK_PASSWORD",
"detect.output.path": "blackduck",
"detect.policy.check": "true",
"detect.project.name": "$PROJECT_NAME",
"detect.project.version.name": "Pre_Release"
}Operating Systems Produced on:
- Linux, version 3.13.0-32-generic
- macOS 10.12.5
Okay so I've been running this down and have created a minimal reproducible example here
If you clone it and run
bash <(curl -s https://blackducksoftware.github.io/hub-detect/hub-detect.sh) it will produce the same example as the logs
npm needs to be installed and 'npm install' must be called before running detect. This will generate a node_modules folder which is needed to run NPM properly. Can you verify you've done all of the above? We've added some additional logging in the most recent version of detect that will notify you if you have a package.json but no node_modules folder yet created.
Installing node_modules does resolve the issue. Some additional logging would definitely improve this.
It may also be nice to fall back to npm-shrinkwrap.json if node_modules isn't present then stopping or moving to a different BomTool if neither are present.
Our idea with detect is to have a tool that runs post build. By this I mean everything needed to allow the program to run should already be installed and available. We've considered adding shrink-wrap parsing but believe it not entirely necessary as the node_modules folder should be there when running post build.