FileWalker failure with large number of files
Downchuck opened this issue · comments
I've got a large project that depends on hive-exec via "provides", because the environment already has it.
<manifest>
<entry>
<key>Dependencies</key>
<value>org.apache.hive:hive-exec:${hive.version}</value>
</entry>
</manifest>
I unpack all of the typical dependencies and build the project:
mvn -DoutputDirectory=target/classes/ dependency:unpack-dependencies
mvn compile
rm target/classes/META-INF/MANIFEST.MF
This gets me a big chunk of unpacked project, much as maven-assembly-plugin will do.
Now: mvn capsule:build
java.io.FileNotFoundException... Too many open files in system.
...
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:146)
at capsule.CapsuleMojo$1.visitFile(CapsuleMojo.java:236)
at capsule.CapsuleMojo$1.visitFile(CapsuleMojo.java:232)
at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:135)
at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:199)
at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:199)
at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:199)
at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:199)
at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:69)
at java.nio.file.Files.walkFileTree(Files.java:2600)
at java.nio.file.Files.walkFileTree(Files.java:2633)
at capsule.CapsuleMojo.buildThin(CapsuleMojo.java:232)
Need to close that FileInputStream:
Ah thanks for reporting. I have no way of replicating the error, perhaps you could share a demo project so I can attempt to fix the error. I closed the stream in the latest snapshot, please give 0.9.1-SNAPSHOT a try.
That fix will work -- you have to have -a-lot- of class files on your path to replicate the error. I also fixed the error by closing the input stream. Marking this as fixed.