Exception in thread "async-thread-macro-1"
dragon99919 opened this issue · comments
After successful page fetch and assets download, while generating local copy, program started to take a lot of memory at once and then it threw an exception shown below:
Looking at the assets folder, it appears that only 7320 from above 70k were saved, and no index was generated.
Logs also don't tell a thing, last entry is about downloading stuff.
@dragon99919: This is an out-of-memory error. Try aborting and then re-running with the -Xmx1G switch, as in:
java -Xmx1G -Djdk.tls.client.protocols=TLSv1,TLSv1.1,TLSv1.2 -jar soupscraper-0.1.0.jar yoursoupJust tried, same thing happened, exactly same error with all the lines related and such...
@dragon99919 It's about the memory. Give the process as much memory as you can.
If your machine has 8GB, try 4gb. If it has more, try more.
Gave 10GB and it worked overnight! Thanks for the help! : )