parallelize downloads
ungoldman opened this issue · comments
It's taking too dang long!
🏠 ng@crimelab ~/dev/github/himawari-urls (master) $ time himawari-bg
Attemping to fetch latest visible light images from Himawari 8
Downloading satellite images [====================] 100% 0.0s
Setting /Users/ng/Pictures/himawari-images/1455267707496.jpg as background...
Complete!
real 0m24.056s
user 0m1.300s
sys 0m0.423s
My code uses async.eachSeries, but it could EASILY use async.each or async.map which runs async operations in parallel. However, I don't think it quite works for some reason related to the file stream/piping logic. I'll run some timing tests on Monday and compare the two operations.
Parallelizing the downloads would make the logs more awkward but that can be a compromise in favor of speed.
Using parallel by default in 1.0.0-beta. Thanks @jakiestfu for adding it to himawari.js!