Do all crawls (fetch) and cheerio-stuff before adding to search index
eklem opened this issue · comments
Espen Klem commented
https://daveceddia.com/waiting-for-promises-in-a-loop/
Use the second example, then it's easier to add a play-nice timer between each fetch()
request:
var chain = $q.when();
for(var i = 0; i < 5; i++) {
chain = chain.then(function() {
return $http.get('/data' + i);
});
}
After the foor-loop a search-index adder can be called.
Espen Klem commented
Actually, no. Do one and one, and let search-index-housekeeper keep track of whats indexed and not. We don't know when the user will click to a new page.
Espen Klem commented
Espen Klem commented
Check promise.all