bda-research / node-crawler

Web Crawler/Spider for NodeJS + server-side jQuery ;-)

Home Page:http://node-crawler.org

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Returning data from callback

Anan5a opened this issue · comments

How can I return the response body from callback function? I'm unable to do anything and practically locked inside the callback only!

const crawler = require('crawler');

module.exports = {
    crawler:function (url){
        let result = null;
        let c = new crawler({
            maxConnections: 1,
            callback: function (error, response, done) {
                if (error) {
                    return console.error(error)
                }
                result = JSON.parse(response.body)
                //returning result from crawler() ??
                done()
            }
        });
        c.queue(url)
        return result
    },

}

During using module architecture you free to organize your code as various of ways, it's not node-crawler package question, but anyway in you case e.g. just pass callback function (or Promise better):

const crawler = require('crawler');

module.exports = {
    crawler:function (url, **callback**) {
    ...
       result = JSON.parse(response.body)
       **callback && callback(result)**
    ...

in your-crawler.js

const myCrawler = require('./my-crawler');

myCrawler.crawler('https://google.com', (result) => {
    console.log(result)
})

if(!error){
res.send(data);
}
else{
console.log(error);
}

In short, res.send might be helpful.