mwpenny / kijiji-scraper

A lightweight node.js module for retrieving and scraping ads from Kijiji

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Kijiji.Ad.Get() Doesn't return to callback

DudeManGuy0 opened this issue · comments

const kijiji = require("kijiji-scraper");

//Random ad url i got off kijiji
var url = "https://www.kijiji.ca/v-cell-phone/calgary/iphone-x-64gb/1553098375"

// Scrape using optional callback paramater
kijiji.Ad.Get(url, function(err, ad) {
    if (!err) {
        // Use the ad object
        console.log(ad.title);
    }
});

This code sample was taken straight out of the documentation and the function in the parameter never gets executed. You can see this by putting a breakpoint anywhere inside the function and it will not trigger. Done using version 6.10 of the module.

As a side note, some ad URLs may end with ?undefined which does return this error from the module:
Error: Invalid Kijiji ad URL. Ad URLs must end in /some-ad-id.

However, the return promise version of this code works fine so this isn't a priority.

An options parameter was added to Ad.Get() and Ad.scrape() a few versions back (6.0.0) so you can specify which scraper type to use ("html" or "api"). The signature for Ad.Get(), for example, is now Ad.Get(url[, options, callback]). This is stated in the README but the example is wrong as you point out (it's also wrong for Ad.scrape()).

You need to do it like:

kijiji.Ad.Get(url, {}, function(err, ad) {
    if (!err) {
        // Use the ad object
        console.log(ad.title);
    }
});

I've updated the relevant examples in the README, thanks.

As for the URLs, the query string is irrelevant. You can remove the question mark and everything after it and it will work (the same is true in your browser). Regardless, I've made the URL check more lenient (it will discard the query string).