the encoding declaration in @types/crawler

[wubinhong]

Issue Description

In the tutorial of Crawler

I cannot assign a null value to CrawlerRequestOptions, when following the tutorial as below:

Trouble-shooting

encoding defined in @types/crawler

    interface CrawlerRequestOptions {
        encoding?: string | undefined;
    }

The implementation for encoding in file lib/crawler.js

Suggestion

Solution 1 (Recommended)

Change the encoding declaration in @types/crawler as below

    interface CrawlerRequestOptions {
        encoding?: string | undefined | null;
    }

Solution 2

Change the implementation of handling encoding as below:

Crawler.prototype._doEncoding = function (options, response) {
	var self = this;

	if (!options.encoding) {
		return;
	}

	if (options.forceUTF8) {
		var charset = options.incomingEncoding || self._parseCharset(response);
		response.charset = charset;
		log('debug', 'Charset ' + charset);

		if (charset !== 'utf-8' && charset !== 'ascii') {// convert response.body into 'utf-8' encoded buffer
			response.body = iconvLite.decode(response.body, charset);
		}
	}

	response.body = response.body.toString();
};

[Mike Chen]

Hi guy, I think this should be a bug of type inference in typescript. I don't like typescript, also suggest you no to use it.