Prevent crawler from changing url
Pau1fitz opened this issue · comments
Do you want to request a feature or report a bug?
What is the current behavior?
When using the crawler I have a url with an !
included the following url gets created:
https://www.test.com/liverpool/players%21/
however, I am looking for the url to look like the following:
https://www.test.com/liverpool/players!/
If the current behavior is a bug, please provide the steps to reproduce.
My code looks like the following:
const SitemapGenerator = require("sitemap-generator");
// create generator
const generator = SitemapGenerator("http://www.test.com", {
stripQuerystring: false,
decodeResponses: true
});
// register event listeners
generator.on("done", () => {
// sitemaps created
console.log("sitemaps created");
});
// start the crawler
generator.start();
When running node generator.js
the url is being replaced with %21
for !
What is the expected behavior?
I want to keep the original url i.e. - https://www.test.com/liverpool/players!/