lgraubner / sitemap-generator

Easily create XML sitemaps for your website.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Prevent crawler from changing url

Pau1fitz opened this issue · comments

Do you want to request a feature or report a bug?

What is the current behavior?

When using the crawler I have a url with an ! included the following url gets created:
https://www.test.com/liverpool/players%21/

however, I am looking for the url to look like the following:

https://www.test.com/liverpool/players!/

If the current behavior is a bug, please provide the steps to reproduce.

My code looks like the following:

const SitemapGenerator = require("sitemap-generator");

// create generator
const generator = SitemapGenerator("http://www.test.com", {
    stripQuerystring: false,
    decodeResponses: true
});

// register event listeners
generator.on("done", () => {
    // sitemaps created
    console.log("sitemaps created");
});

// start the crawler
generator.start();

When running node generator.js the url is being replaced with %21 for !

What is the expected behavior?

I want to keep the original url i.e. - https://www.test.com/liverpool/players!/