ameenmaali / urldedupe

Pass in a list of URLs with query strings, get back a unique list of URLs and query string combinations

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Account for port numbers in URLs

ameenmaali opened this issue · comments

Probably makes sense to discard ports when assessing for duplication, but account for something like:

https://site.com:443/home
https://site.com/home

#17 Also solves this.
I think the only place a ':' can occur in a hostname is before the port, so discarding everything after a ':' should work for this.