domains & exclude domains wildcards
opened this issue · comments
-D --domains Comma-separated list of domains to follow.
--exclude-domains Comma-separated list of domains NOT to follow.
Please add to a these options wildcard or regex support. It's needed for -r option, for ie
cdn1.domain.com
cdn2.domain.com
cdn3.domain.com
& etc.
It is there. If in doubt, try the 'develop' branch.
I'll have to make a new release soon, I guess ;-)
$ src/mget --help|grep -i domains
-D --domains Comma-separated list of domains to follow.
--exclude-domains Comma-separated list of domains NOT to follow.
But yes, wildcards are not possible here (thanks to Wget compatibility).
You can try --accept/--reject with patterns. The check is not limited to the path, but also includes the domain. URLs will still be scanned (but using HEAD request first, and they not going to be saved).
But you are right, --domains/--exclude-domains should work with wildcards.
I'll implement it in the next days.
Wildcard support is now in branch 'develop' (done by fnmatch).
Also, international domain names (IDN) are supported.
The input might be percent-encoded.
Case does not matter.
Example: --domain="x_.example.com,was._.übel.de,ex?mple.com"