Provide a way to disable fetching a fresh public suffix list on compile
axelson opened this issue · comments
Hi, the ability to fetch a public suffix list on compilation does sound useful, but the compile time of Domainatrex is already quite long (over 10s on heroku) and I'd rather not emit any HTTP requests while compiling. So is it possible to create a way disable fetching the public suffix list on compile?
The reason it takes a long time to compile is because it has macros which declare a match function for every TLD in the public suffix list, which is quite long. I know it sucks that it takes this long, but it means that at runtime the matching is highly performant.
I decided to fetch the public suffix list on compile because the list is updated frequently, and it seems silly to need a new version number and interaction from me every time this happens, and also because typically this library won't be built particularly frequently.
Also, just for what it's worth, I did a quick test (on a 7 year old Macbook Air and very average Australian ADSL, so YMMV). Removing the logic which fetches the new list and just using the one from disk shaves about 2 seconds off the build time (14s becomes 12s). I just used time mix compile to get these numbers.
For me this is not really an issue, but if you have a use case which is more compile-speed-dependent than mine and would like to implement a way to compile with an option to not fetch a new public suffix list, I would be more than happy to have a look at it. I would like the current behaviour to be the default though.
Let me know what you think.
@Zensavona I understand your point, but one of my goals is having reproducible (or at least near-reproducible) builds and emitting HTTP requests during a build will definitely cause issues with that. I can also imagine scenarios where you are completely unable to make http requests or would prefer not to for security reasons (such as an attacker taking control of the suffix link website/dns).
I think in the next release I'll make using a local (perhaps old) suffix list optional.