Wrong example in the documentation?
bortzmeyer opened this issue · comments
The example false = robots:is_allowed(<<"bot/1.0.0">>, <<"/Fish.asp">>, RulesIndex) puzzles me. Since the robots.txt before has no Disallow and since RFC 9309 section 2.2.2 clearly says that if no Allow or Disallow match, access must be granted, is_allowed shoudl return true. (Which it does: the code is right.)
This is correct, this is a typo. Thanks for noticing!