Giters

t1gor / Robots.txt-Parser-Class

Php class for robots.txt parse

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Relative paths is always allowed

JanPetterMG opened this issue · comments

commented 
User-agent: *
Disallow: /

$this->assertTrue($parser->isDisallowed("&&1@|"));
$this->assertFalse($parser->isAllowed('+£€@@1¤'));

The two tests above fails, paths allowed according to the parser, the robots.txt says otherwise...

At least the second one could also be an potential relative path, so we can't just ignore this bug either, we have to fix it...

commented

Also applies to:

$this->assertFalse($parser->isAllowed('path'));
$this->assertFalse($parser->isAllowed('*wildcard'));

I'm not sure why I didn't test that earlier... 😕 This is no longer an low-priority issue, but an huge and prioritized one!