Support of multiple User-agents in a record

[SATŌ Kentarō]

The draft http://www.robotstxt.org/norobots-rfc.txt
mentions of having multiple User-agent in a record to share rules.
Would you think it possible to support this syntax?

User-agent: webcrawler
#some webmaster note for webcrawler
User-agent: infoseek
#some webmaster note for infoseek
Disallow: /foo
Disallow: /bar

[Jan-Petter Gundersen]

It should be supported already, if not, we have another bug.
I'll take look at it...

[SATŌ Kentarō]

The code I run on c68a7a8 is:

require_once(__DIR__ . '/vendor/autoload.php');
$parser = new \RobotsTxtParser('User-agent: webcrawler
User-agent: infoseek
Disallow: /foo');
var_dump($parser->isAllowed('/foo', 'webcrawler')); // bool(true)
var_dump($parser->isAllowed('/foo', 'infoseek')); // bool(false)