x-robots-tag

There are 0 repository under x-robots-tag topic.

• lerni / folderindex

  Silverstripe Module to set X-Robots-Tag noindex, nofollow, noimageindex, noarchive, and nosnippet for files in folders and sub-folders when ShowInSearch is unchecked.

  x-robots-tagsilverstripefileassetsnoindex
  Language:PHP 2

• VIPnytt / UserAgentParser

  User-Agent parser for robots.txt, X-Robots-tag and Robots-meta-tag rule sets

  user-agentuser-agent-parserrobots-txtrobots-tagx-robots-tagrobots-meta-tagrobots-exclusion-protocolreprobotspidercrawler
  Language:PHP 1

• toimik / RobotsProtocol

  Parsers for robots.txt (aka Robots Exclusion Standard / Robots Exclusion Protocol), Robots Meta Tag, and X-Robots-Tag

  robots-txtrobots-parserrobotstxtrobotsrobots-exclusion-protocolrobots-exclusion-standardrobots-tagx-robots-tagmeta-tags
  Language:C# 0

• uretgec / roboindex-checker

  Website meta tag checker tool made by both node and golang for my education

  x-robots-tagspider-botsindex-check
  Language:Go