aedan / restify

Enables REST-like access to HTML pages by scraping and parsing them into JSON.

A little utility (written in Go) that enables REST-like access to HTML pages by scraping and parsing them into JSON.

usage: restify [<flags>] <url>

Flags:
  --help                 Show context-sensitive help (also try --help-long and
                         --help-man).
  --class=CLASS          If specified, first-level elements encountered with
                         this class will be extracted.
  --id=ID                If specified, the element with this id will be
                         extracted.
  --attribute=ATTRIBUTE  If specified, as key=value, the element with the given
                         attribute name set to the given value is extracted.
  --version              Print version and exit

Args:
  <url>  A URL to RESTify into JSON

Output Structure

When a successful URL retrieval and match occurs, the utility will output the JSON conversion to stdout.

The top-level structure is an array of jsonNode, where each jsonNode is structured as:

{
  "name":   "...element name...",
  "class":  "...class attribute, if present...",
  "id":     "...id attribute, if present...",
  "href":   "...href attribute, if present...",
  "text":   "...element text content, if present...",
  "elements": [
    ...jsonNodes, if present...
  ]
}

Examples

Locate the latest Minecraft Bedrock server version by picking off the <a>'s with data-platform set:

restify --attribute=data-platform https://www.minecraft.net/en-us/download/server/bedrock/

which produces:

[
  {"name":"a","attributes":{"data-platform":"serverBedrockWindows","role":"button"},"class":"btn btn-disabled-outline mt-4 downloadlink","href":"https://minecraft.azureedge.net/bin-win/bedrock-server-1.12.0.28.zip","text":"Download"},
  {"name":"a","attributes":{"data-platform":"serverBedrockLinux","role":"button"},"class":"btn btn-disabled-outline mt-4 downloadlink","href":"https://minecraft.azureedge.net/bin-linux/bedrock-server-1.12.0.28.zip","text":"Download"}
]

or to grab just the Linux instance:

restify --attribute=data-platform=serverBedrockLinux https://www.minecraft.net/en-us/download/server/bedrock/