syscools / xidel

A command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern templates. It can also create new or transformed XML/HTML/JSON documents.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Xidel

Xidel is a command line tool to download and extract data from HTML/XML pages using CSS selectors, XPath/XQuery 3.0, as well as querying JSON files or APIs (e.g. REST) using JSONiq.

There are dependency-free binaries for Windows, Linux and Mac.

It is a wrapper around my Pascal Internet Tools (see repository internettools), so it supports XPath 2.0, XPath 3.0, XQuery 1.0, XQuery 3.0, JSONiq, CSS selectors and my own extensions/languages (e.g. pattern matching) and if you can compile that project, you can compile Xidel.

A simple example to return the titles of all pages linked by some starting page:

 xidel http://example.org --follow //a --extract //title

or simpler

 xidel http://example.org -f //a -e //title

The language can be explicitly chosen. For example

 xidel input.html --css 'a'
 xidel input.html --xpath '//a/@href'
 xidel input.html --xquery 'for $var in //a order by $var return $var'

returns all links, the target URI of each link or the text of all links alphabetically.

There are more examples on the above page with binaries.

About

A command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern templates. It can also create new or transformed XML/HTML/JSON documents.


Languages

Language:Pascal 46.3%Language:Shell 17.6%Language:JavaScript 8.5%Language:SourcePawn 8.1%Language:Java 7.8%Language:PHP 3.6%Language:XQuery 3.5%Language:HTML 2.9%Language:Roff 1.4%Language:CSS 0.2%Language:Makefile 0.1%Language:Perl 0.1%