javierjulio / dictionary

A Ruby parser for the GCIDE English word dictionary that generates friendly structured JSON files for easy mass database import. Includes other resources if you need more data for an English dictionary database.

English Dictionary

This is a minimally tested and incomplete parser of the Webster Unabriged English Dictionary from the modified GCIDE XML that categorizes content to make it easy to find and parse. I was doing a lot of research on finding a machine readable English dictionary for a project where I didn't want to rely on a third party API (e.g. Wordnik).

Generate Simple JSON

From the project directory, run the following:

ruby parse.rb

This will generate a JSON file for each GCIDE XML file. Each object key is a unique word and the value being an object containing the definitions (array of objects - definition, part of speech, field, and sequence). The files (excluding obsolete content) will contain ~99k unique words and ~160k definitions.

Resources

GCIDE

After reviewing all resources went first with parsing this GCIDE XML. The next best solution seems to be Wikitionary TSV.

• http://rali.iro.umontreal.ca/GCIDE/ (the ZIP download is further down the page)

Wikitionary TSV

• http://aautar.digital-radiation.com/wiktionary-db/wiktionary.E20121127.tsv.zip
• http://semisignal.com/?p=5666 (TSV file linked to above and sample code)
• https://github.com/boyers/asler/tree/master/scratch

Webster's Unabridged Dictionary (1913 - public domain)

• http://www.mso.anu.edu.au/~ralph/OPTED/index.html
• https://github.com/janosgyerik/dictgen (Plain Text parser)
• http://en.wiktionary.org/wiki/Wiktionary:Abbreviations_in_Webster

Moby Word Lists

• https://github.com/drichert/moby (Ruby parser for hyphenation, parts-of-speech, and thesaurus)