Search, filter, and browse any JSON data. Includes full text, categorical data, and search interface with elasticsearch backend. Work in progress.
- Make sure you have elasticsearch and rails 4 installed
- On Mac OS do the following
- Go install Elasticsearch using homebrew
brew install elasticsearch
- Install RubyOnRails by typing
gem install rails
- Go install Elasticsearch using homebrew
- On debian / ubuntu
- Install Elasticsearch
sudo apt-get install elasticsearch
- ...to be added by shidash...
- Install Elasticsearch
- On Mac OS do the following
- Clone repo
git clone git@github.com:TransparencyToolkit/LookingGlass.git
- Then cd into directory
cd LookingGlass
- Install dependencies
bundle install
- Generate simple form data
rails generate simple_form:install
- Browse to dataspec directory
cd app/dataspec/
- Copy importer for example data
cp example_importers/lidata_importer.json importer.json
For Running In Production
- Compile your assets
rake assets:precompile
- Then start up elasticsearch
- On Mac Os type
elasticsearch
- On debian type
/etc/init.d/elasticsearch start
- On Mac Os type
- Then type
rails runner 'IndexManager.import_data(force: true)'
when importing / updating datasets - Start up the app
rails server
- Then access http://0.0.0.0:3000 in your browser
This search should work for any dataset. If you want to use a different dataset with this search, see the app/dataspec folder for the necessary files. You will need to create your own dataspec
sheet such as this current file:
lidata.json
A data spec with the name of each field and details about how it is used/where it should show up.importer.json
A JSON with various import settingshidecolumns.json
The indices of the columns to hide by default in the table.
{
"Field Name": "title",
"Human Readable Name": "Title",
"Type": "String",
"Display Type": "Title",
"Mapping": "english",
"Facet?": "No",
"In Table?": "Yes",
"Show by default in table?": "Yes",
"Searchable?": "Yes",
"In doc?": "Yes",
"Form Params": "title",
"Size": "1000",
"Location": "0",
"Link": "single",
"Item_Field": "No",
"Icon": ""
},{
"Field Name": "description",
"Human Readable Name": "Description",
"Type": "String",
"Display Type": "Short Text",
"Mapping": "english",
"Facet?": "No",
"In Table?": "Yes",
"Show by default in table?": "Yes",
"Searchable?": "Yes",
"In doc?": "Yes",
"Form Params": "description",
"Size": "1000",
"Location": "1",
"Item_Field": "No",
"Icon": ""
},{
"Field Name": "document_date",
"Human Readable Name": "Document Date",
"Type": "Date",
"Display Type": "date",
"Mapping": "not_analyzed",
"Facet?": "No",
"In Table?": "Yes",
"Show by default in table?": "No",
"Searchable?": "Yes",
"In doc?": "Yes",
"Form Params": [
"document_date",
"end_document_date"
],
"Size": "1000",
"Location": "5",
"Item_Field": "No",
"Icon": ""
}
One of the more important fields in the dataspec is the Display Type
field. This field informs the interface of where and how to display certain UI attributes used for ordering and filtering the datasets. The following are the types available.
* Title
* Description
* Short Text
* Medium Text
* Long Text
* Category
* Date
* Numerical
* Picture
* Link
Title - The main title on the search terms and the show page. Probably some shorter bit of text. Might be the link to the show page (though not sure if we should also have a more obvious link or not).
Description - Description has a special role like the title where it is shown at the top of the show page (but this might not appear in every dataset). Should be searchable.
Short Text - One line or name length text that isn't a category. It should be possible to sort by Short and titles.
Medium Text - Like the description, but treated like a normal searchable field. Medium-length text should not be truncated.
Long Text - For even longer versions of text such as document. This is truncated in the search results/
Category - Facets. These appear on the sidebar in the results and show view and are links in the results.
Date - Dates. It's possible to filter by these and should be possible to sort by these. These might show up towards the top of search results.
Numerical - For various number fields that are not date values (population, money, etc...). Date fields are sortable.
Picture - A small picture. This probably shows up towards the top of search results and show pages.
Link - A link to document itself?