TransparencyToolkit / Harvester

Web crawling and document processing through a usable interface.

https://transparencytoolkit.org

TransparencyToolkit/Harvester Issues

Add authentication to post requests to LookingGlass
Closed 5 years ago3
Index Harvester dataitem on LookingGlass
Closed 8 years ago1
Append dataset_name and data_source to dataitem
Closed 8 years ago1
Only create new index if it doesn't exist
Closed 8 years ago1
Change dataspec repository names to work with format for loader
Closed 8 years ago1
Get LG dataspec object and create index from Harvester
Closed 8 years ago1
Why do neo4j associations take so long to create?
Closed 5 years ago2
Figure out how to index data from neo4j in elasticsearch
Closed 8 years ago2
Associate dataitem with term
Closed 8 years ago1
Make sure neo4j dataitems are accessible from dataset
Closed 8 years ago3
Save search terms in neo4j dataset object
Closed 8 years ago1
Button links on dataset index view
Closed 8 years ago
Make index view for datasets
Closed 8 years ago
Better handle line breaks, newlines, and dots
Updated 4 years ago
Add crawler for Rapportive data
Closed 8 years ago
neo4j does not work in DocumentLoader
Updated 8 years ago2
Add file chooser to input form for datasources that require directory input
Closed 5 years ago3
Add Icons for Indeed and Load Files
Closed 8 years ago1
Implement overview of "collected datasets" view
Closed 8 years ago2
Add auto-suggested "terms" to Step 2
Updated 8 years ago
Implement collected "dataset / terms" view
Closed 8 years ago1
Add ability to add multiple input query fields
Closed 8 years ago2