TransparencyToolkit / Harvester

Web crawling and document processing through a usable interface.

Home Page:https://transparencytoolkit.org

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Add file chooser to input form for datasources that require directory input

Shidash opened this issue · comments

Right now for data sources that require a directory as input, a text field is shown. This is because the rails file_field_tag only allows you to choose files, not directories. The code currently managing this is at https://github.com/TransparencyToolkit/Harvester/blob/master/app/views/datasets/_query_fields.html.erb

This is not ideal as people need to know the full file path. There should really be some sort of file picker. We just need the path to the directory, not the files themselves, so this may be doable by some other beans. @bnvk do you know if there's some sort of javascript file picker we could use that allows selection of directories and will give us the path? If so, we could then save the path in a hidden form field with the appropriate name/id.

@Shidash i'm 99.8% percent sure the only ways to get data about the file system (not files) with front-end code (and a picker of some sort) involves using either Flash or Java plugins, as it's a pretty big security / privacy vulnerability otherwise!

If you're talking about a user accessing an instance of Harvester over the internet, there are JS upload plugins that allow selecting of multiple files at once, but not full folders!

But if you're talking about a case where a user is loading files from same computer that Harvester is running on (like we do some from our dev machines), then I have a few ideas that I could share!

Yeah, I'm just talking about the use case of a user loading the files from the same computer Harvester is running on.

I think the approach here is to start with the users "home directory" and present rendered in a virutal file browser (interface) with UI buttons / actions like:

  • Clicking on file selects that file to import
  • Clicking on a directory opens the contents of that directory in the view
  • A button that goes "Up One Directory" which presents a scan of the parent view

It can kind of be seen as a "virtual file system browser" just rendered in a webview, like what ownCloud shows, basically.