Giters

alephdata / aleph

Search and browse documents and data; find the people and companies you look for.

Home Page:http://docs.aleph.occrp.org

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

BUG: unable to upload a large amount of files through the UI

stchris opened this issue · comments

commented

Describe the bug
Uploading a large amount of files through the UI can make an Aleph installation freeze and stop processing.

To Reproduce
Steps to reproduce the behavior:

  1. Go to an investigation
  2. Click on 'Upload documents'
  3. Select a large amount of files (thousands)
  4. Aleph will choke and stop processing them.

Expected behavior
Aleph is either able to show and enforce a limit to the number of files being uploaded or it is able to queue the files so that they are processed correctly.

Aleph version
3.15.1

Screenshots

Additional context
The current workaround is to use alephclient

commented

Thanks for helping us with this issue, Chris!
Extra context can be found in #3383 .
"Uploading a folder with 8K files worked, but not another with 300K files"

commented

Options here:

  1. Document a limit (1,000 - 5,000?) files max, possibly size could have an impact.
  2. Try to fix the UI to handle larger file sets (harder, more time consuming).
  3. Limit uploads to single files at a time
commented