Error 409 uploading many large files
sebasegovia01 opened this issue · comments
I'm with this error when i upload many large files in parallel and I can't find a way to fix it. i'm uploading largest files, example: 3 or 4 files of 4gb each one.
When the upload of the files is completed, they are uploaded corrupt.
I am using uppy.js with the implementation of the tus protocol (previously use only tus js client and have the same error).
for the server I am using: //github.com/ankitpokhrel/tus-php/ tus-php for laravel 7.3
This is my client side uppy.js configuration:
const uppy = new Uppy({
debug: true,
locale: UppyES,
autoProceed: false,
restrictions: {
maxFileSize: 9000000000,
maxNumberOfFiles: 4,
minNumberOfFiles: 1,
allowedFileTypes: ['video/*'],
requiredMetaFields: ['dtpicker','tmpicker'],
}
})
.use(UppyTus, {
endpoint: tus_url,
retryDelays: [0, 1000, 3000, 5000],
chunkSize: 5000000,
removeFingerprintOnSuccess: true,
limit: 0
})
TusServiceProvider.php
public function register()
{
$this->app->singleton('tus-server', function ($app) {
$server = new TusServer();
$server
->setApiPath('/tus') // tus server endpoint.
->setUploadDir(public_path('storage/private/videos'));
return $server;
});
}
and tus laravel routes:
//tus
Route::any('/tus/{any?}', function () {
return app('tus-server')->serve();
})->where('any', '.*')->name('tus');
I don't know if it's server or client error 😪
Please if you can help me or guide me where I should consult for this error, thank you!
@sebasegovia01 It can fail because of various reasons eg: not enough memory, not able to store cache data properly etc. Your server logs should give you a hint on what might be the issue. Please check the logs to gather additional info.
@sebasegovia01 It can fail because of various reasons eg: not enough memory, not able to store cache data properly etc. Your server logs should give you a hint on what might be the issue. Please check the logs to gather additional info.
The cache is stored in this document, right? If so, it has sufficient permissions:
Regarding the memory I have the server with 15gb of ram memory, 2 cores and 40gb of available disk space
and this is the only log write that is generated when uploading the file:
access.log tail:
190.44.248.11 - - [08/Oct/2021:10:16:47 -0300] "PATCH /tus/e4029786-3ec6-48db-8a0e-3b07ff917ca3 HTTP/1.1" 204 - "https://developer.whwdata.com/videos" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.81 Safari/537.36"
190.44.248.11 - - [08/Oct/2021:10:16:55 -0300] "GET /videos/data HTTP/1.1" 200 30065 "https://developer.whwdata.com/videos" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.81 Safari/537.36"
190.44.248.11 - - [08/Oct/2021:10:16:48 -0300] "PATCH /tus/c46bf674-1219-446a-9f3a-8ae68946bda9 HTTP/1.1" 204 - "https://developer.whwdata.com/videos" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.81 Safari/537.36"
190.44.248.11 - - [08/Oct/2021:10:16:53 -0300] "PATCH /tus/5308696a-84df-43af-9b39-1279c368c159 HTTP/1.1" 204 - "https://developer.whwdata.com/videos" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.81 Safari/537.36"
190.44.248.11 - - [08/Oct/2021:10:17:01 -0300] "GET /videos/data HTTP/1.1" 200 30065 "https://developer.whwdata.com/videos" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.81 Safari/537.36"
190.44.248.11 - - [08/Oct/2021:10:16:54 -0300] "PATCH /tus/e4029786-3ec6-48db-8a0e-3b07ff917ca3 HTTP/1.1" 204 - "https://developer.whwdata.com/videos" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.81 Safari/537.36"
As can be seen, no more detailed error is printed in log
Today i try upload again and here is the print:
To take into account it only happens when I upload large files in parallel, not in single files upload
I have fixed the bug. I have changed the caching from file to redis and voila! 😁