Webdav Upload Fail Umlauts (äöü)
WeWin55 opened this issue · comments
Hello,
ive got a problem with uploading pdfs with umlauts in the name.
[example: 20161028 - Kündigung.pdf]
The webdav client just gives the error "upload failed" in seafile the file has 0 byte and is named "20161028 - Kündigung.pdf".
Its an encoding error in fast-cgi.
And my seafdav.conf looks like this.
[WEBDAV]
enabled = true
port = 8080
fastcgi = true
share_name = /seafdav
Furthermore i added this to my nginx /seafdav config.
charset utf-8;
override_charset on;
Here is the error log:
tail -fn 200 /var/log/nginx/seafdav.error.log
2017/07/08 15:32:11 [error] 30552#30552: *30791 FastCGI sent in stderr: "" while reading response header from upstream, client: 192.168.16.101, server: seafile.de, request: "PUT /seafdav/Dokumente-Mac/iPhone%20Scan%20Upload/20170708%20-%20Ku%CC%88ndigungsbesta%CC%88tigung%20.pdf HTTP/2.0", upstream: "fastcgi://127.0.0.1:8080", host: "seafile.de"
2017/07/08 15:32:11 [error] 30552#30552: *30791 FastCGI sent in stderr: "Traceback (most recent call last):
File "/opt/seafile/seafile-server-6.1.1/seahub/thirdpart/flup-1.0.2-py2.7.egg/flup/server/fcgi_base.py", line 558, in run
protocolStatus, appStatus = self.server.handler(self)
File "/opt/seafile/seafile-server-6.1.1/seahub/thirdpart/flup-1.0.2-py2.7.egg/flup/server/fcgi_base.py", line 1120, in handler
for data in result:
File "/opt/seafile/seafile-server-6.1.1/seahub/thirdpart/wsgidav/wsgidav_app.py", line 397, in call
for v in app_iter:
File "/opt/seafile/seafile-server-6.1.1/seahub/thirdpart/wsgidav/debug_filter.py", line 154, in call
for v in app_iter:
File "/opt/seafile/seafile-server-6.1.1/seahub/thirdpart/wsgidav/error_printer.py", line 43, in call
for v in app_iter:
File "/opt/seafile/seafile-server-6.1.1/seahub/thirdpart/wsgidav/request_resolver.py", line 201, in call
for v in app_iter:
File "/opt/seafile/seafile-server-6.1.1/seahub/thirdpart/wsgidav/request_server.py", line 113, in call
app_iter = method(environ, start_response)
File "/opt/seafile/seafile-server-6.1.1/seahub/thirdpart/wsgidav/request_server.py", line 638, in doPUT
res = parentRes.createEmptyResource(util.getUriName(path))
File "/opt/seafile/seafile-server-6.1.1/seahub/thirdpart/wsgidav/addons/seafile/seafile_dav_provider.py", line 320, in createEmptyResource
seafile_api.post_empty_file(self.repo.id, self.rel_path, name, self.username)
File "/opt/seafile/seafile-server-6.1.1/seafile/lib64/python2.6/site-packages/seaserv/api.py", line 225, in post_empty_file
filename, username)
File "/opt/seafile/seafile-server-6.1.1/seafile/lib64/python2.6/site-packages/pysearpc/client.py", line 112, in newfunc
return fret(ret_str)
File "/opt/seafile/seafile-server-6.1.1/seafile/lib64/python2.6/site-packages/pysearpc/client.py", line 11, in _fret_int
raise SearpcError(dicts['err_msg'])
SearpcError: file already exists" while reading response header from upst
2017/07/08 15:32:48 [error] 30552#30552: *30804 FastCGI sent in stderr: "Traceback (most recent call last):
File "/opt/seafile/seafile-server-6.1.1/seahub/thirdpart/flup-1.0.2-py2.7.egg/flup/server/fcgi_base.py", line 558, in run
protocolStatus, appStatus = self.server.handler(self)
File "/opt/seafile/seafile-server-6.1.1/seahub/thirdpart/flup-1.0.2-py2.7.egg/flup/server/fcgi_base.py", line 1120, in handler
for data in result:
File "/opt/seafile/seafile-server-6.1.1/seahub/thirdpart/wsgidav/wsgidav_app.py", line 397, in call
for v in app_iter:
File "/opt/seafile/seafile-server-6.1.1/seahub/thirdpart/wsgidav/debug_filter.py", line 154, in call
for v in app_iter:
File "/opt/seafile/seafile-server-6.1.1/seahub/thirdpart/wsgidav/error_printer.py", line 43, in call
for v in app_iter:
File "/opt/seafile/seafile-server-6.1.1/seahub/thirdpart/wsgidav/request_resolver.py", line 201, in call
for v in app_iter:
File "/opt/seafile/seafile-server-6.1.1/seahub/thirdpart/wsgidav/request_server.py", line 113, in call
app_iter = method(environ, start_response)
File "/opt/seafile/seafile-server-6.1.1/seahub/thirdpart/wsgidav/request_server.py", line 638, in doPUT
res = parentRes.createEmptyResource(util.getUriName(path))
File "/opt/seafile/seafile-server-6.1.1/seahub/thirdpart/wsgidav/addons/seafile/seafile_dav_provider.py", line 320, in createEmptyResource
seafile_api.post_empty_file(self.repo.id, self.rel_path, name, self.username)
Same Problem here...
Needs to be fixed
I cannot reproduce the problem.
I created a file called Kündigung.md on my mobile and added Test as content. After that I uploaded it through WebDAV using FolderSync and it was uploaded just fine. It has 4 Byte, can be viewed through Seahub and there is absolutely no issue.
Edit: What I can see in the log is that my client encoded it differently ("PUT /seafdav/tmp/K%C3%BCndigung.md HTTP/1.1" 201 376 "-" "OkHttp", so in your case the ü was encoded as %CC%88 while it was encoded as %C3%BC in my case). So I think it is very likely your client encodes the ü wrong (but I'll edit the answer in case I find something else).
Edit2: At http://sabre.io/dav/character-encoding/ both encodings are listed for an ü. It would now be interesting what the WebDAV standard says about the encoding (I think there is one)
Edit3: I don't want to dig in deeper but it pretty much looks like both encodings are correct (in terms of an URI).
Edit4: Here is a similar ticket where they have an issue with nginx mar10/wsgidav#18. So it could be interesting if the error also occurs in non fastcgi (http) mode.
Hi,
Ive tested it again with a file "StartÜEnd"
Here is the upload.
[08/Nov/2017:13:20:14 +0100] "PUT /seafdav/Dokumente-Mac/iPhone%20Scan%20Upload/StartU%CC%88End.pdf HTTP/2.0" 500 453 "-" "ReaddleDAV ScannerPro/7.3.1.480 (redirects)"
The file itself is called StartÜEnd in seafile, but with 0 Bytes.
What encoding settings do you have in nginx?
Ive tested the webdav upload with macos and here is the error. (sorry its german9
Der Finder konnte diesen Vorgang nicht abschließen, da einige Daten in „StartÜEnd.pdf“ nicht gelesen oder geschrieben werden konnten.
(Fehler: -36)
Same as above. Our clients do encode an ü / Ü differently which is where I suppose the issue.
I've not set any encoding related option in nginx. If nginx uses the systems settings it is likely UTF-8 in my case.
Just to exclude that option: Can you try to upload a file at a path without spaces? E.g. create a library tmp and upload the StartÜEnd file to it.
Same thing here, a file StartÜEnd with 0 bytes..
I use UTF-8 in my debian stretch seafile server too.
If i find my android device, i will try it with foldersync too.
EDIT: So apparently my adroid device is broken (bootloop)
I think its an issue in nginx, as mentioned here mar10/wsgidav#18.
EDIT2: Do you use fastcgi with webdav?
I Want to test it without fastcgi, but cant find any nginx https config without it.
https://manual.seafile.com/extension/webdav.html
Ive installed a fresh seafile and the error is the same with plain seafile and webdav upload.
Here is the error from the fresh seafile installation:
==> seafdav.log <==
[2017-11-10 12:56:51,809]: Traceback (most recent call last):
File "/root/seafile-server-6.2.2/seahub/thirdpart/wsgidav/error_printer.py", line 43, in call
for v in app_iter:
File "/root/seafile-server-6.2.2/seahub/thirdpart/wsgidav/request_resolver.py", line 201, in call
for v in app_iter:
File "/root/seafile-server-6.2.2/seahub/thirdpart/wsgidav/request_server.py", line 113, in call
app_iter = method(environ, start_response)
File "/root/seafile-server-6.2.2/seahub/thirdpart/wsgidav/request_server.py", line 638, in doPUT
res = parentRes.createEmptyResource(util.getUriName(path))
File "/root/seafile-server-6.2.2/seahub/thirdpart/wsgidav/addons/seafile/seafile_dav_provider.py", line 335, in createEmptyResource
raise DAVError(HTTP_INTERNAL_ERROR)
DAVError: DAVError(500 Internal Server Error: An internal server error occurred)
None
And here the error from my seafile server:
==> seafdav.log <==
[2017-11-10 12:53:02,049]: Traceback (most recent call last):
File "/opt/seafile/seafile-server-6.2.2/seahub/thirdpart/wsgidav/error_printer.py", line 43, in call
for v in app_iter:
File "/opt/seafile/seafile-server-6.2.2/seahub/thirdpart/wsgidav/request_resolver.py", line 201, in call
for v in app_iter:
File "/opt/seafile/seafile-server-6.2.2/seahub/thirdpart/wsgidav/request_server.py", line 113, in call
app_iter = method(environ, start_response)
File "/opt/seafile/seafile-server-6.2.2/seahub/thirdpart/wsgidav/request_server.py", line 638, in doPUT
res = parentRes.createEmptyResource(util.getUriName(path))
File "/opt/seafile/seafile-server-6.2.2/seahub/thirdpart/wsgidav/addons/seafile/seafile_dav_provider.py", line 335, in createEmptyResource
raise DAVError(HTTP_INTERNAL_ERROR)
DAVError: DAVError(500 Internal Server Error: An internal server error occurred)
None
Yes, I use it with fastcgi and nginx.
Thanks, can you please execute "locale" and send me the output?
LANG=en_US.UTF-8
LANGUAGE=
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=en_US.UTF-8
Thanks, its the same as on my system.
Can you please send me your nginx settings?
nginx.conf
user www-data;
worker_processes 4;
pid /run/nginx.pid;
events {
worker_connections 768;
# multi_accept on;
}
http {
##
# Basic Settings
##
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
include /etc/nginx/mime.types;
default_type application/octet-stream;
##
# SSL Settings
##
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_dhparam /etc/nginx/ssl/dhparam.pem;
ssl_prefer_server_ciphers on;
ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA';
add_header X-Frame-Options SAMEORIGIN;
add_header X-XSS-Protection "1; mode=block";
add_header Strict-Transport-Security max-age=15768000;
ssl_session_cache shared:SSL:10m;
ssl_session_timeout 60m;
##
# Logging Settings
##
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
##
# Gzip Settings
##
gzip on;
gzip_disable "msie6";
include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*;
}
seafile
server {
listen 80;
server_name home.hoeper.me;
rewrite ^ https://$host$request_uri?;
}
server {
listen 443 ssl http2;
server_name home.hoeper.me;
error_page 497 https://$host$request_uri;
client_max_body_size 0;
large_client_header_buffers 4 128k;
ssl_certificate /opt/letsencrypt/certs/home.hoeper.me/fullchain.pem;
ssl_certificate_key /opt/letsencrypt/certs/home.hoeper.me/privkey.pem;
ssl_dhparam /etc/nginx/ssl/mattermost.mno.pw/dhparam.pem;
proxy_set_header X-Forwarded-For $remote_addr;
location / {
fastcgi_pass 127.0.0.1:8000;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param PATH_INFO $fastcgi_script_name;
fastcgi_param SERVER_PROTOCOL $server_protocol;
fastcgi_param QUERY_STRING $query_string;
fastcgi_param REQUEST_METHOD $request_method;
fastcgi_param CONTENT_TYPE $content_type;
fastcgi_param CONTENT_LENGTH $content_length;
fastcgi_param SERVER_ADDR $server_addr;
fastcgi_param SERVER_PORT $server_port;
fastcgi_param SERVER_NAME $server_name;
fastcgi_param HTTPS on;
fastcgi_param HTTP_SCHEME https;
fastcgi_param REMOTE_ADDR $remote_addr;
access_log /var/log/nginx/seahub.access.log;
error_log /var/log/nginx/seahub.error.log;
}
location /seafhttp {
rewrite ^/seafhttp(.*)$ $1 break;
proxy_pass http://127.0.0.1:8082;
client_max_body_size 0;
proxy_connect_timeout 36000s;
proxy_read_timeout 36000s;
access_log /var/log/nginx/seafhttp.access.log;
error_log /var/log/nginx/seafhttp.error.log;
}
location /seafdav {
fastcgi_pass 127.0.0.1:8080;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param PATH_INFO $fastcgi_script_name;
fastcgi_param SERVER_PROTOCOL $server_protocol;
fastcgi_param QUERY_STRING $query_string;
fastcgi_param REQUEST_METHOD $request_method;
fastcgi_param CONTENT_TYPE $content_type;
fastcgi_param CONTENT_LENGTH $content_length;
fastcgi_param SERVER_ADDR $server_addr;
fastcgi_param SERVER_PORT $server_port;
fastcgi_param SERVER_NAME $server_name;
fastcgi_param REMOTE_ADDR $remote_addr;
fastcgi_param HTTPS on;
fastcgi_param HTTP_SCHEME https;
access_log /var/log/nginx/seafdav.access.log;
error_log /var/log/nginx/seafdav.error.log;
# allow large file upload
fastcgi_ignore_client_abort on;
# kill after 30 Minutes
fastcgi_read_timeout 30m;
}
location /media {
root /seafile/seafile-server-latest/seahub;
access_log /var/log/nginx/seafmedia.access.log;
error_log /var/log/nginx/seafmedia.error.log;
}
}
Thanks for the config, ive updated mine a bit. Unfortunately the upload doesn't work.
How did you solve this? i have the same issue.
that. is not the answer i hoped for. Thank you.
Just stumbled across this error. I'm using Scanner Pro to quickly take photos of documents and automatically upload them to seafile. (which works pretty well)
But the default filename includes the date and as it is now "März" (March) yesterday the upload stopped working with the same problems as described above:
- 0 byte file
SearpcError: file already exists" while reDAVError: DAVError(500 Internal Server Error: An internal server error occurred)
Changing the filename to not include the umlaut fixes everything.
My locale:
LANG=en_US.UTF-8
LANGUAGE=
LC_CTYPE=de_AT.UTF-8
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=
nginx config:
location /seafdav {
fastcgi_pass 127.0.0.1:8001;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param PATH_INFO $fastcgi_script_name;
fastcgi_param SERVER_PROTOCOL $server_protocol;
fastcgi_param QUERY_STRING $query_string;
fastcgi_param REQUEST_METHOD $request_method;
fastcgi_param CONTENT_TYPE $content_type;
fastcgi_param CONTENT_LENGTH $content_length;
fastcgi_param SERVER_ADDR $server_addr;
fastcgi_param SERVER_PORT $server_port;
fastcgi_param SERVER_NAME $server_name;
fastcgi_param REMOTE_ADDR $remote_addr;
fastcgi_param HTTPS on;
fastcgi_param HTTP_SCHEME https;
client_max_body_size 0;
proxy_connect_timeout 36000s;
proxy_read_timeout 36000s;
proxy_send_timeout 36000s;
send_timeout 36000s;
# This option is only available for Nginx >= 1.8.0. See more details below.
proxy_request_buffering off;
}HI Findus23, thanks for your contribution. I will check if this is fixing my problem, as webdav is working again in seafile. Currently its broken.
This was the differnce between our locales:
< LC_CTYPE="en_US.UTF-8"
LC_CTYPE=de_AT.UTF-8
As im waiting for seafile to fix webdav in 6.3.12, im using my hotfix with an extra webdav instance, that writes to a local seafile-cli client.
Hi there,
The issue also happens for me. Same errors as described above. Using Scanner Pro app, the upload fails when a Umlaut is present in the filename.
2019/03/12 17:02:55 [error] 1397#1397: *3 readv() failed (104: Connection reset by peer) while reading upstream, client: 213.182.143.74, server: cloud.foo.org, request: "PUT /seafdav/private/2019-03-12%20-%20Bra%CC%88uer2.pdf HTTP/1.1", upstream: "http://127.0.0.1:8080/seafdav/private/2019-03-12%20-%20Bra%CC%88uer2.pdf", host: "cloud.foo.org:444"
[2019-03-12 17:02:55,377]: e.srcexception:
None
[2019-03-12 17:02:55,782]: ErrorPrinter: caught Exception
[2019-03-12 17:02:55,782]: Traceback (most recent call last):
File "/usr/local/seafile/seafile-server-6.3.4/seahub/thirdpart/wsgidav/error_printer.py", line 43, in __call__
for v in app_iter:
File "/usr/local/seafile/seafile-server-6.3.4/seahub/thirdpart/wsgidav/request_resolver.py", line 201, in __call__
for v in app_iter:
File "/usr/local/seafile/seafile-server-6.3.4/seahub/thirdpart/wsgidav/request_server.py", line 113, in __call__
app_iter = method(environ, start_response)
File "/usr/local/seafile/seafile-server-6.3.4/seahub/thirdpart/wsgidav/request_server.py", line 638, in doPUT
res = parentRes.createEmptyResource(util.getUriName(path))
File "/usr/local/seafile/seafile-server-6.3.4/seahub/thirdpart/wsgidav/addons/seafile/seafile_dav_provider.py", line 320, in createEmptyResource
seafile_api.post_empty_file(self.repo.id, self.rel_path, name, self.username)
File "/usr/local/seafile/seafile-server-6.3.4/seafile/lib64/python2.7/site-packages/seaserv/api.py", line 269, in post_empty_file
filename, username)
File "/usr/local/seafile/seafile-server-6.3.4/seafile/lib64/python2.7/site-packages/pysearpc/client.py", line 112, in newfunc
return fret(ret_str)
File "/usr/local/seafile/seafile-server-6.3.4/seafile/lib64/python2.7/site-packages/pysearpc/client.py", line 11, in _fret_int
raise SearpcError(dicts['err_msg'])
SearpcError: file already exists
We will fix this bug in 7.1.