Truncated nar pulling through S3 storage backend
devplayer0 opened this issue · comments
I've noticed an issue with the S3 storage backend with atticd. Seems like the requests to the server that reassemble chunks from S3 can sometimes (about 1 in 10 times) be truncated, example below. The truncated length appears to be inconsistent too.
root@f8219b85a1f4:/# curl -4H 'Host: nix-cache.nul.ie' -v http://object-ctr.ams1.int.nul.ie:8069/main/nar/3gch1y43xp3qslz2v4axbd5j02pdnwla.nar | sha256sum
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Trying 10.100.2.7:8069...
* Connected to object-ctr.ams1.int.nul.ie (10.100.2.7) port 8069 (#0)
> GET /main/nar/3gch1y43xp3qslz2v4axbd5j02pdnwla.nar HTTP/1.1
> Host: nix-cache.nul.ie
> User-Agent: curl/7.74.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< x-attic-cache-visibility: public
< transfer-encoding: chunked
< date: Fri, 17 Nov 2023 20:18:21 GMT
<
{ [7629 bytes data]
100 12.1M 0 12.1M 0 0 52.9M 0 --:--:-- --:--:-- --:--:-- 52.9M
* Connection #0 to host object-ctr.ams1.int.nul.ie left intact
45cd8d682ab81f264a01810ba9bff2221992ade72c3e9f652b5db0f2ddbd8952 -
root@f8219b85a1f4:/# curl -4H 'Host: nix-cache.nul.ie' -v http://object-ctr.ams1.int.nul.ie:8069/main/nar/3gch1y43xp3qslz2v4axbd5j02pdnwla.nar | sha256sum
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Trying 10.100.2.7:8069...
* Connected to object-ctr.ams1.int.nul.ie (10.100.2.7) port 8069 (#0)
> GET /main/nar/3gch1y43xp3qslz2v4axbd5j02pdnwla.nar HTTP/1.1
> Host: nix-cache.nul.ie
> User-Agent: curl/7.74.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< x-attic-cache-visibility: public
< transfer-encoding: chunked
< date: Fri, 17 Nov 2023 20:18:22 GMT
<
{ [3525 bytes data]
* transfer closed with outstanding read data remaining
100 11.7M 0 11.7M 0 0 45.3M 0 --:--:-- --:--:-- --:--:-- 45.3M
* Closing connection 0
curl: (18) transfer closed with outstanding read data remaining
0e488c38739c2428a0906c522cbb9250fe232a6866d0f397fd48a98ffb503f0e -
root@f8219b85a1f4:/#
For now I've switched to the local backend and it's been working great. :)