getting superfluous response.WriteHeader call from github.com/chrislusf/seaweedfs/weed/server.processRangeRequest (common.go:280) from weed volume after updating aws API on EKS, getting random issues while fatching data
nirmalk47 opened this issue · comments
Describe the bug
We are using Seaweedfs in one of our EKS server in march 23 we our EKS node AMI version got updated and after that we are getting issues while we are fetching file from weedfs.
randomly we are getting below issues,
Not able to download files in one shot, we have to try multiple times, sometimes it gets downloaded 50% some times not.
in seaweed fs volume we are getting superfluous response.WriteHeader error, The master is getting connection issue connection error within the same pod but pod is not failing
System Setup
- Sever is on EKS
- 1 weed master and 4 weed volumes (clients)
AMI version: amazon-eks-node-1.23-v20240315 , kernal: 5.4.269-183.369.amzn2.x86_64 - version 30GB 2.79 f3c789d linux amd64
Expected behavior
the data should be successfully accessible and downloadable.
logs
weed master
2024-03-26T20:04:45.081Z | I0326 20:04:45 1 master_grpc_server.go:278] - client master@192.168.111.149:9333
2024-03-26T20:04:45.081Z | I0326 20:04:45 1 masterclient.go:125] master masterClient failed to receive from 192.168.111.149:9333: rpc error: code = Unavailable desc = closing transport due to: connection error: desc = "error reading from server: EOF", received prior goaway: code: NO_ERROR
2024-03-26T20:04:46.082Z | I0326 20:04:46 1 master_grpc_server.go:262] + client master@192.168.111.149:9333
2024-03-27T06:02:59.753Z | I0327 06:02:59 1 master_grpc_server.go:278] - client master@192.168.111.149:9333
2024-03-27T06:02:59.753Z | I0327 06:02:59 1 masterclient.go:125] master masterClient failed to receive from 192.168.111.149:9333: rpc error: code = Unavailable desc = closing transport due to: connection error: desc = "error reading from server: EOF", received prior goaway: code: NO_ERROR
2024-03-27T06:03:00.754Z | I0327 06:03:00 1 master_grpc_server.go:262] + client master@192.168.111.149:9333
Weed volume (client)
2024/03/26 19:48:53 http: superfluous response.WriteHeader call from github.com/chrislusf/seaweedfs/weed/server.processRangeRequest (common.go:280)
2024/03/26 19:49:10 http: superfluous response.WriteHeader call from github.com/chrislusf/seaweedfs/weed/server.processRangeRequest (common.go:280)
2024/03/26 21:55:35 http: superfluous response.WriteHeader call from github.com/chrislusf/seaweedfs/weed/server.processRangeRequest (common.go:280)
2024/03/26 21:59:21 http: superfluous response.WriteHeader call from github.com/chrislusf/seaweedfs/weed/server.processRangeRequest (common.go:280)
2024/03/27 06:01:07 http: superfluous response.WriteHeader call from github.com/chrislusf/seaweedfs/weed/server.processRangeRequest (common.go:280)
2024/03/27 06:01:09 http: superfluous response.WriteHeader call from github.com/chrislusf/seaweedfs/weed/server.processRangeRequest (common.go:280)
2024/03/27 06:06:06 http: superfluous response.WriteHeader call from github.com/chrislusf/seaweedfs/weed/server.processRangeRequest (common.go:280)
2024/03/27 06:06:08 http: superfluous response.WriteHeader call from github.com/chrislusf/seaweedfs/weed/server.processRangeRequest (common.go:280)
2024/03/27 06:10:05 http: superfluous response.WriteHeader call from github.com/chrislusf/seaweedfs/weed/server.processRangeRequest (common.go:280)
2024/03/27 06:10:06 http: superfluous response.WriteHeader call from github.com/chrislusf/seaweedfs/weed/server.processRangeRequest (common.go:280)
2024/03/27 06:10:07 http: superfluous response.WriteHeader call from github.com/chrislusf/seaweedfs/weed/server.processRangeRequest (common.go:280)
2024/03/27 06:17:45 http: superfluous response.WriteHeader call from github.com/chrislusf/seaweedfs/weed/server.processRangeRequest (common.go:280)
2024/03/27 06:31:45 http: superfluous response.WriteHeader call from github.com/chrislusf/seaweedfs/weed/server.processRangeRequest (common.go:280)
2024/03/27 06:31:47 http: superfluous response.WriteHeader call from github.com/chrislusf/seaweedfs/weed/server.processRangeRequest (common.go:280)
2024/03/27 06:46:05 http: superfluous response.WriteHeader call from github.com/chrislusf/seaweedfs/weed/server.processRangeRequest (common.go:280)
You are using too old a version, the new version does not have this error
We are in the process of upgrading seaweeds to a newer version but it will take some time, meanwhile can you suggest any solution to fix this issue...?