seaweedfs / seaweedfs

SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, cross-DC active-active replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

After upgrading the seaweedfs version from 0.76 to 3.64, file access on TTL volumes will return 404 not found.

monchickey opened this issue · comments

Describe the bug
After upgrading the seaweedfs version from 0.76 to 3.64, 404 not found will appear when accessing files in TTL volumes, but accessing files without TTL is normal.

System Setup

  • List the command line to start "weed master", "weed volume". In Additional context.
  • OS version: Debian 11
  • output of weed version: 0.76 and 3.64.

Expected behavior
After the upgrade, the files can be accessed normally.

Additional context
The reproduction process is as follows.

First run the 0.76 version of SeaweedFS:

./weed version
version 0.76 linux amd64
# Start master
./weed master
# Start volume
./weed volume -dir=data -mserver=localhost:9333 -ip=0.0.0.0

Then upload a file and set the TTL:

curl localhost:9333/dir/assign?ttl=3M
# Got fid: 5,0149e3da60
curl -F file=@seaweedfs.png http://localhost:8080/5,0149e3da60?ttl=3M

The file was uploaded successfully and can be accessed normally.

Then end the master and volume processes and upgrade to version 3.64 of SeaweedFS.

./weed version
version 30GB 3.64 b74e8082bac408138be99e128b8c28fd19eca7a6 linux amd64
# Start master
./weed master -ip=0.0.0.0
# Start volume
./weed volume -dir=data -mserver=localhost:9333 -ip=0.0.0.0

Now visit the url just now: http://localhost:8080/5,0149e3da60 response 404.

wget http://localhost:8080/5,0149e3da60
--2024-04-12 13:26:11--  http://localhost:8080/5,0149e3da60
Resolving localhost (localhost)... 127.0.0.1
Connecting to localhost (localhost)|127.0.0.1|:8080... connected.
HTTP request sent, awaiting response... 404 Not Found
2024-04-12 13:26:11 ERROR 404: Not Found.

Possible solutions
The relevant code snippets are as follows:

count = int(n.DataSize)
if !n.HasTtl() {
return
}
ttlMinutes := n.Ttl.Minutes()
if ttlMinutes == 0 {
return
}
if !n.HasLastModifiedDate() {
return
}
if time.Now().Before(time.Unix(0, int64(n.AppendAtNs)).Add(time.Duration(ttlMinutes) * time.Minute)) {
return
}
return -1, ErrorNotFound

The n.AppendAtNs attribute is used when determining whether it has expired. But for Version2, the n.AppendAtNs value is 0, so the file will be judged to be expired.

You can add the following judgments:

	if time.Now().Before(time.Unix(0, int64(n.AppendAtNs)).Add(time.Duration(ttlMinutes) * time.Minute)) {
		return
	}

	// add
	if v.Version() == needle.Version2 &&
		time.Now().Before(time.Unix(int64(n.LastModified), 0).Add(time.Duration(ttlMinutes)*time.Minute)) {
		return
	}

	return -1, ErrorNotFound

After compilation, replace the executable file again and you can access it normally.

Is this modification reasonable? Thanks.