s3cmd (2.4.0) not preserving timestamps
griesi007 opened this issue · comments
- I am uploading to an empty bucket using sync option
- x-amz-meta-s3cmd-attrs is there with the correct modification/creation time
- ls, get, reverse sync do not preserve timestamps, instead are using the upload timestamp
Is this a bug, what am I doing wrong? The IAM user has all privileges in the policy assigned
Hi as mentioned in AWS S3 documentation you cannot preserve the last modified date of an object: https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html#AmazonS3-GetObject-response-header-LastModified
it is handled by AWS S3 itself.
Hi as mentioned in AWS S3 documentation you cannot preserve the last modified date of an object: https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html#AmazonS3-GetObject-response-header-LastModified
it is handled by AWS S3 itself.
That is not what I am talking about. I am talking about timestamps in x-amz-meta-s3cmd-attrs, which should be preserved when restoring from s3. As the s3 documentation states this should work but it doesn´t => bug
I'm struggling with the exact problem. I started out on a old version 2.1.0 and just upgraded to the latest 2.4.0-dev through the master.zip.
Ran a put/get with the following commands.
s3cmd put --preserve testfile s3://kims-test/testfile
s3cmd get --preserve s3://kims-test/testfile testfile-retrieved
Retrieved the x-amz-meta-s3cmd-attrs which looks good.
$ s3cmd info s3://kims-test/testfile
s3://kims-test/testfile (object):
...
x-amz-meta-s3cmd-attrs: atime:1715703876/ctime:1715711128/gid:1002/gname:kea/md5:3bb101bf8f11e18243ccf74886985886/mode:33188/mtime:1674469831/uid:1001/uname:kea
stat for the original and retrieved file
$ stat test*
File: testfile
Size: 2535 Blocks: 5 IO Block: 131072 regular file
Device: 30h/48d Inode: 10249679901039112730 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 1001/ kea) Gid: ( 1002/ kea)
Access: 2024-05-14 21:26:10.393650133 +0300
Modify: 2023-01-23 12:30:31.439638979 +0200
Change: 2024-05-14 21:25:28.565179979 +0300
Birth: -
File: testfile-retrieved
Size: 2535 Blocks: 5 IO Block: 131072 regular file
Device: 30h/48d Inode: 10484978917994643524 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 1001/ kea) Gid: ( 1002/ kea)
Access: 2024-05-14 18:26:11.000000000 +0300
Modify: 2024-05-14 18:26:11.000000000 +0300
Change: 2024-05-14 21:27:07.394260310 +0300
Birth: -
I ran the get with --debug, looks like the x-amz-meta-s3cmd-attrs are properly retrieved and parsed, the mtime is correct as 1674469831 but the last row in the output sets the mtime to 1715700371.
's3cmd-attrs': {'atime': '1715703876',
'ctime': '1715711128',
'gid': '1002',
'gname': 'kea',
'md5': '3bb101bf8f11e18243ccf74886985886',
'mode': '33188',
'mtime': '1674469831',
'uid': '1001',
'uname': 'kea'},
'status': 200}
^M 2535 of 2535 100% in 0s 44.03 KB/sDEBUG: ConnMan.put(): connection put back to pool (https://kims-test.s3.amazonaws.com#2)
^M 2535 of 2535 100% in 0s 43.41 KB/s done
DEBUG: ReceiveFile: Computed MD5 = 3bb101bf8f11e18243ccf74886985886
DEBUG: set mtime to 1715700371.0
Looks like this same issue has been investigated in #670 and identified as a bug to be fixed.
The mtime is properly set if you do a remote2local sync, but get will not update.
sync command
s3cmd sync --preserve --force --debug s3://kims-test/testfile testfile-retrieved
stat output
$ stat test*
File: testfile
Size: 2535 Blocks: 5 IO Block: 131072 regular file
Device: 30h/48d Inode: 10249679901039112730 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 1001/ kea) Gid: ( 1002/ kea)
Access: 2024-05-14 21:26:10.393650133 +0300
Modify: 2023-01-23 12:30:31.439638979 +0200
Change: 2024-05-14 21:25:28.565179979 +0300
Birth: -
File: testfile-retrieved
Size: 2535 Blocks: 5 IO Block: 131072 regular file
Device: 30h/48d Inode: 13426160269611585713 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 1001/ kea) Gid: ( 1002/ kea)
Access: 2024-05-14 19:24:36.000000000 +0300
Modify: 2023-01-23 12:30:31.000000000 +0200
Change: 2024-05-14 22:21:49.150610889 +0300
Birth: -