failed extract- bucket name is being removed frpom the command?

[Arik Yakir]

Hi

i've tried as below, it fails and the output looks as the bucket name (arik-share) is missing
is it because of the tgz?

[ec2-user@ip-172-31-46-20 ~]$ s3tar --region eu-west-1 -xvf s3://arik-share/vapp-ovf/ovf.tgz -C s3://arik-share/vapp-ovf/
2023/03/16 07:29:39 object does not exist s3://vapp-ovf/ovf.tgz/vapp-ovf/ovf.tgz
[ec2-user@ip-172-31-46-20 ~]$ 

[Arik Yakir]

checked with plain tar, looks the same

[ec2-user@ip-172-31-46-20 ~]$ s3tar --region eu-west-1 -xvf s3://arik-share/IoT_demo.tar -C s3://arik-share/
2023/03/16 07:44:20 object does not exist s3://IoT_demo.tar/IoT_demo.tar
[ec2-user@ip-172-31-46-20 ~]$ 

[Arik Yakir]

now i get different output
haven't done any change

[ec2-user@ip-172-31-46-20 ~]$ s3tar --region eu-west-1 -xvf s3://arik-share/vapp-ovf/ovf.tgz -C s3://arik-share/vapp-ovf/
2023/03/16 08:06:02 unable to parse header from TAR
[ec2-user@ip-172-31-46-20 ~]$ 

[Arik Yakir]

and
seems indeed it working, but have different issue

[ec2-user@ip-172-31-46-20 ~]$ s3tar --region eu-west-1 -xvf s3://arik-share/IoT_demo.tar -C s3://arik-share/
panic: runtime error: index out of range [1] with length 1

goroutine 1 [running]:
github.com/awslabs/amazon-s3-tar-tool.extractCSVToc({0xc509b0, 0xc0001ce870}, 0x7f586d3332a8?, {0x7ffca19855a6, 0xa}, {0x7ffca19855b1, 0xc})
	/home/ec2-user/amazon-s3-tar-tool/extract.go:186 +0x4d8
github.com/awslabs/amazon-s3-tar-tool.Extract({0xc509b0, 0xc0001ce870}, 0xc0000fe580, {0x0, 0x0}, 0xc0001ca0a0)
	/home/ec2-user/amazon-s3-tar-tool/extract.go:34 +0xc5
main.main.func1(0xc000168200)
	/home/ec2-user/amazon-s3-tar-tool/cmd/s3tar/main.go:189 +0x893
github.com/urfave/cli/v2.(*Command).Run(0xc0000fe2c0, 0xc000168200, {0xc000134000, 0x7, 0x7})
	/home/ec2-user/go/pkg/mod/github.com/urfave/cli/v2@v2.25.0/command.go:273 +0xa42
github.com/urfave/cli/v2.(*App).RunContext(0xc0000003c0, {0xc50940?, 0xc000122000}, {0xc000134000, 0x7, 0x7})
	/home/ec2-user/go/pkg/mod/github.com/urfave/cli/v2@v2.25.0/app.go:332 +0x605
github.com/urfave/cli/v2.(*App).Run(...)
	/home/ec2-user/go/pkg/mod/github.com/urfave/cli/v2@v2.25.0/app.go:309
main.main()
	/home/ec2-user/amazon-s3-tar-tool/cmd/s3tar/main.go:212 +0xda7
[ec2-user@ip-172-31-46-20 ~]$ s3tar --region eu-west-1 -xvvf s3://arik-share/IoT_demo.tar -C s3://arik-share/
panic: runtime error: index out of range [1] with length 1

goroutine 1 [running]:
github.com/awslabs/amazon-s3-tar-tool.extractCSVToc({0xc509b0, 0xc0001c4c00}, 0x7f8a7e6fa3c8?, {0x7ffee5c3a5a6, 0xa}, {0x7ffee5c3a5b1, 0xc})
	/home/ec2-user/amazon-s3-tar-tool/extract.go:186 +0x4d8
github.com/awslabs/amazon-s3-tar-tool.Extract({0xc509b0, 0xc0001c4c00}, 0xc000170160, {0x0, 0x0}, 0xc0001c00a0)
	/home/ec2-user/amazon-s3-tar-tool/extract.go:34 +0xc5
main.main.func1(0xc0001610c0)
	/home/ec2-user/amazon-s3-tar-tool/cmd/s3tar/main.go:189 +0x893
github.com/urfave/cli/v2.(*Command).Run(0xc0001706e0, 0xc0001610c0, {0xc00012c000, 0x7, 0x7})
	/home/ec2-user/go/pkg/mod/github.com/urfave/cli/v2@v2.25.0/command.go:273 +0xa42
github.com/urfave/cli/v2.(*App).RunContext(0xc0001141e0, {0xc50940?, 0xc00011a000}, {0xc00012c000, 0x7, 0x7})
	/home/ec2-user/go/pkg/mod/github.com/urfave/cli/v2@v2.25.0/app.go:332 +0x605
github.com/urfave/cli/v2.(*App).Run(...)
	/home/ec2-user/go/pkg/mod/github.com/urfave/cli/v2@v2.25.0/app.go:309
main.main()
	/home/ec2-user/amazon-s3-tar-tool/cmd/s3tar/main.go:212 +0xda7
[ec2-user@ip-172-31-46-20 ~]$ 

[Yanko Bolanos]

@arikyakir when you say "checked with plain tar" what do you mean? If you want to extract files using s3tar they have to be created with s3tar, it doesn't support un-taring regular files since there is a toc.csv file at the beginning that is added by s3tar.

[Yanko Bolanos]

I updated the logging in v1.0.3, try that one, it might give us an idea of why you can't extract the file created with s3tar

[Yanko Bolanos]

The tars created with s3tar have an extra file at the beginning, a TOC that specifies the byte location and size of each file in the tar. This is what we use to extract the tar. Since existing files don't have this TOC, we don't support it.