[Question] Can I export local image to .tar file within podman?

Question

[Question] Can I export local image to .tar file within podman?

TyeolRik opened this issue 6 months ago · comments

Thanks for providing good program!
among 3 packages, crane, skopeo, regctl, only your program works well on my system!

Question

Can I export local image to .tar file? (I am currently using podman)

I want to preserve my Digests or RepoDigests (whatever at least one) for closed network system. I successed with below commands

# origin node
$ ./regctl image export <IMAGE_NAME>@sha256:<IMAGE_DIGEST> ./regctl.tar
$ scp ./regctl.tar <ANOTHER_NODE>

# another node
$ podman load -i regctl.tar
$ podman run -it <IMAGE_NAME>@sha256:<IMAGE_DIGEST> # SUCCESS!

But the thing is, above commands only works when online (internet connected). I am looking for export command if there is no internet connection. Is it possible?

Version

$ ./regctl version
VCSTag:     v0.5.6
VCSRef:     c848ed2ec4c23f88a7e1a96f75270ad416671eb3
VCSCommit:  c848ed2ec4c23f88a7e1a96f75270ad416671eb3
VCSState:   clean
VCSDate:    2023-12-24T14:14:41Z
Platform:   linux/amd64
GoVer:      go1.21.5
GoCompiler: gc

$ podman -v
podman version 4.6.1

Environment

Running as binary or container: binary
Host platform: x86_64, CentOS 8 Stream, podman (v4.6.1)
Registry description: nothing. only local in closed network with some tar files. (podman save -o)

Brandon Mitchell · Answer 1 · Fri Jan 05 2024 03:25:37 GMT+0800 (China Standard Time)

regctl can read/write from registries, an OCI layout, and an export to a tar file with either the OCI Layout or Docker archive format. So in this case, you would run a podman save ... to output to any of those formats (their docs describe the ability to output to multiple formats), and then either podman load to import that save on another machine, or a regctl image import or regctl image copy ocidir://$path/to/dir $dest to work with the Docker archive or OCI Layout.

Note that regclient doesn't talk directly to podman, docker, containerd, kubernetes, or any other container runtimes directly. The furthest it goes is the Docker archive tar file.

TyeolRik · Answer 2 · Fri Jan 05 2024 09:45:41 GMT+0800 (China Standard Time)

I am so sorry, but, due to my lack of English, you didn't understand what I am talking about. Please let me explain the question again.

For example, There is a situation that host1 save the image to .tar file and send to host2.

Please be aware that host2 has no internet connection. only connect available with host1.

there is the ubi 9.3 image. Its digest is 6b95efc1. From host1, the image is pulled, saved, and sent to host2.

[host1]$ podman pull registry.access.redhat.com/ubi9/ubi@sha256:6b95efc134c2af3d45472c0a2f88e6085433df058cc210abb2bb061ac4d74359
[host1]$ podman image inspect --format {{.Digest}} registry.access.redhat.com/ubi9/ubi:9.3-1361.1699548029
sha256:6b95efc134c2af3d45472c0a2f88e6085433df058cc210abb2bb061ac4d74359

# And doing podman save as you said.
[host1]$ podman save -o ubi.tar registry.access.redhat.com/ubi9/ubi@sha256:6b95efc134c2af3d45472c0a2f88e6085433df058cc210abb2bb061ac4d74359

# Send to host2
[host1]$ scp ubi.tar root@host2:/root/

host2 get the image file. podman load -i. Now the problem happened. Host2 can't run the image because digest is broken (changed).

[host2]$ podman load -i ubi.tar
Getting image source signatures
Copying blob 584953c6b22f done
Copying config 2a2c2b7af8 done
Writing manifest to image destination
Loaded image: sha256:2a2c2b7af8db0755dce3da01d17da4eef17841d6ebebdb201ecd5a88629457ce

# Now the image digest is changed. No longer sha256:6b95efc1 anymore.
[host2]$ podman images --digests
REPOSITORY                    TAG         DIGEST                                                                   IMAGE ID      CREATED      SIZE
<none>                        <none>      sha256:125e16dd5f58c761f66a603b8184b353bc3f8659be320e6972aa25cb234a8441  2a2c2b7af8db  8 weeks ago  219 MB

# So, according to changed digest, I can't use image with digest.
[host2]$ podman run -it registry.access.redhat.com/ubi9/ubi@sha256:6b95efc134c2af3d45472c0a2f88e6085433df058cc210abb2bb061ac4d74359 /bin/bash
Trying to pull registry.access.redhat.com/ubi9/ubi@sha256:6b95efc134c2af3d45472c0a2f88e6085433df058cc210abb2bb061ac4d74359...
WARN[0000] Failed, retrying in 1s ... (1/3). Error: initializing source docker://registry.access.redhat.com/ubi9/ubi@sha256:6b95efc134c2af3d45472c0a2f88e6085433df058cc210abb2bb061ac4d74359: pinging container registry registry.access.redhat.com: Get "https://registry.access.redhat.com/v2/": dial tcp 127.0.0.1:443: connect: connection refused
WARN[0001] Failed, retrying in 1s ... (2/3). Error: initializing source docker://registry.access.redhat.com/ubi9/ubi@sha256:6b95efc134c2af3d45472c0a2f88e6085433df058cc210abb2bb061ac4d74359: pinging container registry registry.access.redhat.com: Get "https://registry.access.redhat.com/v2/": dial tcp 127.0.0.1:443: connect: connection refused
WARN[0002] Failed, retrying in 1s ... (3/3). Error: initializing source docker://registry.access.redhat.com/ubi9/ubi@sha256:6b95efc134c2af3d45472c0a2f88e6085433df058cc210abb2bb061ac4d74359: pinging container registry registry.access.redhat.com: Get "https://registry.access.redhat.com/v2/": dial tcp 127.0.0.1:443: connect: connection refused
Error: initializing source docker://registry.access.redhat.com/ubi9/ubi@sha256:6b95efc134c2af3d45472c0a2f88e6085433df058cc210abb2bb061ac4d74359: pinging container registry registry.access.redhat.com: Get "https://registry.access.redhat.com/v2/": dial tcp 127.0.0.1:443: connect: connection refused

**I want to preserve the digest when copying from host1 local image to host2. **

I also tried to docker-archive or oci-archive but doesn't work. Is that because I am using not docker but podman?

[host1]$ regctl image export docker-archive:2a2c2b7af8db test.tar
WARN[0001] Failed to get manifest                        err="failed to get manifest docker.io/library/docker-archive:2a2c2b7af8db: unauthorized" ref="docker.io/library/docker-archive:2a2c2b7af8db"
failed to get manifest docker.io/library/docker-archive:2a2c2b7af8db: unauthorized
[host1]$ regctl image export oci-archive:2a2c2b7af8db test.tar
WARN[0001] Failed to get manifest                        err="failed to get manifest docker.io/library/oci-archive:2a2c2b7af8db: unauthorized" ref="docker.io/library/oci-archive:2a2c2b7af8db"
failed to get manifest docker.io/library/oci-archive:2a2c2b7af8db: unauthorized

Brandon Mitchell · Answer 3 · Sat Jan 06 2024 00:00:24 GMT+0800 (China Standard Time)

Apologies, I misinterpreted your question. I thought you were asking if regctl can talk directly to podman (which it doesn't). From the errors shown, I believe the issue is that the image is not being tagged in podman. From containers/podman#2210 I believe you can explicitly tag the image by running:

podman load -i ubi.tar registry.access.redhat.com/ubi9/ubi:9.3-1361.1699548029

There's also multiple digests here, which gets confusing. First, there's the index that holds the platform specific manifests. The index/manifest list has it's own digest that you were using:

$ regctl manifest get registry.access.redhat.com/ubi9/ubi@sha256:6b95efc134c2af3d45472c0a2f88e6085433df058cc210abb2bb061ac4d74359
Name:        registry.access.redhat.com/ubi9/ubi@sha256:6b95efc134c2af3d45472c0a2f88e6085433df058cc210abb2bb061ac4d74359
MediaType:   application/vnd.docker.distribution.manifest.list.v2+json
Digest:      sha256:6b95efc134c2af3d45472c0a2f88e6085433df058cc210abb2bb061ac4d74359
             
Manifests:   
             
  Name:      registry.access.redhat.com/ubi9/ubi@sha256:5dc85ec81a0d2cc5d19164f80b8d287b176483fd09a88426ca2f698bb2bd09de
  Digest:    sha256:5dc85ec81a0d2cc5d19164f80b8d287b176483fd09a88426ca2f698bb2bd09de
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/amd64
             
  Name:      registry.access.redhat.com/ubi9/ubi@sha256:113204124d07727303e50b2e6b87a01784180c187043f842a1af92e4f389c0e5
  Digest:    sha256:113204124d07727303e50b2e6b87a01784180c187043f842a1af92e4f389c0e5
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/arm64
             
  Name:      registry.access.redhat.com/ubi9/ubi@sha256:15098e9778f5502d928d8b0e5df356e4ffa53cb8c9356d57e12b60873f4b47b7
  Digest:    sha256:15098e9778f5502d928d8b0e5df356e4ffa53cb8c9356d57e12b60873f4b47b7
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/ppc64le
             
  Name:      registry.access.redhat.com/ubi9/ubi@sha256:8466c9efca1a9342ba174e9540cf939d0538adc46c2b93255567025a07a0f33b
  Digest:    sha256:8466c9efca1a9342ba174e9540cf939d0538adc46c2b93255567025a07a0f33b
  MediaType: application/vnd.docker.distribution.manifest.v2+json
  Platform:  linux/s390x

But when podman imports it, it may need the platform specific manifest digest, and it will display that platform's config digest in various references the image:

$ regctl manifest get registry.access.redhat.com/ubi9/ubi@sha256:6b95efc134c2af3d45472c0a2f88e6085433df058cc210abb2bb061ac4d74359 --platform linux/amd64
Name:        registry.access.redhat.com/ubi9/ubi@sha256:6b95efc134c2af3d45472c0a2f88e6085433df058cc210abb2bb061ac4d74359
MediaType:   application/vnd.docker.distribution.manifest.v2+json
Digest:      sha256:5dc85ec81a0d2cc5d19164f80b8d287b176483fd09a88426ca2f698bb2bd09de
Total Size:  78.772MB
             
Config:      
  Digest:    sha256:2a2c2b7af8db0755dce3da01d17da4eef17841d6ebebdb201ecd5a88629457ce
  MediaType: application/vnd.docker.container.image.v1+json
  Size:      6409B
             
Layers:      
             
  Digest:    sha256:b824f4b30c465e487e640bdc22e46bafd6983e4e0eabf30085cacf945c261160
  MediaType: application/vnd.docker.image.rootfs.diff.tar.gzip
  Size:      78.772MB

And the blob digest comes from the uncompressed blob, which can be seen in the rootfs section of the config:

$ regctl image config registry.access.redhat.com/ubi9/ubi@sha256:6b95efc134c2af3d45472c0a2f88e6085433df058cc210abb2bb061ac4d74359 --platform linux/amd64
...
  "rootfs": {                                     
    "type": "layers",                                                                                                                                                                                              
    "diff_ids": [        
      "sha256:584953c6b22faba1a93fb106297d108abefed3b1866099baa8cc6b0ab92bc47a"
    ]
  }, 
...

I believe podman is working similar to docker here, where only a single platform image is imported. When a single platform manifest is exported in regctl, both the OCI layout and docker archive format are included in the same tar, which may give podman the metadata it needs to automatically tag the image, e.g.:

regctl image export --platform linux/amd64 <IMAGE_NAME>@sha256:<IMAGE_DIGEST> ./regctl.tar
...
podman load -i regctl.tar

Exporting a single platform, if you don't need the multi-platform manifest, can save you a lot of download time and disk space. But realize the digest will be the platform specific digest and not the manifest list digest.

Without testing, I can't say if podman would use the image name from the docker archive, or if it prefers the OCI Layout since both are included that regctl export. If none of the above help, podman may include additional metadata it their own export that regclient can include. To see what they are creating, try the following:

# this should output the image in the ubi directory if I'm reading their documentation correctly
podman save --format oci-dir -o ubi registry.access.redhat.com/ubi9/ubi@sha256:6b95efc134c2af3d45472c0a2f88e6085433df058cc210abb2bb061ac4d74359
jq . <ubi/index.json

I'd be interested in seeing if they include any annotations in there pointing to registry.access.redhat.com/ubi9. If so, that's something I can easily add to regclient.

TyeolRik · Answer 4 · Mon Jan 08 2024 09:28:37 GMT+0800 (China Standard Time)

Thanks for your kind reply.

However, I tried as you said,

[host1]$ podman save --format oci-dir -o ubi registry.access.redhat.com/ubi9/ubi@sha256:6b95efc134c2af3d45472c0a2f88e6085433df058cc210abb2bb061ac4d74359
[host1]$ scp -r ubi host2:/root/

[host2]$ podman load -i /root/ubi
[host2]$ podman image inspect localhost/ubi:latest
[
     {
          "Id": "2a2c2b7af8db0755dce3da01d17da4eef17841d6ebebdb201ecd5a88629457ce",
          "Digest": "sha256:19cf2549663ac9e7dead59a723dfd2e2ae9f861efbe65335623094a0d2373044",
          "RepoTags": [
               "localhost/ubi:latest"
          ],

Doesn't work as I wanted.. :( Image digest always changed..

Thanks for trying to help. I think I have to do something with opening local registry for solving this problem :(