jq: error (at <stdin>:110): Cannot iterate over null (null)

[Xavi]

Hello,
I have a registry v2 and docker engine v1.10.

Removing only a tag I recieve this error:
[root@dregistry01 v2]# delete_docker_registry_image --image gitlab:latest
jq: error (at <stdin>:110): Cannot iterate over null (null)

Looking the code I see this jq command
jq -r '.fsLayers | map(.blobSum)[]'

But the file where is trying to filter the data have this format:
.....
"layers": [
{
"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
"size": 65687630,
"digest": "sha256:a64038a0eeaa782fedb5762a37742005751e88406836d67375c1d7a438f5940e" },

I tried changing the line with this and now seems that is working
jq -r '.layers | map(.digest)[]'

It may be possible that has changed the format of the blob files?
data.txt

[Kevin Burnett]

yeah, i think you're right about the format of those blobs changing. if you coded up a check for "schemaVersion": 2 and then did what you proposed in that case you would be my hero. here's what one of my old and busted (schema version 1) blobs looks like:

}user@awesomeserver00:/opt/registry_data/docker/registry/v2$ head blobs/sha256/fe/fef363e9fa0514843aca85ace61c585c745f239aa1477f0ca0fe2d1d2072bb4c/data
{
   "schemaVersion": 1,
  "etc"....

[Xavi]

Hi, I created a new fork and added the function "check_blob_version". If you want to take a look: https://github.com/xesteve/delete-docker-registry-image
Probably is not the most elegant way to do it, if you want to change anything no problem.

The function check the data file and depending the version, define what strings have to use the jq command. I changed the command to:
jq -r "$LAYERS | map($MAP)[]"
The $LAYERS and $MAP variables are set depending the version.

I only tried with real version 2 files and simulated version 1 files and it looks fine, but if you can try with your files.

[Alexander Bulimov]

In rewritten python version supporting new metadata is even simpler:

We can just check 'schemaVersion' in get_layers_from_blob

def get_layers_from_blob(path):
    """parse json blob and get set of layer digests"""
    with open(path, "r") as blob:
        data_raw = blob.read()
        data = json.loads(data_raw)
        if data["schemaVersion"] == 1:
            result = set([entry["blobSum"].split(":")[1] for entry in data["fsLayers"]])
        else:
            result = set([entry["digest"].split(":")[1] for entry in data["layers"]])
        return result

But the problem is that tests are failing, new Docker 1.10 storage schema seems to use some more layers, as after deleting image by tag some layers are not deleted - in your modified bash version as well.

[Kevin Burnett]

Thanks for the fix, @xesteve , and python port of the fix, @abulimov . I applied it in 050406e. There are now comments in the README about those failing tests.