ProGet: Docker image not showing all layers

[dlundqvist]

Hi,

Thanks for clarifying the differences.

But still, ProGet does not show the all information that docker image history shows, it's cut off two-thirds in. As you saw from above, there are 36 lines to the CLI output, but ProGet only shows the 24 first of those:

Best regards,
Daniel

[atripp]

Hi @daniel-lundqvist_1790 ,

The Image Layers and Image History are two different things.

I'm not sure how to better explain this... but ProGet is not designed to show you the "Image History" -- if you want that, then use the docker image history command.

Instead, ProGet's "Layers" page shows the Image Layers.

It's confusing because these things are similar, but they are in fact different.

Thanks,
Alana

[dlundqvist]

Hi,

I understand that. But again, there are layers after what ProGet shows. Regardless if ProGet only shows layers or not, it's missing the latter layers from the image. There are about 7 or 8 after the last one from the screenshot.

Thank you for your patience,
Daniel

[atripp]

Hi @daniel-lundqvist_1790,

Can you paste in the manifest file here? It's JSON and it's going to be on the "Metadata" page in a textbox

Thanks,
Alana

[dlundqvist]

Manifest:

{
   "schemaVersion": 2,
   "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
   "config": {
      "mediaType": "application/vnd.docker.container.image.v1+json",
      "size": 14147,
      "digest": "sha256:ab4eca7160c7cbb7ca656fc567ff13eed767066b4ea662ceea31b9ce8e44f2ad"
   },
   "layers": [
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 79078981,
         "digest": "sha256:e60c9fe2676d1fc11442d4ccd62ed958bdeb07c2a637cbdc10e2eef235b66814"
      },
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 22354231,
         "digest": "sha256:c061909f4a7a21b0ffe2c5fcc606ce8d0106b461ec7b1c2f414cdb50e061e2d3"
      },
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 3817,
         "digest": "sha256:8de8dab7a9d012b3e23f5afa40b62627f2c263be798552b855b3a3630950cc3f"
      },
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 32,
         "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1"
      },
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 4430,
         "digest": "sha256:aa3ef9e322efe6117e81bc3d5e90858570d330d12bbb4d83f8bf494b97282634"
      },
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 21657612,
         "digest": "sha256:ceeb0cf101bb99324b382ffa5bcf6c9d8b457f1c139e1ba29996366722a5711f"
      },
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 1379,
         "digest": "sha256:bff29e0a6e38370cd786eb41e5b804651fc004d6981526e0ba2eb2ced51cff37"
      },
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 945288,
         "digest": "sha256:5549a014dacbbe1fb9703b00faed6ce3e39b7c83bafaee5e13f56b24e38899e5"
      },
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 325129588,
         "digest": "sha256:e594dc179bfc37e9c8c746b7c303651c5472dd6702f9aac03e4babdff540531e"
      },
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 32,
         "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1"
      },
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 32,
         "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1"
      },
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 1238929,
         "digest": "sha256:6897a98c90464f6521787c0a5410533790d266c63c732afa860b2d5fab2720b2"
      },
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 16767228,
         "digest": "sha256:d69ae6da49ed40a5c9cf51194ea5f4f7d18de50e85a0b9d4da276785cadd0626"
      },
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 8906037,
         "digest": "sha256:982da0f0876345605ae557d61a92ee45399553bd0a5806b8c3e62548df32e002"
      },
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 130,
         "digest": "sha256:d7a1da484429e49793b180eeef361b16534a7f7bbe4b00067fa0397bc1f92fad"
      },
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 4940215,
         "digest": "sha256:4ee7dcf865d60d11363aa61338a4aa22116540d3b87ea087730419922c22e68e"
      },
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 16399337,
         "digest": "sha256:4529bcd3ea1bf888fce1d9d7887243ce879e69408e65ce764a99860eef9ae68f"
      },
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 92,
         "digest": "sha256:b33640e583ab6f826c3dcb7956be9a4cad8f9b1eb8c83594518b8fb67e6c5281"
      },
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 1842985,
         "digest": "sha256:95c6edbc8b63924db8268f4b698fa82daa10af2d6d81020fc29c3a2ad7418222"
      },
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 11610611,
         "digest": "sha256:f4340cf7e51f815647974cc1771b2dffa77e0f5eba4a57721bc05e9af6c6ad57"
      },
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 32,
         "digest": "sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1"
      },
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 25434,
         "digest": "sha256:80d412a28329389087d676a767297becd692057d56043d1019af006c430fac7e"
      },
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 43891830,
         "digest": "sha256:a6a36d1c2e9abece59b383c3f93c39d060a48ae4a9d3047408bc05aa09c9c5b1"
      },
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 13829131,
         "digest": "sha256:b7593e740f79fdea31249ff3717f4fe50dc1925ca728bb423e56e12e2e1b4b6e"
      }
   ]
}

I could also provide the configuration JSON, but rather send that outside of forum.

// Daniel

[atripp]

Hi @daniel-lundqvist_1790 ,

Thanks for sharing this; as you can see from the manifest file you shared, there are 24 layers in your image. These are the 24 layers that ProGet is showing on the Layers tab.

We can clearly see that there are no additional layers; the 24th layer that ProGet is showing is indeed the last (24th) entry in the manifest file:

      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 13829131,
         "digest": "sha256:b7593e740f79fdea31249ff3717f4fe50dc1925ca728bb423e56e12e2e1b4b6e"
      }

The docker image history command does not show layers. It shows image history, which is orthogonal to layers. ProGet is not designed to show the image history, only the layers.

If you were to look at your Container Configuration File, under the "history" property, you will see the exact number of items that docker image history shows`. The command is simply giving a user-friendly printout of that file.

Hope that helps clarify,
Alana

[dlundqvist]

Hi,

I understand that docker image history does not show layers per se, but layers are show in a way. It's correct that there are 24 layers and ProGet does show 24 rows. But the commands shown for some/most layers are wrong, it's as if it does not skip non-layer generating commands. There are several commands in our Dockerfile that sets environment variables for following RUN commands.

Here's part of command from Dockerfile that generates layer 24:
RUN /bin/sh -c : && tar -C /tmp -xf /tmp/...

And here's what ProGet show for layer 24:
RUN |2 proxy_host= proxy_port=8888 /bin/sh -c...

The command that ProGet show is used to generate a layer earlier in the Dockerfile.

Best regards,
Daniel

[atripp]

Hi @daniel-lundqvist_1790 ,

Thanks, now I understand what the issue is: basically the command history isn't lining up to the layers. This is a minor UI/display thing.

I can fix that fairly easy by grouping history entries marked with empty_layer into the following command. Here is what it looks like when grouped (ignore red warnings, it's dev environment):

Assuming the history entries are accurate, the bottom-most entry should be the one that generated the layer.

This will be in the next maintenance release (friday) via PG-2848.

Thanks,
Alana

[dlundqvist]

Hi,

Great, thanks! Apologies for not able to describe the problem better.

Best regards,
Daniel

[atripp]

@daniel-lundqvist_1790 no problem.. docker is really confusing/silly... , glad we got it solved :)