Support for DELETE BLOB docker HTTP API

Alexander.Kholodov

Hello!

We need to remove docker images from the ProGet feed in accordance with our CI process. Starting with version 5.2, we were able to delete manifests via Docker HTTP Api v2 - thanks for that.

However, we still cannot delete the blobs - the HTTP method DELETE is still prohibited (StatusCode = 405).

The necessary methods are already implemented in your source code of the "DockerFeed" class. Could you add support for DELETE API for blobs (https://docs.docker.com/registry/spec/api/#delete-blob) to the ProGet HTTP controller (DockerHandler class)?

PS: we understand how the blob storage system is organized and that they may be referenced by other manifests in the registry

Thanks,
Alexander Kholodov

atripp

Hi Alexander,

We've got a few things to consider.

First and foremost, we should probably put manifest-blob relationships in the database before we even attempt this; so that adds a bit of complexity. Currently we need to scan each manifest to find these relationships.

When you say, "we know other manifests might reference it" -- does that mean you're okay with it breaking other images horribly? Or, would you expect ProGet to return "bad request" if it would do that?

But most importantly, why is this needed? Is it just to clean up/remove unused data? In this case, you can just use the existing feed cleanup task?

Thanks
Alana

Alexander.Kholodov

Hi Alana,

Thanks for the reply.

We do not expect additional logic from the ProGet, only Docker HTTP API V2 compliance. This means that ProGet does not need to think about how this blob can be used, it just needs to perform the deletion.

When processing the "DELETE /v2 /<repository>/blobs/<digest>" request, we expect that the specified blob will be deleted from the ProGet database and physical data will be deleted from the ProGet file storage.

Unfortunately, the standard feed cleanup task for our process is not enough. We want to delete Docker images and blobs pointwise, and we cannot tell ProGet which data should be deleted and at what point in time. Otherwise, data that we do not need are subject to the rules of backup (and this is a transfer of large amounts of data, long-term storage, etc.) - waste of resources.

Thanks,
Alexander.

atripp

Got it, thanks! I've logged this as PG-1632, and the idea will be to just do a "blind delete" without worrying about manifests. It's currently awaiting assessment from an engineer, hopefully it's trivial :)

Alexander.Kholodov

Great news, Alana! Thanks.