Thank you @atripp !

GET requests on https://our.proget.com/v2/qa-feed-1/library/nginx/manifest/latest with header Accept: application/vnd.docker.distribution.manifest.v1+json return a payload of type application/vnd.docker.distribution.manifest.v2+json which doesn’t contain the data we are expecting (i.e. we can’t explore the layers of the manifest through the requested version of this manifest).

See screenshot:

Note: Regarding requested URL: /api/registries/1 – part of the URL is only a backend proxy, passing the request as-is to the Proget API – basically transforming http://localhost:9000/api/registries/1/v2/… to https://our.proget.com/v2/…

This issue was mentioned on forum topic 3541, but we thought it best to just log as a separate topic.

Hi @atripp

We loaded your SQL Server script and tested and it did resolve the DELETE + re-INSERT issue!
So, if you do include that in your next patch release, that'd work - thank you.

Regarding this:

Is this a Docker API limitation? We were thinking of adding a API for tag management for our own CI/CD platform (BuildMaster), but just worked-around it, and no one ever asked.

Yes, it is a Docker API limitation.
It might be a good idea to have tag management API endpoints but it's not something we need specifically, so I wouldn't worry.

Thank you
James

Hi @atripp

We are using ProGet Version 5.3.28 (Build 16), and calling the API directly.

Usage scenario: removing or renaming tags for a manifest. As there is no specific route for this use case, we are DELETE-ing the manifest through the v2 docker API and re-adding (with PUT) the new/remaining tags.

Basically, we are batching (so, executing in parallel) PUT requests to the URL with the format:

https://our.proget.com/v2/<feed>/<repository>/manifest/<tag>

e.g. with the following values:
• feed: qa-feed-1
• repository: library/nginx
• tag: in [latest, 1, 2, 3, test, test2]

we are concurrently executing 6 PUT requests:

PUT https://our.proget.com/v2/qa-feed-1/library/nginx/manifest/latest
PUT https://our.proget.com/v2/qa-feed-1/library/nginx/manifest/1
PUT https://our.proget.com/v2/qa-feed-1/library/nginx/manifest/2
PUT https://our.proget.com/v2/qa-feed-1/library/nginx/manifest/3
PUT https://our.proget.com/v2/qa-feed-1/library/nginx/manifest/test
PUT https://our.proget.com/v2/qa-feed-1/library/nginx/manifest/test2

In general, all of them return with a code 201, but one (random one) returns with a 500. Sometimes, all PUTs return with a code 201 (and that’s good as it is what we expect).

Response payload on HTTP 500:

{
   "errors":[
      {
         "code":"UNKNOWN",
         "message":"Violation of PRIMARY KEY constraint 'PK__DockerBlobIndex'. Cannot insert duplicate key in object 'dbo.DockerImageLayers'. The duplicate key value is (107, 1).\r\nThe statement has been terminated.",
         "detail":[]
      }
   ]
}

Our test images are the official nginx and hello-world.

I can forward a video we've taken of this in action - let me know an email address to send to (as I can't attach the mp4 here).

And here is a screenshot from the video, recapping the DELETE/PUT requests that occurred during the video:

Screenshot caption (use waterfall for request groups):

• DELETE + concurrent PUTs = “retag” action or “remove a single tag” action
• Single PUT = create new tag for manifest (top-right form of the page)

Also, sometimes only PUT-ing a single manifest (same URL as above) after a
DELETE on https://our.proget.com/v2/qa-feed-1/library/nginx/manifest/sha:xxxxx
results in the PUT returning HTTP 500.
This occurs more often with larger images, but not every time.

As a side note (unrelated to this error), GET requests on https://our.proget.com/v2/qa-feed-1/library/nginx/manifest/latest with header Accept: application/vnd.docker.distribution.manifest.v1+json return a payload of type application/vnd.docker.distribution.manifest.v2+json which doesn’t contain the data we are expecting (e.g. we can’t explore the layers of the manifest through the requested version of this manifest).

See screenshot:

Note: Regarding requested URL: /api/registries/1 part of the URL is only a backend proxy, passing the request as-is to the Proget API – basically transforming http://localhost:9000/api/registries/1/v2/… to https://our.proget.com/v2/…

Note: This occurs when sending a request to

PUT /v2/<name>/manifests/<reference>

As per https://docs.docker.com/registry/spec/api/#pushing-an-image-manifest

Proget gives the following error when I re-add an image that has multiple Docker image layers:

Violation of PRIMARY KEY constraint 'PK__DockerBlobIndex'. Cannot insert duplicate key in object 'dbo.DockerImageLayers'. The duplicate key value is (47, 1).\r\n

Any idea what is causing this?