Can't pull docker image from remote connector

pariv_0352

I have a feed with remote connector to docker hub. It was working ok for a long time. But recently I discovered that I can't pull new published images from docker hub through connector.
If I delete cached images, and execute, for example, docker pull our_proget_url/feed_name/mongo:5.0
then answer will be Error response from daemon: manifest for our_proget_url/feed_name/mongo:5.0 not found: manifest unknown: manifest unknown.
I even can search for that image in proget web interface and load and see all remote tags, but can't pull it.

Dan_Woolf

Hi @pariv_0352,

I have a few questions to get us started.

What version of ProGet are you using?
When did this start happening?
What is your Docker Hub connector URL
Does it work if you try to pull using docker pull {our_proget_url}/{feed_name}/library/mongo:5.0 (please not the inclusion of the library namespace)?

Thanks,
Dan

pariv_0352

I use 2022.1 version of proget.
I noticed it just before upgrading proget to a new version, because I use proget to retrieve new image versions of proget )
But I was hoping that after upgrade all will work like always, so I got proget image direct from inedo repository
Docker Hub connector URL is https://registry.hub.docker.com
No it still fails with library and without with same response.

I have multiple other connectors to other public registries, and all of them stopped to work like this.

pariv_0352

For test purposes I created clean install of proget to other pc using docker.
Created one feed proxy with connector to docker hub with url https://registry.hub.docker.com

Tried to search for mongo in web interface - no results
Then I tried to run docker pull {proget_url}/proxy/library/mongo:5.0
docker response is Error response from daemon: manifest for {proget_url}/proxy/library/mongo:5.0 not found: manifest unknown: manifest unknown

docker container logs:

proget   | info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
proget   |       Request starting HTTP/1.1 GET http://{proget_url}/v2/_auth?scope=repository:proxy/mongo:pull&service={proget_url} - -
proget   | info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
proget   |       Request finished HTTP/1.1 GET http://{proget_url}/v2/_auth?scope=repository:proxy/mongo:pull&service={proget_url} - - - 200 21 application/json 0.4792ms
proget   | info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
proget   |       Request starting HTTP/1.1 HEAD http://{proget_url}/v2/proxy/mongo/manifests/5.0 - -
proget   | Running Drop Path Monitor...
proget   | Running Drop Path Monitor...
proget   | Getting list of feeds with a drop path...
proget   | Getting list of feeds with a drop path...
proget   | No feeds with a drop path.
proget   | No feeds with a drop path.
proget   | Drop Path Monitor completed.
proget   | Drop Path Monitor completed.
proget   | Running Execution Dispatcher...
proget   | Running Execution Dispatcher...
proget   | Checking for executions to dispatch...
proget   | Checking for executions to dispatch...
proget   | Execution Dispatcher completed.
proget   | Execution Dispatcher completed.
proget   | info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
proget   |       Request finished HTTP/1.1 HEAD http://{proget_url}/v2/proxy/mongo/manifests/5.0 - - - 404 81 application/json 8167.3262ms
proget   | info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
proget   |       Request starting HTTP/1.1 GET http://{proget_url}/v2/proxy/mongo/manifests/5.0 - -
proget   | info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
proget   |       Request finished HTTP/1.1 GET http://{proget_url}/v2/proxy/mongo/manifests/5.0 - - - 404 81 application/json 1993.6862ms

After that request I can search for mongo in web interface and there will be result with that name. And I can retrieve all remote tags for that image. But that's all I can do with it.

I thought that may be something wrong with our network, but this clean install I created on different network and provider.

rhessinger

Hi @pariv_0352,

Thanks for all the information. I have been able to recreate the issue and I'm currently looking into it. I'll have an update for you soon.

Thanks,
Rich

rhessinger

Hi @pariv_0352,

I was able to fix the issue, PG-2160, and it will be released next week on ProGet 2022.2. If you would like to use the fix earlier, you can install a pre-release version of PrGet by installing ProGet 2022.0.2-CI.7 or higher. We have a walk-through on how to install prerelease products in our documentation.

Please let me know if you have any questions.

Thanks,
Rich