API Endpoint URL Errors: Could not establish trust relationship for the SSL/TLS secure channel.

alex.giersch_2967

We wanted to check in to get your thoughts on why our internal API endpoint URL addresses are getting caught (404 error) by the proxy:

https://<internalProGet>.<domain>/pypi/PyPI/
https://<internalProGet>.<domain>/pypi/Python/

This is our original request to our internal Exceptions team:

We need to add a Python feed to our test <internalProGet> site for testing a process for potential governance of quant Python libraries.
<internalProGetSVC>@<domain> account needs a proxy exception for https://pypi.org/

This was approved and https://pypi.org/ can be accessed (on the server as well with the svc account). But the API endpoint URLs, which we use to access the PyPi repos on our package repository site doesn’t connect. Errors from <internalProGet> logs:

Connector error: The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel.

But our other feeds, for example, do work:

External: https://registry.npmjs.org
API endpoint URL: https://<internalProGet>.<domain>/npm/npmjs.org/

External: https://www.nuget.org/api/v2
API endpoint URL: https://<internalProGet>.<domain>/nuget/nuget.org/

Just curious if you have seen this before and what we might be missing.

atripp

Hello;

Perhaps this is it? https://inedo.com/support/kb/1161/tls-v12-configuration-and-connection-errors

If not, it must be a sort of internal connection problem. ProGet does not operate at the SSL/TLS level; that's handled all by the operating system. So hopefully, the network admins can help to diagnose it some more. I've seen problems with certificate servers and trust causing a problem as well.

alex.giersch_2967

@atripp I will work with my supervisor on working this out. I will get back to you and confirm the solution.

alex.giersch_2967

We are still working on this.

When doing a "Pull to ProGet" for packages in the feed, we get this error:

There was an error pulling the package: End of Central Directory record could not be found.

We accounted for the Python MIME types as well, adding .whl and .tar and .gz to our types in IIS on the server.

And again when looking at the Connectors section on ProGet, it says PyPi is healthy and the package count is 220,935, but we can't pull, download or promote these packages.

atripp

This error means that the package file data (i.e. what is being returned by the URL that ProGet is instructed to download from) is invalid. So I'm thinking some sort of intermediary is blocking/rewriting these requests.

Sometimes, I see firewalls / proxies inspecting the contents, and then displaying "this content is blocked by corporate firewall" instead. The proxy should produce an error, but sometimes it's just 200. So, ProGet expects package data, but instead gets random HTML.

You should be able to do this.

Create PyPi Feed
Add Connector to PyPi.org
Pull package (I used girth since it was at the top of the list)

If that's not working, then something is blocking the download from pypi.org.

alex.giersch_2967

UPDATE:

Initially, we were given an exception to for <Internalsvcaccount>@<domain> account to https://pypi.org/ from our Exceptions.

But we then realized the URL hosting the actual package files is on a different domain: http://files.pythonhosted.org/

One of our developers pointed out that if you open this link: https://pypi.org/simple/ and click on any package it shows you the versions. If you hover on any of those files they are hosted here: http://files.pythonhosted.org/

That seemed to work once we were given the exception to http://files.pythonhosted.org/, but unfortunately we still seem to be having issues. For some packages when I pull them to proget from pypi.org, they seem to get pulled but there is an error which leaves proget in a bad state.

For azure-storage-blob, I can see that ProGet is pulling down the distribution file and putting it on the filesystem. But I think its not working properly as ProGet reports an error, and I can’t pull the package using PIP.

There was an error pulling the package: ProGet only supports wheels and source distribution packages.

The package count in the Python feed rose initially when we gave an exception to the files.pythonhosted.org domain, but since has dropped for some odd reason.

This error has been in our log for some time, but not specific enough to resolve (could be related):

Connector error: Input string was not in a correct format.

atripp

I don't know if it will be sufficient to provide access to only http://files.pythonhosted.org/ and https://pypi.org/ - it's very possible that the file hosting locations will change. This is the case on a lot of other package galleries, including NuGet.org.

If ProGet can only download a portion of the files, then, there will probably be some strange errors. It's too difficult to generalize, so if you can provide us with a specific package and a specific reproduction situation, we'll be happy to try it.

But, as the error says, ProGet does only support those wheel/source packages; the legacy "egg" format (10+ years old I think) is not supported. Your developers should be able to help convert it, and you can rehost the handful of "egg" packages as needed.

As far as other errors you may see.... don't think of the "Diagnostic Center" as "checklist of things to fix", it's there to help diagnose a problem... and unless you have one reported by a user, it's probably fine. They can come from so many sources (including temporary network outages, users typing in wrong urls / passwords, etc).