Cannot pip install from PyPI feed connected to another feed

fbiryukov_8162

I have two python feeds: DevPyPI and PrdPyPI.
DevPyPI has a connector to PrdPyPI.
I publish mypackage on PrdPyPI.
mypackage shows up in DevPyPI (web interface) but it cannot be installed from DevPyPI using pip.

PS> pip install mypackage==1 -i https://domain.com/pypi/DevPyPI/simple --user --no-cache-dir
Looking in indexes: https://domain.com/pypi/DevPyPI/simple
Collecting mypackage==1
ERROR: Could not install packages due to an EnvironmentError: HTTPSConnectionPool(host='domain.com', port=443): Max retries exceeded with url: /pypi/DevPyPI/download/mypackage/1/mypackage-1-py3-none-any.whl (Caused by ResponseError('too many 500 error responses',))

atripp

Hello;

A 500 error should be logged in ADmin > Diagnostic Center, so if you can check there and find it, that should help us identify what could be causing it.

Is it only that package, or all packages? If it seems to be package-specific, please share the package to us (or a version that still breaks, but has sensitive information removed), then we can try to reproduce the bug and fix it.

fbiryukov_8162

Strange, I didn't receive a notification that you replied... Ah, I had to subscribe.

I just have reproduced the issue. Here's what I have in the Diagnostic Center.

First there is a warning (5 times the same).

Logged: 4/3/2020 8:57:54 AM
Level: Warning
Category: Feed
Details: none

Message:
Unable to open file mypackage/1/: 
System.IO.DirectoryNotFoundException: Could not find a part of the path 
'E:\ProGetData\Packages\.pypi\F11\mypackage\1\'.
at System.IO.__Error.WinIOError(Int32 errorCode, String 
maybeFullPath)
at System.IO.FileStream.Init(String path, FileMode mode, FileAccess 
access, Int32 rights, Boolean useRights, FileShare share, Int32 
bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String 
msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess 
access, FileShare share, Int32 bufferSize, FileOptions options)
at Inedo.IO.FileEx.Open(String fileName, FileMode fileMode, 
FileAccess fileAccess, FileShare fileShare, FileOptions fileOptions)
at 
Inedo.ProGet.Extensions.FileSystems.DirectoryFileSystem.OpenFileAsync(String
fileName, FileMode mode, FileAccess access, FileShare share, Boolean 
requireRandomAccess)
at 
Inedo.ProGet.Extensibility.PackageStores.FileSystemPackageStore.&lt;TryOpenStreamAsync&gt;d__18.MoveNext()

Then there is an error (6 times the same).

Logged: 4/3/2020 9:05:11 AM
Level: Error
Category: Web

Message:
An error occurred processing a GET request to 
https://mydomain/pypi/DevPyPI/download/mypackage/1/mypackage-1-py3-none-any.whl:
Invalid URI: The format of the URI could not be determined.

Details:
System.UriFormatException: Invalid URI: The format of the URI could not be determined.   at System.Uri.CreateThis(String uri, Boolean dontEscape, UriKind uriKind)   at System.Net.WebRequest.CreateHttp(String requestUriString)   at Inedo.ProGet.Feeds.StrongConnector.CreateWebRequestAsync(String url)   at Inedo.ProGet.Feeds.Pypi.PypiConnector.&lt;OpenPackageFileAsync&gt;d__11.MoveNext()--- End of stack trace from previous location where exception was thrown ---   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)   at Inedo.ProGet.Feeds.Pypi.PypiFeed.&lt;OpenPackageFileAsync&gt;d__20.MoveNext()--- End of stack trace from previous location where exception was thrown ---   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)   at Inedo.ProGet.WebApplication.FeedEndpoints.Pypi.PackageDownloadHandler.&lt;ProcessRequestAsync&gt;d__0.MoveNext()--- End of stack trace from previous location where exception was thrown ---   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)   at Inedo.ProGet.WebApplication.FeedEndpoints.Pypi.PypiHandler.&lt;ProcessRequestAsync&gt;d__2.MoveNext()--- End of stack trace from previous location where exception was thrown ---   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at 
Inedo.ProGet.WebApplication.FeedEndpoints.FeedEndpointHandler.FeedRequestHandler.&lt;ProcessRequestAsync&gt;d__8.MoveNext()

My package is very simple, it has no dependencies and one python module. You could try this.

setup.py

from setuptools import setup

setup(
    name='mypackage',
    version='1',
    description='My package',
    author='My Name',
    author_email='My@Email.com',
    py_modules=["mypackage"],
    zip_safe=False
)

mypackage.py

print("Hello, world!")

atripp

The first error is a known issue, it's related to browsing for packages in the UI, and it's something we've already addressed. I'm not sure about the second error.

As for the package, can you provide a package file that we can actually upload to a test instance of ProGet?

Basically I want to take a package file, then upload it from the UI, then try to download it from the UI. By not involving Python tools, we can eliminate a lot of problems and simplify finding solution.

fbiryukov_8162

@atripp
I can't upload files here (except images). Can you get them from wetransfer?
If not, see packaging and uploading python packages.

atripp

Hello; we transfer would be great. I can download it, and attach to the internal ticket for this forum post! I don't have a local python environment, so best to just get files and manually upload.

fbiryukov_8162

@atripp The link will expire in 3 days: wetransfer

atripp

Thanks, just confirming it was received, and I've attached it to the internal ticket. Stay tuned!

atripp

Hello; thanks to your package, we were able to reproduce this, and it's going to fixed as PG-1695 in the next maintenance release. Thanks!