I've recreated the approved / unapproved feeds, initially the downloads for all packages is in the feed dev-pypi (unapproved feed that's hooked the PyPI connector) were correctly showing as 0

However after promoting 1 package from dev-pypi to prod-pypi all of the downloads for all packages are now showing -2 in the dev feed. The prod feed (which the package was promoted to) is just showing at one version, and correctly showing 0 downloads, however everything seems to be doubled up;

When I look at local packages under prod-pypi after promoting requests==2.31.0 from dev-pypi to prod-pypi, I can see 2 local copies of the same package promoted at the exact same time.

Likewise, the dev feed dev-pypi is showing the same package cached twice

Not sure if this behavior is contributing to the strange negative downloads in any way.

EDIT:

It seems to have started to happen again even on the newly created feeds, I've not interacted with these feeds in anyway aside from promoting a single package from the dev feed to the prod feed - the below example of ngrok shows both -6 and -10 downloads.

Hello,

I've been testing the functionality for reporting package download statistics based on feed downloads. I've noticed something strange, all versions of all packages (that have never been downloaded before) show as having either -1 or -2 downloads, some examples below of random packages that I've never used;

Other packages it gets even stranger, this example below shows numpy, I've never pulled any of the visible versions (which show -25, -28 or -32 downloads) - when I view the full version list, numpy shows varying levels of negative downloads for all 50+ versions of the package.

If I dig deeper into one of these versions (the example below shows numpy==1.26.0 which shows -32 downloads), you can see that they've never been pulled from the feed;

Just wondering what the best course of action is to debug this further?

Thanks!

Hey @Dan_Woolf - thanks for your response,

I figured out the issue with the LDAP queries, my LastPass chrome extension was trying to be helpful and it was populating the LDAP Overrides field User name Property Value with the name of my service account, this was causing all LDAP queries to break down after I navigate to the LDAP overrides

This is now fixed and working! Thanks for the details on the SAML claim, I tried the transform as suggested (ExtractMailPrefix(user.userprincipalname)) - however I still couldn't get this to work in our environment, as the AD group synced in Proget via LDAP was expecting usernames in username@company.local format, but the transform stripped the UPN entirely (so the format was just username) which meant that the mapping still didn't match up.

Since our primary UPN suffix is company.com (but the domain is called company.local) I was able to get around this by instead using a join transform to combine the attributes sAMAccountName and DNSDomainName with an @ separator.

The approach seems a little hacky, but it worked :)

Thanks!

Hi @atripp,

Thanks for your reply, that's worked! I was able to successfully populate the administers role with a group from our AD domain;

However I ran into another issue, the user membership for this group was being resolves as a different user, as the UPN suffix for SAML logons is a routable public domain (user@company.com), whereas the UPN suffix being presented for the user was shown as the domain name (user@company.local) - this meant that the role mappings didn't match up and logins via SAML had no permissions despite the underlying user being a member of the group.

We do present (user@company.com) as the primary UPN suffix within Active Directory, so I think I just need to have a play around with some of the advanced LDAP query settings to get this to match up properly on the proget side - however I have strangely hit another issue.

This was initially working as you can see, I've been able to successfully detect and map the group Proget - Role -- Administrators from AD to the Administer role in Proget in the first screenshot - however I was tweaking some of the advanced LDAP settings and my queries suddenly stopped working. I am now not able to run LDAP queries at all through Proget, I am just getting a principal not found error;

I've fully deleted and re-created the LDAP configuration as it was when it worked 30mins ago, however I just can't get the queries to run anymore. I've confirmed that;

1. The service account being used for LDAP is unlocked and the password Proget is using is correct
2. The Proget container can access our domain via FQDN, and LDAP/LDAPS to all routable domain controllers is working correctly

I am a bit stuck on what to try next, do you have any suggestions, or is there any way to enable more verbose logging to see whats going on on the Proget side?

Thanks!

Hello,

I am running Proget via docker compose using your provided container image. I am looking at what possibilities I have for configuring AD integrated access control for our end users.

I've configured SAML via an Azure Enterprise Application - this is working great, but its a pretty clunky implementation, I had a few questions around SAML and your supported authentication methods as a whole;

1. In order to assign roles/permissions to users, they must first sign-in via the SSO flow to populate their identity on the Proget side - only them am I able to assign them a role. Ideally you'd support JIT provisioning but I cannot see this anywhere on your docs, is there any method supported for prepopulating SAML ussers?

2. I don't seem to be able to expose groups to Proget for SAML authentication, in an ideal world I would have a group called "Proget Administrators" in Active directory. I could assign all admin users AD accounts to this group, and then give this group access to login to the Enterprise Application in Azure. Is there any method to do this?

3. I have the option to configure an LDAP connector (under User Directories / Domains) - I've been able to successfully set this up within my Proget container, and can query users and groups within our AD domain forest. However, I don't seem to be able to do anything else with it? It looks like configuration can only be used with Windows Integrated Authentication, which cannot be setup on your container image as its not Windows-based. Am I missing something here?

Thanks in advance!

Hi @atripp,

Thanks for the clarification! That has resolved the issue :)

Strangely, the connector already had the box ticked for exact matching;

However, after unticking it, saving, then re-ticking it - I am now able to correctly identify packages when searching for their names from attached feeds

Thanks!

Morning @atripp,

I've updated to the latest available version of Proget, recreated the PyPI connector and setup a new feed. It still seems to be an issues - Proget will seemingly index a handful of packages from PyPI (see the below screenshot) which are visible straight away;

But when attempting to search for any other common public packages that exist on PyPI, they are not displayed;

As with before, a pip install against the index will work successfully, and then the installed package will be correctly populated in the Proget feed.

I'd appreciate if you'd be able to take a closer look on what's going on in the background, as I suspect something might not be working as intended! Please let me know if you need any further details from me

Thanks

@atripp Hi Alana, thanks for your response - I'll give this a try again tomorrow and let you know how it looks!

Hello,

I am trying to test the functionality of a package approval workflow for PyPI packages. I've setup an unapproved and approved feed (via the setup wizard "Yes, Create Two Feeds") which has generated both feeds, along with the underlying connector to PyPI.org.

The feeds created are;

unapproved-pypi (set to source packages from PyPI public index)
public-pypi (promoted packages only)

The connector is showing healthy and has detected ~486k packages on the public index.

I am trying to work out the process on how to promote packages from the unapproved feed (unapproved-pypi) to the approved feed (public-pypi) - the problem I have at the moment is that if a package is not pulled from this feed, it is seemingly unavailable in the UI, which means I cannot promote it without first pulling it via pip. Please see below my example of this behavior;

1. Attempting to search for the package numpy under the feed "unapproved-pypi" returns no results;

2. Running a pip install against the feed unapproved-pypi will pull numpy from PyPI public index via the connector.

pip install numpy --index-url=https://{Redacted_URL}/pypi/unapproved-pypi/simple
Looking in indexes: https://{Redacted_URL}/pypi/unapproved-pypi/simple
Collecting numpy
  Downloading https://{Redacted_URL}/pypi/unapproved-pypi/download/numpy/1.24.4/numpy-1.24.4-cp38-cp38-win_amd64.whl (14.9 MB)
     |████████████████████████████████| 14.9 MB ...
Installing collected packages: numpy
Successfully installed numpy-1.24.4

3. I am now able to search the unapproved-pypi feed and can see numpy as an available package.

4. Only after doing the above, am I able to successfully promote the package from the unapproved feed to an approved feed
   

I wanted to check is this the intended method of using approval feeds for this use case, it seems a bit cumbersome and counterintuitive, am I missing something here?

Thanks

Thanks for your response - great to hear!

Would it also be possible to add support for PyPI connectors to package indexes that are not based on /simple please?

An example of this would be the package index hosted on https://www.haver.com/Python - where the service is hosting a PEP503 compliant simple repository, but the endpoint is /Python

Thanks!