Feature Request: Evaluate package based on policies before they are pulled with connector

[gisleso]

Hello

I think this is a feature request on how connectors work. I'm using PyPi.org connector as an example here but I guess this can apply to many if not all public registries.

What would be awesome to see is if the connector could evalute if a remote package is compliant or not with the feeds policies, and if not try, another package version (depending on what violates the policy). E.g. newly published packages are blocked for two days, instead of always trying the latest version it would be check the date on PyPi.org and get a package that is compliant.

I know we can target a specific version in pip install, but for a big dependency graph that is not so user friendly to manually install a package dependency each time we get a error.

[dean-houston]

Hi @gisleso ,

What you're describing has come up lately, though with a slightly different angle.

I'm going to quote a variant of this, from a recent support ticket.

We are trying to establish a policy where packages newer than 30 days are blocked on the package server level.

Unfortunately, we are unable to use the feature they way it currently works, unless we are missing something.

As soon as we block packages in any capacity, we are unable to do even the most rudimentary pip install or uv sync operations, as transitive dependencies cannot be downloaded (same for npm actually, but that is not an important package manager for us).

I assume it has to do with how these package managers generally work:

1. They construct a dependency graph from the available versions, taking various rules like locks and version ranges into account.
2. After they built the graph, they will try and download the versions they have selected, which fails in most cases on ProGet, because the versions that are advertised by the index may not be available.

We're a bit surprised by this behavior, because it seems like versions that cannot possibly be downloaded should not be present in the index.

We think ProGet should not advertise versions in the index that it blocks.

There are options with some package managers to take a cut-off date into account when calculating the dependency graph, but defining that in code on every single piece of code we have throughout the org defeats the purpose of having a central repository that should be able to do that.

Are we missing something? Can we hide the versions that are noncompliant from the PyPI package index, or could that be enabled? This is a rather critical feature for us.

And for more context, here's some of our response:

This is a somewhat frustrating technical limitation; it comes down to the fact that we often can't know if a package is Noncompliant until we have the full metadata and even the package file.

The API examples you shared illustrate this exact problem; consider what happens when you make that same request to PyPi.org (i.e. curl https://pypi.org/simple/certifi/). It's a long list of versions of that package, but there's no additional metadata.

That means ProGet wouldn't be able to filter that list without querying the metadata of each version -- and that's just not technically feasible to do. Some clients can actually be configured to do that (like pnpm), but they can take a more context-sensitive approach. For example, they'll start by downloading B-1.2; then, if it's too new, it downloads B-1.1, and so on. Obviously a server can't know what the client wants with the data.

Anyway, this is a known issue it's something we're researching. We call it "Metadata Suppression", but there are a few other major Developer Experience issues as well. The biggest one is that there's simply just no indication that these packages are being hidden by ProGet. So a Package Restore through ProGet will behave drastically different to the public repository, and eventually that will just lead to developers not trusting or using ProGet due to erratic behavior. That's a not a situation either of us (i.e. us as the software vendor, you as the platform ops team) want.

Our general guidance is to simply not block downloads at all actually; we consider that an anti-pattern. It adds no protection whatsoever while substantially increasing risk by lowering the organizational security posture.

Quite simply, developers will work-around what they consider "braindead policies" to get their job done. Upper management will not intervene nor punish them for delivering on time, etc. Ultimately the only consequence will be that no one will take your team (i.e. those enforcing policies) seriously. We've seen this happen time and time again; it's a version of "shadow IT".

Instead, you should use pgutil builds scan and then consider these on a case-by-case basis if they are issues: https://guides.inedo.com/vulnerability-management/containment/

To summarize, the "version dance" isn't realistic to do at the server level, for the reasons articulated above. Even if it were technically feasible, It would create a poor developer experience.

THAT SAID -- we are considering an experimental approach. ProGet could "auto-yank" noncompliant packages. I'm envisioning something with three options for Noncomplaint packages:

• Block File Download (not recommended)
• Emit Deprecated in the API (experimental)
• Allow Downloads (default)

Will clients like pip and uv avoid deprecated, newer packages? We aren't users of these tools (or python at all), and we'd love if you could help us "test" this by manually deprecating some newer versions, and see how it handles dependency resoltuion.

-- Dean

[denis.krienbuehl_4885]

I think adding a yanked flag is a good approach when it comes to PyPI. It is how packages are "deleted" from that index anyway, and package managers do have to deal with that on a regular basis. I'm happy to give that a try on our end.

I think purely for the "let's not show the users packages newer than x days", newer packages could also be entirely omitted. This should not break anything if it was applied this way:

• Stop showing packages newer than the moment the policy is enabled.
• Wait x amount of days.
• Start showing packages uploaded x days after the moment the policy was enabled.

Essentially ProGet would delay everything it pulls in from PyPI. Maybe that could be an option on the connector.

This doesn't work for "noncompliance" in general, and I don't know how well it translates to other package indices.