Hi @mcascone ,

We don't have a single API method that can be used to delete all package versions from the API, but the foreach loop will do the trick!

I should add that I am doing this as the first stab at an attempt to automatically delete packages from a development feed, when the corresponding branch in github is deleted

I don't know the specifics/details of your use-case, but based on what I read, I'd recommend these guidelines:

• assuming: one GitHub repository, one project, one package you want to release
• use the same package name/group for all packages you create for this project, regardless of branch or development status
• create your "dev" packages using a prerelease version number, that has a sort of -ci.## version (assuming you use CI to build packages)
• embed the commit id and branch in your upack metadata file, for traceability
• if you want to see which branch the packages was created from using the version number alone, add a +branch metadata label to the version number for branches (don't do this for master)
• use repackaging and promotion to take your -ci packages to -rc to stable (and the desired feed)
• let retention policies automatically cleanup up the -ci packages

Hi @kichikawa_2913 ,

I think it's this way for "historic reasons" - mostly all the other feed types came later, and it seems no one ever changes these paths or noticed.

Easy enough to make it configurable, but can you share your use case? Why do you want to use something other than a single root path with all of your packages?

Anyway I added a feature for this, and we should be able to get it in the next maintenance release PG-2006

Cheers,

Steve

Hi @benjamin-soddy_9591,

No problem "resurrecting" topics! We definitely want to hear from users about feedback/feature requests.

We still haven't had anyone else ask for deprecation since this request, but I wonder if there's a better solution to solving your challenges than this feature. It sounds like you want to increase governance of your NuGet Packages, potentially with some sort of compliance in mind.

The dotnet list package --vulnerable is probably not what you want for your organization; NuGet's Built-in Vulnerability Scanning is really limited, in part because it only reports on a fraction of known package vulnerabilities (164 as of today). It also won't block packages that you deem problematic, unlike ProGet's feature.

The same is true with dotnet list package --outdated -- it's probably not what you want, because it relies on developers to have to know (1) to run the command, and (2) know what to do if there's an outdated dependency.

There are better ways to manage third-party packages (see How to Create a Package Approval Workflow for NuGet), and you'd better served knowing who's consuming outdated packages (see Use Package Consumers to Track Dependencies

Just some thoughts; like I said, we haven't had any demand for this feature, but these are proven solutions for improving governance of packages as organizations grow/expand their NuGet usage like you are.

Cheers,
Steve

Hi @kichikawa_2913 ,

The NuGet client's behavior is based on NuGet.org, where no authentication is ever required to view/download packages. As such, it doesn't pass the API key when doing those queries; instead, you can use a username of api and the password of your api key.

Based on the issue though, it sounds like ProGet is unable resolve the groups; I would use the "test privileges" function on the Tasks page to verify this. Thatw ill show you if the username can download packages or not.

The most common reason that groups aren't resolving is that the member is not directly in the group (i.e. they're in a group which is a member of the group), and you don't have recursive groups enabled; do note that this is really slow on some domains.

Cheers,
Steve

Hi @mcascone ,

Just looking at the code real quick, I suspect we have a bug where it writes out the wrong files name for the new package:
https://github.com/Inedo/upack/blob/master/src/upack/Repack.cs#L120

That's probably an easy fix, which we can do as part of this Q&A item. I'll wait to hear back about this one.

As for the error, "The underlying connection was closed: An unexpected error occurred on a send.", that sounds like it's HTTPS related. Could you attach Fiddler, or something like that, to find out what's happening under the hood? We may be able to error message to better report it if so.

Cheers,
Steve

Hi @cimen-eray1_6870 ,

Great questions; there's no problem having a second instance with ProGet Free Edition.

The relevant restriction is that you can't use a Connector in ProGet Free Edition to connect to another instance of ProGet (either another Free Edition or your paid edition).

Hopefully you can use your Maven feed as a proof of concept for implementing it in the main instance. Good luck!

Cheers,
Steve

Hi @brett-polivka,

I've added it to our Other Feed Types page, and linked this as the official discussion thread.

There's a lot of things to consider in developing a new feed type, but ultimately it all comes down to two things: (1) how much more value does this feature bring to our users, and (2) how many new licenses of ProGet would this feature sell.

The second question is where internal market research comes in, but we would love your opinion on the first question.

Here's a nice and simple way to help understand value: how much more do you suppose your company/organization would pay for this feature if it were available as a hypothetical add-on? $100/year? $1,000/year? $10,000/year? Etc. And why? What time is it saving, risk is it mitigating, etc.

The second part of the value equation is how much effort will it take, technically speaking. It's more than 15 minutes obviously, but is it 10 hours? 100 hours? Etc.

On the plus side, the package format seems to be documented pretty well. However, the registry API has a huge red flag:

The index key should be a URL to a git repository with the registry's index.

Does this mean their API is Git-based, and we'd need to first add private Git repository hosting to ProGet? And did they test it with private/authenticated Git repositories, or just their public (probably GitHub) repository?

@stefan-hakansson_8938 as you noticed, ProGet's Debian connectors are not currently designed to handle the gigantic, operating-system mirrors very well. This is because they are always refreshed "on demand" - which is what you want for CI/CD workflows.

It's not great for public repository mirroring, however. In Q4 or later, we will explore adding an option to do periodic updates.

Hi @m-webster_0049 ,

The first thing I would try is to troubleshoot this is to switch to a very basic API key like hello. That just eliminates any typos, spacing, etc.

Next, I would try specifying the API Key via X-ApiKey header (see docs) - just to see if you get a different error. It's possible there is a regression somewhere.

Best,
Steve

@joshuagilman_1054 this is currently planned for 5.3.27 as PG-1934 (April 17) - we'll let you know if plans change!

Hi @fabrice-mejean_5174 ,

Thanks for the suggestion; to help move this forward, we thought about this from a technical standpoint. We're a little hesitant to call the command pgutil packages audit since there's already an audit command for builds and vulnerabilities.

One idea is to deprecate pgutil vulnerabilities audit and call it pgutil packages audit instead.

In any case, this new command is a combination of pgutil builds audit and pgutil vulns audit.

The pgutil packages audit command would input a project, something like this:

pgutil packages audit --project=c:\projects\MyProject.csproj

Behind the scenes, pgutil would parse and POST a packageset:

POST /api/sca/audit-packages
[ 
    {
        "name": myPackage
        "version": 1.2.3
        "type": "nuget"
    },
    {
        "name": myPackage
        "version": 1.2.3
        "type": "nuget"
    }
]

The API would return an array I suppose (we don't have any samples for that, but it's a serialized BuildInfo.cs), and the end result would look like this:

$> pgutil packages audit --project=c:\projects\MyProject.csproj

Parsing MyProject.csproj... found X packages.

Azure.Core-1.35.0
 Compliance      : Compliant
 License         : MIT
 Vulnerabilities : None

Microsoft.Extensions.Configuration.EnvironmentVariables 8.0.0 
 Compliance      : Noncompliant
 License         : MIT, Apache-2.0
 Vulnerabilities : PG-123456 (High)
   the vulnerability title of this vulnerability goes here

The API could obviously contain more info. It's documented via serialized .cs classes here:
https://github.com/Inedo/pgutil/tree/thousand/Inedo.ProGet

Thanks,
Steve

@alex_6102 thanks for the additional information! That makes sense - you are effectively looking for an external authority for content verification.

As far as I'm aware, Debian is the only packaging system that support this, although though NuGet has some limited concept of "signing" for author verification. We evaluated it and it simply doesn't make sense.

That being said, if you're willing to experiment with what we have in ProGet today, we might be able to make it work. It'll require a little database "hacking".

First, you'll need a new instance of ProGet without an encryption key set-up. That's the default configuration for a Docker image.

After creating a Debian feed, take a look in the Feeds table for the FeedConfiguration_Xml column. You will find a base-64 encoded property called FeedSigningKeyRing.

Here is how we generate that:

var bundle = new PgpKeyCollection { PgpKey.Generate(4096, $"{this.FeedName}@proget") };
this.FeedConfig.FeedSigningKeyRing = bundle.Encode();

Honestly I have no idea what that does or what format it's encoded as. This is a black box from the Org.BouncyCastle.Bcpg.OpenPgp library, and we just store the bytes as you can see.

Here is how we use those bytes:

context.Response.AppendHeader("Content-Disposition", $"attachment; filename=\"{feed.FeedName}.asc\"");

var keys = new PgpKeyCollection(feed.FeedConfig.FeedSigningKeyRing);
using var output = context.GetOutputStreamWithCompression();
keys.WritePublicKeys(output);

If you're able to "replace those bytes" with a format that works, then we can add a page that allows you to specify your own. If it's just UTF8-encoded ASCII, awesome. If not, we need some kind of instructions on how to replace those bytes too.

Otherwise, this is beyond our current understanding and would require us to (re-)learn how all this stuff works, test it, etc. Which is what makes it nontrivial.

Thanks,
Steve

Hi @mickey-durden_1899 ,

Great! I've added the property as discussed and published AWS v3.1.4-RC.1 to our prerelease feed.

You can download and manually install, or update to use the prerelease feed:
https://docs.inedo.com/docs/proget/administration/extensions#manual-installation

After installing the extension, the "Disable Payload Signing" will show up on the advanced tab - and that property will be forwarded to the Put Request. In theory that will work, at least according to the one post from above.

One other thing to test would be uploading Assets (i.e. creating an Asset Directory) via the Web UI. That is the easiest way to do multi-part upload testing. I did NOT see an option to DisablePayloadSigning on the multi-part uploads, just on Put.

Thanks,
Steve

Hi @yaakov-smith_7984 , sounds like progress.... can you try to do a "group lookup" using the AD testing tool (page)? That's what the permissions page will do prior to saving.

That might give you a clue to the queries that are happening.

Hi @caterina,

It looks like Consoleman is an internal Inedo NuGet library. It's available on https://proget.inedo.com/nuget/NuGetLibraries/v3/index.json but I think we should publish to nuget.org

Will discuss with team.

Thanks,
Steve

Hi @todos-lobular7j_4769,

The built-in Configuration doesn't seem to include IPs, which means you'd want to create a PowerShell script that could read/set the configuration.

Alternatively, it could be added relatively easy as a property, assuming it's something that can be relatively easily configured with the library/code we're using:

https://github.com/Inedo/inedox-windows/blob/master/Windows/InedoExtension/Configurations/Firewall/NetFirewallRuleConfiguration.cs

This is something we could definitely help to do as a paid user, but we're not focused on Otter free/community at the moment. If you're working with someone on the sales side, I'd bring it up -- these days, the Otter sales tend to involve a lot of implementation services from our partners and a lot gets done with scripting - but if you're having success implementing on your own we could likely work together in that regard.

Cheers,
Steve

Hi @m-karing_2439 ,

The link on the "Dependencies" is provided for convenience; it's simply wrapping a <a href=... around items in the .nuspec file's Dependencies element. Without performing expensive lookups, there is no way to know if those packages are in the current feed, or if they're even valid packages or versions at all.

Keep in mind that, in many cases, dependencies won't even reference a single package (e.g. MyPackage-4.8.7) - but a whole range of versions (e.g. MyPackage-4.*). This is why it's impossible for ProGet (or any tool) to "resolve" the dependencies - it requires knowing all packages in project (plus the project's environmental configuration) to know which ranges to pick.

Note that, if those links yield a 404 then clients will also not be able to download them from thet feed.

Thanks,
Steve

Hi @yaakov-smith_7984 ,

Thanks for the additional details; so it sounds like the issue is that ProGet's Active Directory Integration does not support Active Directory group lookup using a "pre-Windows 2000" Group Name?

I'm not sure if this is something we'll want to support in the Active Directory integration. That integration is sensitive and performance-critical, so trying to add-in support for group names on domain configurations from the 1990's doesn't make a lot of sense.

For reference, here is our Active Directory Integration code:
https://github.com/Inedo/inedox-inedocore/tree/master/InedoCore/InedoExtension/UserDirectories/ActiveDirectory

Why Entra is using those ancient fields is beyond me, especially since all AD objects already have a unique ID. I suspect those are simply not intended to be queried or used by third-party tools. Maybe they're unstable and subject to change. I would try to get ahold of someone at Microsoft to get some answers on how to handle this.

That said, did you try the OpenLDAP/Generic directory? That will allow you to define your own LDAP queries. I think that's what you'll need to do.

Thanks,
Steve

@steviecoaster the main headaches come when you try to use a kind of CDN (Front Door) or anything else that will cache/manipulate/munge requests; it's often slower anyways so there's no real benefit

I know a few customers were hit with some kind of bug in Gateway that caused problems in clients (Docker particularly) - it was a long standing, multi-year bug in Gateway, so who knows if it ever got fixed

Hi @yaakov-smith_7984 ,

I'm afraid I'm not really sure what's happening; it's clear that they're doing something "pretty weird" behind the scenes, and the integration is designed to use Active Directory, not this system.

You can see the queries that ProGet is making -- and if those queries don't return those "weird" groups like $JQA200-TQZJ4H3I77X8, then it's not going to work.

Hopefully the system will work with LDAP queries. If you use the OpenLDAP/Generic directory, you can customize the property names and queries to hopefully get it working: https://docs.inedo.com/docs/installation/security-ldap-active-directory/various-ldap-openldap

Let us know what you find out,

Thanks,
Steve