Hi @mcascone ,

We don't have a single API method that can be used to delete all package versions from the API, but the foreach loop will do the trick!

I should add that I am doing this as the first stab at an attempt to automatically delete packages from a development feed, when the corresponding branch in github is deleted

I don't know the specifics/details of your use-case, but based on what I read, I'd recommend these guidelines:

• assuming: one GitHub repository, one project, one package you want to release
• use the same package name/group for all packages you create for this project, regardless of branch or development status
• create your "dev" packages using a prerelease version number, that has a sort of -ci.## version (assuming you use CI to build packages)
• embed the commit id and branch in your upack metadata file, for traceability
• if you want to see which branch the packages was created from using the version number alone, add a +branch metadata label to the version number for branches (don't do this for master)
• use repackaging and promotion to take your -ci packages to -rc to stable (and the desired feed)
• let retention policies automatically cleanup up the -ci packages

Hi @kichikawa_2913 ,

I think it's this way for "historic reasons" - mostly all the other feed types came later, and it seems no one ever changes these paths or noticed.

Easy enough to make it configurable, but can you share your use case? Why do you want to use something other than a single root path with all of your packages?

Anyway I added a feature for this, and we should be able to get it in the next maintenance release PG-2006

Cheers,

Steve

Hi @benjamin-soddy_9591,

No problem "resurrecting" topics! We definitely want to hear from users about feedback/feature requests.

We still haven't had anyone else ask for deprecation since this request, but I wonder if there's a better solution to solving your challenges than this feature. It sounds like you want to increase governance of your NuGet Packages, potentially with some sort of compliance in mind.

The dotnet list package --vulnerable is probably not what you want for your organization; NuGet's Built-in Vulnerability Scanning is really limited, in part because it only reports on a fraction of known package vulnerabilities (164 as of today). It also won't block packages that you deem problematic, unlike ProGet's feature.

The same is true with dotnet list package --outdated -- it's probably not what you want, because it relies on developers to have to know (1) to run the command, and (2) know what to do if there's an outdated dependency.

There are better ways to manage third-party packages (see How to Create a Package Approval Workflow for NuGet), and you'd better served knowing who's consuming outdated packages (see Use Package Consumers to Track Dependencies

Just some thoughts; like I said, we haven't had any demand for this feature, but these are proven solutions for improving governance of packages as organizations grow/expand their NuGet usage like you are.

Cheers,
Steve

Hi @kichikawa_2913 ,

The NuGet client's behavior is based on NuGet.org, where no authentication is ever required to view/download packages. As such, it doesn't pass the API key when doing those queries; instead, you can use a username of api and the password of your api key.

Based on the issue though, it sounds like ProGet is unable resolve the groups; I would use the "test privileges" function on the Tasks page to verify this. Thatw ill show you if the username can download packages or not.

The most common reason that groups aren't resolving is that the member is not directly in the group (i.e. they're in a group which is a member of the group), and you don't have recursive groups enabled; do note that this is really slow on some domains.

Cheers,
Steve

Hi @mcascone ,

Just looking at the code real quick, I suspect we have a bug where it writes out the wrong files name for the new package:
https://github.com/Inedo/upack/blob/master/src/upack/Repack.cs#L120

That's probably an easy fix, which we can do as part of this Q&A item. I'll wait to hear back about this one.

As for the error, "The underlying connection was closed: An unexpected error occurred on a send.", that sounds like it's HTTPS related. Could you attach Fiddler, or something like that, to find out what's happening under the hood? We may be able to error message to better report it if so.

Cheers,
Steve

Hi @cimen-eray1_6870 ,

Great questions; there's no problem having a second instance with ProGet Free Edition.

The relevant restriction is that you can't use a Connector in ProGet Free Edition to connect to another instance of ProGet (either another Free Edition or your paid edition).

Hopefully you can use your Maven feed as a proof of concept for implementing it in the main instance. Good luck!

Cheers,
Steve

Hi @brett-polivka,

I've added it to our Other Feed Types page, and linked this as the official discussion thread.

There's a lot of things to consider in developing a new feed type, but ultimately it all comes down to two things: (1) how much more value does this feature bring to our users, and (2) how many new licenses of ProGet would this feature sell.

The second question is where internal market research comes in, but we would love your opinion on the first question.

Here's a nice and simple way to help understand value: how much more do you suppose your company/organization would pay for this feature if it were available as a hypothetical add-on? $100/year? $1,000/year? $10,000/year? Etc. And why? What time is it saving, risk is it mitigating, etc.

The second part of the value equation is how much effort will it take, technically speaking. It's more than 15 minutes obviously, but is it 10 hours? 100 hours? Etc.

On the plus side, the package format seems to be documented pretty well. However, the registry API has a huge red flag:

The index key should be a URL to a git repository with the registry's index.

Does this mean their API is Git-based, and we'd need to first add private Git repository hosting to ProGet? And did they test it with private/authenticated Git repositories, or just their public (probably GitHub) repository?

@stefan-hakansson_8938 as you noticed, ProGet's Debian connectors are not currently designed to handle the gigantic, operating-system mirrors very well. This is because they are always refreshed "on demand" - which is what you want for CI/CD workflows.

It's not great for public repository mirroring, however. In Q4 or later, we will explore adding an option to do periodic updates.

Hi @m-webster_0049 ,

The first thing I would try is to troubleshoot this is to switch to a very basic API key like hello. That just eliminates any typos, spacing, etc.

Next, I would try specifying the API Key via X-ApiKey header (see docs) - just to see if you get a different error. It's possible there is a regression somewhere.

Best,
Steve

@joshuagilman_1054 this is currently planned for 5.3.27 as PG-1934 (April 17) - we'll let you know if plans change!

Hi @inedo_1308 ,

This is likely a regression due to PG-3074 -- we'll take a look and report back!

Thanks for letting us know.

-- Dean

Hi @misael-esperanzate_5668 ,

When one of the connectors reports a 404, ProGet should move on to the next connector to search. Can you help set up a reproduction case using the ProGet UI and URLS (perhaps some basic curl commands)? You should be able to do everything with basic GET requests, and see different results.

That'll make it a lot easier to test and see if we can reproduce the issue.

Thanks,
Steve

@yaakov-smith_7984 great news!

We are planning to include it as is in the next maintenance release, later today. So it's "stable enough" we figure :)

Hi @jim-borden_4965 ,

Thanks for the feedback! We will add a note to the migration wizard via PG-3080 that clarifies that mounting the volumes are required - there's no easy way for us to tell if the user did that.

Thanks,
Steve

Hi @arose_5538 , we will have this fixed in Friday's maintenance release via PG-3077

Thanks,
Steve

Hi @m-lee_3921 ,

Sounds like you're off to a great start with Universal Packages - that's exactly what we intended them for.

Great spot on the docs. In retrospect, using the word "deprecated" was incorrect - we should have said "legacy", for the exact reason you mentioned. We intended to add those capabilities to the common packages API when we wrote the docs, but priorities...

Anyway I clarified the docs:

Legacy Universal Feed Endpoints:

The following endpoints are duplicative of the Common Packages API endpoints and should be avoided when possible:

• List Universal Packages - lists specified packages
• List Universal Package Versions - describes versions of specified packages
• Delete Universal Package - deletes a specified package
• Download Universal Package - downloads a specified package

While we don't plan on removing them in the foreseeable future, they are considered legacy. Once the Common Packages API includes metadata queries, we will likely call these endpoints "deprecated".

It wouldn't be "too hard" to add metadata queries to common packages API now that everything has been refactored... but there hasn't been any real user demand for it. And it's already easy to do with upacks as you can see.

Anyway, when folks start asking for this from other package types, we'll add it to our roadmap. For now, "legacy" is good enough.

Thanks,
Steve

@yaakov-smith_7984 good news -- we're working on a new version of the Active Directory integration that may address this problem -- want to give it a shot?

It's in our pre-release feed:
https://docs.inedo.com/docs/proget/administration/extensions#pre-releases

The extension is InedoCore 3.0.5-CI.2:
https://proget.inedo.com/feeds/PrereleaseExtensions/inedox/InedoCore/3.0.5-CI.2

We have been running it in our environment for a while.

This would be an ideal place to fix the issue, or at least add some kind of option to make it work better.

Hi @fabrice-mejean_5174 ,

Thanks for the suggestion; to help move this forward, we thought about this from a technical standpoint. We're a little hesitant to call the command pgutil packages audit since there's already an audit command for builds and vulnerabilities.

One idea is to deprecate pgutil vulnerabilities audit and call it pgutil packages audit instead.

In any case, this new command is a combination of pgutil builds audit and pgutil vulns audit.

The pgutil packages audit command would input a project, something like this:

pgutil packages audit --project=c:\projects\MyProject.csproj

Behind the scenes, pgutil would parse and POST a packageset:

POST /api/sca/audit-packages
[ 
    {
        "name": myPackage
        "version": 1.2.3
        "type": "nuget"
    },
    {
        "name": myPackage
        "version": 1.2.3
        "type": "nuget"
    }
]

The API would return an array I suppose (we don't have any samples for that, but it's a serialized BuildInfo.cs), and the end result would look like this:

$> pgutil packages audit --project=c:\projects\MyProject.csproj

Parsing MyProject.csproj... found X packages.

Azure.Core-1.35.0
 Compliance      : Compliant
 License         : MIT
 Vulnerabilities : None

Microsoft.Extensions.Configuration.EnvironmentVariables 8.0.0 
 Compliance      : Noncompliant
 License         : MIT, Apache-2.0
 Vulnerabilities : PG-123456 (High)
   the vulnerability title of this vulnerability goes here

The API could obviously contain more info. It's documented via serialized .cs classes here:
https://github.com/Inedo/pgutil/tree/thousand/Inedo.ProGet

Thanks,
Steve

@alex_6102 thanks for the additional information! That makes sense - you are effectively looking for an external authority for content verification.

As far as I'm aware, Debian is the only packaging system that support this, although though NuGet has some limited concept of "signing" for author verification. We evaluated it and it simply doesn't make sense.

That being said, if you're willing to experiment with what we have in ProGet today, we might be able to make it work. It'll require a little database "hacking".

First, you'll need a new instance of ProGet without an encryption key set-up. That's the default configuration for a Docker image.

After creating a Debian feed, take a look in the Feeds table for the FeedConfiguration_Xml column. You will find a base-64 encoded property called FeedSigningKeyRing.

Here is how we generate that:

var bundle = new PgpKeyCollection { PgpKey.Generate(4096, $"{this.FeedName}@proget") };
this.FeedConfig.FeedSigningKeyRing = bundle.Encode();

Honestly I have no idea what that does or what format it's encoded as. This is a black box from the Org.BouncyCastle.Bcpg.OpenPgp library, and we just store the bytes as you can see.

Here is how we use those bytes:

context.Response.AppendHeader("Content-Disposition", $"attachment; filename=\"{feed.FeedName}.asc\"");

var keys = new PgpKeyCollection(feed.FeedConfig.FeedSigningKeyRing);
using var output = context.GetOutputStreamWithCompression();
keys.WritePublicKeys(output);

If you're able to "replace those bytes" with a format that works, then we can add a page that allows you to specify your own. If it's just UTF8-encoded ASCII, awesome. If not, we need some kind of instructions on how to replace those bytes too.

Otherwise, this is beyond our current understanding and would require us to (re-)learn how all this stuff works, test it, etc. Which is what makes it nontrivial.

Thanks,
Steve

Hi @mickey-durden_1899 ,

Great! I've added the property as discussed and published AWS v3.1.4-RC.1 to our prerelease feed.

You can download and manually install, or update to use the prerelease feed:
https://docs.inedo.com/docs/proget/administration/extensions#manual-installation

After installing the extension, the "Disable Payload Signing" will show up on the advanced tab - and that property will be forwarded to the Put Request. In theory that will work, at least according to the one post from above.

One other thing to test would be uploading Assets (i.e. creating an Asset Directory) via the Web UI. That is the easiest way to do multi-part upload testing. I did NOT see an option to DisablePayloadSigning on the multi-part uploads, just on Put.

Thanks,
Steve