@mcascone ha! I guess I should say "Onedrive" or whatever it's called -- basically a place to version it as simply as possible.

Just FYI --- we're a pretty small team here, and had that same "gathering dust" worry. But our SOP are a collection of .txt and .ppt files in one of two places:

• "training" folder, which new staff use to follow common process (e.g. change management process)
• "program" folder, which describes a process someone is owns and is responsible or reporting/improving upon (e.g. release process)

It was a little bit of a culture shift for everyone, but now it's great to "not have to think" about the steps (just jump back to the folder), which also benefits new staff learning process (or someone covering for another person).

Hi @harald-somnes-hanssen_2204 ,

Just some random thoughts here...

ProGet has the Package Consumers feature, but that's not quite a traceable a BOM.

We largely see BuildMaster's artifacts and metadata serving as the BOM (several customers implemented it like that), though we don't necessarily call it "software bill of materials". We probably should from a marketing/positioning standpoint :)

There is no format/standard for a SBOM file, but an Asset Directory (and it's directory- and file-level metadata) could severe as such a repository. Universal Packages could as well, but I would imagine a SBOM would be like a XML or JSON file or something.

Steve

Hi @mcascone

Just to confirm, are you referring to this field?

I didn't get a chance to test (want to confirm I understand first), but the "Target Feed" field should be either:

• a be drop down of all NuGet Feeds (when "Promote To Feed" is not set)
• a dropdown of two feeds (current, and "Promote To Feed")

Is that not the case? If not let's get that fixed :)

Cheers,
Steve

Hi @mcascone ,

Thanks for this suggestion, too! Along the same vein as ProGet: Feature Request: native integration with GitHub to perform automated tasks, this is more "process control/workflow management", and not something ProGet is built for.

If you develop a Manual Package Promotions/Repackaging Process (for example, when automation doesn't make sense), then that process should be a SOP document/checklist that lives somewhere (wiki, sharepoint, etc), along side other change management processes.

A bit outside our scope, but I always recommend SOP for organizations of all sizes (even 1). Helps with everything from onboarding to consistency to process improvement. Doesn't need to be anything more than a simple .txt file on Sharepoint with some basic instructions. It can always be improved later.

BuildMaster does have this built-in for CI/CD-related processes, so a lot of SOP we see are mostly "follow the pipeline/prompts in BuildMaster", etc.

Hi @mcascone,

Thanks for the feature request; this type of orchestration is something that should happen by at the CI/CD Pipeline-level, in a tool like BuildMaster.

For example, consider our open-source Inedo.AssetDirectories .NET library (GitHub, Package).

In the corresponding Inedo.AssetDirectories BuildMaster application, when someone promotes the package to Release (see the Deployment Pipeline), the source code is tagged immediately after the promotion:

ProGet::Repack-Package
(
    PackageSource: NuGetLibraries,
    Name: Inedo.AssetDirectories,
    Version: $ReleaseNumber-rc.$BuildNumber,
    NewVersion: $ReleaseNumber
);

... snip ...

# Tag Source Code
# Tag the commit in source control using the id captured in the Build plan
{
    GitHub::Tag
    (
        From: GitHub,
        Tag: Inedo.AssetDirectories-$ReleaseNumber.$BuildNumber,
        Branch: $Branch,
        CommitHash: $CommitId
    );
}

You may be able to do this with ProGet's webhooks, but ultimately ProGet isn't an orchestrator (i.e. it doesn't have an execution/automation engine), and you won't get the level of reliability or flexibility needed.

Hi @shiv03_9800 , thanks for the detailed report on these!

We've identified these as OT-440 and OT-441, and they will be fixed in the next Otter maintenance release (3.0.14), scheduled for tomorrow.

Cheers,
Steve

It seems the package deployment endpoint is POST-only, but using the Native API you could use the Packages_GetPackageDeployments method. Down the line we can certainly consider adding GET method on the deployment api... if you can help us know what you/someone will use that info for ;)

Glad that helped @mcascone , I'll try to update the docs when I get a chance!

In older versions, that information used to be displayed more prominently, but unfortunately relatively few users seem to utilize that data, and several have moved to Package Usage. I think someone even built Usage Scanner that queried that deployment history table but we couldn't get much info beyond that...

We don't maintain that Jenkins plugin, but it is open-source and if you're comfortable enough with Java you might be able to add the required header fields?

https://github.com/jenkinsci/inedo-proget-plugin/blob/master/src/main/java/com/inedo/proget/api/ProGetApi.java#L189

But you may find upack.exe to be better, since it can "install" packages, which then maintains a history on the server itself.

Hi @mcascone ,

These are two different features...

Deployment records show which servers a package has been deployed to from that instance of ProGet, at some point in the past. They are usually added when a package is downloaded, and the GET request has a special header parameter or a user agent string.

Package usage is more complex, in that it shows which servers/hosts a package (or container) is currently installed on, regardless of whether it was deployed from ProGet or not. It requires a PackageContainerScanner component, which is intended to bridge the gap between servers and packages.

We currently have two scanners: one that interacts with Otter's API (which can return Docker, Debian, Rpm, Universal, and Chocolatey packages across your servers) and Kubernetes API (which is just Docker and Helm charts).

Hope that helps,
Steve

@mcascone yep, that's a good way of putting it!

Hi @kichikawa_2913 ,

I think it's this way for "historic reasons" - mostly all the other feed types came later, and it seems no one ever changes these paths or noticed.

Easy enough to make it configurable, but can you share your use case? Why do you want to use something other than a single root path with all of your packages?

Anyway I added a feature for this, and we should be able to get it in the next maintenance release PG-2006

Cheers,

Steve

@mcascone sounds like some great progress!

I got a little confused at the combination of "keep only last 5 versions" plus the 30-day window. From what i've read in the docs, all conditions must pass for the item to be deleted. How do i set up, "keep only the last 5 versions, but when nothing has been requested for 30 days, delete them all"?

We could definitely improve the docs on this area, and you're right all conditions must be passed for items to be deleted. When you add the "keep only the last 5 versions" rule, there will be, at a minimum 5 versions of a package.

You could be able to add a second rule, but it operates independently. More like an "OR" than an "UNLESS" I guess. Perhaps you could adjust the time-windows a bit?

• Rule 1. "Delete unused versions not requested in last 10 days." AND "Keep only last 5 versions"
• Rule 2. "Delete unused versions not requested in last 60 days."

I would look at your release cycles for guidance. For example, we release our products every two weeks, though maybe we'll skip a week every now and then. So, no -ci package will be needed past 1 month. And as you said, you can just rebuild if needed.

Hi @mcascone ,

We don't have a single API method that can be used to delete all package versions from the API, but the foreach loop will do the trick!

I should add that I am doing this as the first stab at an attempt to automatically delete packages from a development feed, when the corresponding branch in github is deleted

I don't know the specifics/details of your use-case, but based on what I read, I'd recommend these guidelines:

• assuming: one GitHub repository, one project, one package you want to release
• use the same package name/group for all packages you create for this project, regardless of branch or development status
• create your "dev" packages using a prerelease version number, that has a sort of -ci.## version (assuming you use CI to build packages)
• embed the commit id and branch in your upack metadata file, for traceability
• if you want to see which branch the packages was created from using the version number alone, add a +branch metadata label to the version number for branches (don't do this for master)
• use repackaging and promotion to take your -ci packages to -rc to stable (and the desired feed)
• let retention policies automatically cleanup up the -ci packages

Hi @brett-polivka,

I haven't tried ghcr, but in my experience GitHub is really unstable on the API/integration side when it's outside of their core source/Git hosting functions (e.g. Packages are notoriously buggy), so if your PAT is okay, then this is the most likely scenario.

The Connector Health Check for Docker uses the catalog API (/v2/_catalog), and the response should look something like this: https://proget.inedo.com/v2/_catalog. This endpoint is particularly buggy in other repositories (especially ones that require authentication/authorization), so my guess is GitHub introduced a regression or something on their implementation.

Another possibility... 403 is access related, so it could be something on your proxy side; please also check your Proxy settings, or something on your network side.

We have a lot of customers who maintain a separate test instance of ProGet; while upgrade testing is important of course, a dedicated testing instance also lets you evaluate new ProGet feature usage patterns (such as requiring promotion workflows, etc.), try out new tools (perhaps new version of visual studio, etc.), and conduct training on ProGet usage -- all without risking/disturbing your production instance.

To keep things simple from a licensing perspective, we just treat testing instances separate instances (and thus require a separate license key). Many customers use a ProGet Free License for this, but of course not all the features are available. It's rare to see a second license be cost prohibitive, especially given the labor/server costs involved with maintaining a testing instance -- even ProGet Enterprise customers will have full instances just for testing and even DR purposes.

You're right --- Active Directory is usually a pain point; sometimes our code changes (we try to never touch this), but also people want to change their AD configuration (move to LDAPS, etc.). Wrong settings, and you can lock-out your instance. If it's an uncommon / one-off testing case then a temporary trial license is fine for this.

Hi @antony-booth_1029,

What version did you upgrade from? That could help trace code changes.

Does this happen only on one page (i.e. /releases)? Then it's probably related to a bad query/unexpected data, but you only have 161 releaes according to your query. Not so much.... easy way to test that is by adding querystring params: /releases?ApplicationId=2&Status=Active for example.

If it's easy to see queryes on your RDS server, then we can see what query might be bad.

Does this happen only intermittently/randomly? If it's only you, then the problem probably isn't database/server load. And even with a ton of people, that's really rare. On very old instances with lots of retention jobs and years/gigs of data, doing an index cleanup is necessary, but I don't think that's the case here. The simplest/fastest thing I can think to do is reboot the BuildMaster server, and hope it goes away (maybe it's a weird underlying network stack thing).

Does this happen all the time (like nothing at all works on the website). Then it's probably network related?

Thanks,
Steve

@yogurtearl_0881 this is the first I've heard of Homebrew... at first glance, it looks like a kind of open-source/hobbyist/alternative package manager for MacOS?

hi @internalit_7155

Any updates on this issue? Were you able to resolve/fix this problem?

We've had another customer who reported a very similar problem (activation of a key on our new server, using sha256 vs sha1, causes an error in an old ProGet version). But we still can't reproduce it, and now I wonder if it's related to operating system version, or another operating system patch that's missing.

Thanks,
Steve

Hi @jeff-peirson_4344 ,

That's definitely what it's intended for, so I think this must be a bug...

I haven't had a chance to reproduce or look any further, but I wanted to at least share the code ASAP...

https://github.com/Inedo/pgscan/blob/master/pgscan/Program.cs#L90

... so please feel free to look/fix yourself, but we'll also take a look in the coming days as well! Just an FYI.

Thanks,
Steve

Hi @paul-reeves_6112 ,

This is by design; Configuration File Templates were intended to simplify maintenance of Configuration File Instances by combining common things into the template. Not saying it's the "right" design, but that's the use case.

However, we can definitely consider changing the behavior, to allow you to specify the default Template or a different Template when deploying.

Could I trouble you to share the configuration files (sensitive data redacted of course), so we can see the use-case better? We really want to document the configuration files better in the coming months, and having examples like this will help us tremendously.

We also want to make sure it's the best way to solve the problem. There is also the option of using those ASP/PHP-like OtterScript Snippets in Configuration Files, too. Maybe that's better to put that in your template? I don't know,...

Lots of options, and we want to make sure we document how/when to choose which ones.

Thanks,
Steve