Hi @kichikawa_2913 ,

I think it's this way for "historic reasons" - mostly all the other feed types came later, and it seems no one ever changes these paths or noticed.

Easy enough to make it configurable, but can you share your use case? Why do you want to use something other than a single root path with all of your packages?

Anyway I added a feature for this, and we should be able to get it in the next maintenance release PG-2006

Cheers,

Steve

@mcascone sounds like some great progress!

I got a little confused at the combination of "keep only last 5 versions" plus the 30-day window. From what i've read in the docs, all conditions must pass for the item to be deleted. How do i set up, "keep only the last 5 versions, but when nothing has been requested for 30 days, delete them all"?

We could definitely improve the docs on this area, and you're right all conditions must be passed for items to be deleted. When you add the "keep only the last 5 versions" rule, there will be, at a minimum 5 versions of a package.

You could be able to add a second rule, but it operates independently. More like an "OR" than an "UNLESS" I guess. Perhaps you could adjust the time-windows a bit?

• Rule 1. "Delete unused versions not requested in last 10 days." AND "Keep only last 5 versions"
• Rule 2. "Delete unused versions not requested in last 60 days."

I would look at your release cycles for guidance. For example, we release our products every two weeks, though maybe we'll skip a week every now and then. So, no -ci package will be needed past 1 month. And as you said, you can just rebuild if needed.

Hi @mcascone ,

We don't have a single API method that can be used to delete all package versions from the API, but the foreach loop will do the trick!

I should add that I am doing this as the first stab at an attempt to automatically delete packages from a development feed, when the corresponding branch in github is deleted

I don't know the specifics/details of your use-case, but based on what I read, I'd recommend these guidelines:

• assuming: one GitHub repository, one project, one package you want to release
• use the same package name/group for all packages you create for this project, regardless of branch or development status
• create your "dev" packages using a prerelease version number, that has a sort of -ci.## version (assuming you use CI to build packages)
• embed the commit id and branch in your upack metadata file, for traceability
• if you want to see which branch the packages was created from using the version number alone, add a +branch metadata label to the version number for branches (don't do this for master)
• use repackaging and promotion to take your -ci packages to -rc to stable (and the desired feed)
• let retention policies automatically cleanup up the -ci packages

Hi @brett-polivka,

I haven't tried ghcr, but in my experience GitHub is really unstable on the API/integration side when it's outside of their core source/Git hosting functions (e.g. Packages are notoriously buggy), so if your PAT is okay, then this is the most likely scenario.

The Connector Health Check for Docker uses the catalog API (/v2/_catalog), and the response should look something like this: https://proget.inedo.com/v2/_catalog. This endpoint is particularly buggy in other repositories (especially ones that require authentication/authorization), so my guess is GitHub introduced a regression or something on their implementation.

Another possibility... 403 is access related, so it could be something on your proxy side; please also check your Proxy settings, or something on your network side.

We have a lot of customers who maintain a separate test instance of ProGet; while upgrade testing is important of course, a dedicated testing instance also lets you evaluate new ProGet feature usage patterns (such as requiring promotion workflows, etc.), try out new tools (perhaps new version of visual studio, etc.), and conduct training on ProGet usage -- all without risking/disturbing your production instance.

To keep things simple from a licensing perspective, we just treat testing instances separate instances (and thus require a separate license key). Many customers use a ProGet Free License for this, but of course not all the features are available. It's rare to see a second license be cost prohibitive, especially given the labor/server costs involved with maintaining a testing instance -- even ProGet Enterprise customers will have full instances just for testing and even DR purposes.

You're right --- Active Directory is usually a pain point; sometimes our code changes (we try to never touch this), but also people want to change their AD configuration (move to LDAPS, etc.). Wrong settings, and you can lock-out your instance. If it's an uncommon / one-off testing case then a temporary trial license is fine for this.

Hi @antony-booth_1029,

What version did you upgrade from? That could help trace code changes.

Does this happen only on one page (i.e. /releases)? Then it's probably related to a bad query/unexpected data, but you only have 161 releaes according to your query. Not so much.... easy way to test that is by adding querystring params: /releases?ApplicationId=2&Status=Active for example.

If it's easy to see queryes on your RDS server, then we can see what query might be bad.

Does this happen only intermittently/randomly? If it's only you, then the problem probably isn't database/server load. And even with a ton of people, that's really rare. On very old instances with lots of retention jobs and years/gigs of data, doing an index cleanup is necessary, but I don't think that's the case here. The simplest/fastest thing I can think to do is reboot the BuildMaster server, and hope it goes away (maybe it's a weird underlying network stack thing).

Does this happen all the time (like nothing at all works on the website). Then it's probably network related?

Thanks,
Steve

@yogurtearl_0881 this is the first I've heard of Homebrew... at first glance, it looks like a kind of open-source/hobbyist/alternative package manager for MacOS?

hi @internalit_7155

Any updates on this issue? Were you able to resolve/fix this problem?

We've had another customer who reported a very similar problem (activation of a key on our new server, using sha256 vs sha1, causes an error in an old ProGet version). But we still can't reproduce it, and now I wonder if it's related to operating system version, or another operating system patch that's missing.

Thanks,
Steve

Hi @jeff-peirson_4344 ,

That's definitely what it's intended for, so I think this must be a bug...

I haven't had a chance to reproduce or look any further, but I wanted to at least share the code ASAP...

https://github.com/Inedo/pgscan/blob/master/pgscan/Program.cs#L90

... so please feel free to look/fix yourself, but we'll also take a look in the coming days as well! Just an FYI.

Thanks,
Steve

Hi @paul-reeves_6112 ,

This is by design; Configuration File Templates were intended to simplify maintenance of Configuration File Instances by combining common things into the template. Not saying it's the "right" design, but that's the use case.

However, we can definitely consider changing the behavior, to allow you to specify the default Template or a different Template when deploying.

Could I trouble you to share the configuration files (sensitive data redacted of course), so we can see the use-case better? We really want to document the configuration files better in the coming months, and having examples like this will help us tremendously.

We also want to make sure it's the best way to solve the problem. There is also the option of using those ASP/PHP-like OtterScript Snippets in Configuration Files, too. Maybe that's better to put that in your template? I don't know,...

Lots of options, and we want to make sure we document how/when to choose which ones.

Thanks,
Steve

Hi @paul-reeves_6112 ,

The moving is a bit strange and definitely shouldn't do that... but the remote browsing not working is a concern. I can see why it's not, and it's a way with how new agents are constructed behind the scenes in v7.

This feature was originally removed from v7 due to UI/JavaScript challenges, but we ultimately brought it back.. but clearly this part was overlooked in testing.

Anyways we'll get it fixed pretty quickly in BM-3716 - thanks for reporting it!

Thanks,
Steve

Hi @hwittenborn ,

I can definitely see how this could get annoying; this has been the design of our products for quite a while, mostly for simplicity/security reasons, and there hasn't been much demand for changing it. We're definitely open, so if other users are interested we'll certainly consider it.

Most administrators prefer "short sessions" (i.e. logged out at browser close or with no activity) for their own management simplicity; if we were to add "long sessions" (the "Remember Me" checkbox using persistent cookies), then administrators would need to worry about which users are "logged in", for how long, and terminate those sessions. And then we'd have to add all the features to support that capability - so nontrivial.

Best,
Steve

Hi @brett-polivka,

I've added it to our Other Feed Types page, and linked this as the official discussion thread.

There's a lot of things to consider in developing a new feed type, but ultimately it all comes down to two things: (1) how much more value does this feature bring to our users, and (2) how many new licenses of ProGet would this feature sell.

The second question is where internal market research comes in, but we would love your opinion on the first question.

Here's a nice and simple way to help understand value: how much more do you suppose your company/organization would pay for this feature if it were available as a hypothetical add-on? $100/year? $1,000/year? $10,000/year? Etc. And why? What time is it saving, risk is it mitigating, etc.

The second part of the value equation is how much effort will it take, technically speaking. It's more than 15 minutes obviously, but is it 10 hours? 100 hours? Etc.

On the plus side, the package format seems to be documented pretty well. However, the registry API has a huge red flag:

The index key should be a URL to a git repository with the registry's index.

Does this mean their API is Git-based, and we'd need to first add private Git repository hosting to ProGet? And did they test it with private/authenticated Git repositories, or just their public (probably GitHub) repository?

@saml_4392 said in ProGet hosting in k8s or VM?:

Btw, how do you measure the performance? Do you test it with a reverse proxy or use dotnet core directly?

In this case, it's mostly about what happens when the servers get overwhelmed with traffic - and it's largely anecdotal. So it could just as well be related to other factors, like other programs running on the servers running the Linux clusters, or SQL Client for Linux not performing as well, or who knows.

With Windows, people tend to set up dedicated or virtual servers with strict hardware provisioning.

@Stephen-Schaff let us know! If that works, then we can just inject that in the package, and read it in on the history page, similar to this NuGet package:

https://proget.inedo.com/feeds/NuGetLibraries/Inedo.ExecutionEngine/100.1.1/history

Hi @Stephen-Schaff ,

Thanks for the suggestion! So we had considered the "auto-generated one-time key" in our initial design, but decided against it for several reasons.

1. This enables the less-secure "API Key Spreadsheet Antipattern" - basically people want to store keys they generate -- and since the software doesn't allow it, they go this route. It's the same problem with "change your password every 30 days" policies that create easier-to-guess passwords.

2. This tends to create a lot of stale keys due to a fear to delete them. Administrators can "back-up" the API Keys if they can see them, and add-them back if cleaning up causes a problem.

3. Allowing keys to be entered allows users to more easily migrate from one instance to another - just do a DNS change, and all old automations /old keys will work fine.

API Keys are similar to passwords, but different; passwords are entered by a human to log-in, and in theory should only be "in that human's head" -- where as API Keys are always entered somewhere (usually in a script).

In general, in ProGet, we recommend keeping API Key as limited as possible. This simplifies things for everyone. There's no practical security problems in allowing all users to publish packages to feed.... you should be using package promotion to test/verify packages anyway.

Hi @Stephen-Schaff ,

We already have "Personal API Keys" coming, so I think this will address those concerns.

The User Impersonation is really only used by the "Feed API" Endpoints anyways, and the only "problematic" endpoints might be "Feed Management API" (they could delete feeds) or "Native API" (they could do anything).

Otherwise, I think this would best be handled by training and documentation. Perhaps just a warning to put on the Create API Key page?

We've learned the hard way that advanced settings like this are really hard to support -- everyone forgets they exist (including support team).

steve

Just some random thoughts...

We've had several customers try Kubernetes, but report that, in general, they found Windows to be easier to manage and maintain. "Anyone" can manage a product on Windows, but "only 1 guy" can manage things on Kubernetes, so he became the bottleneck of all things. In addition, the "Kubernetes guy" was constantly tweaking with their clusters, so the "rules kept changing" for how ProGet was supposed to work.

We don't officially support high-availability mode on Kubernetes yet, for the exact above reason. We aren't "Kubernetes experts" and don't know the best configuration.

Performance-wise, Windows tends to perform better on the same hardware, especially during high network traffic. The Windows network stack must handle it differently, as it can pass requests directly, instead of doing "whatever Docker/Containers and NGINX does"? Hard to say though, that's only anecdotal.

I did see a thread on how to run ProGet on a k8 cluster - we don't have an official Helm chart, but this is a great way to get that sort of stuff started

cc/ @viceice @saml_4392

Hey @Stephen-Schaff ,

We'd love to expand repackaging to more feed types! It's still a pretty new concepts, and not too many folks understand the value of the process/feature.

The main thing that ProGet adds is the audit-trail -- basically metadata added to do the package itself, so the "history" lives with it. Plus this way, you don't have to reproduce that download/repack/reupload process over and over again.

Any suggestions on where to add this to the chart? I think we just inject a basic file in the NuGet package, maybe that would work in Helm as well?

Cheers
Steve