Posts made by atripp

Hi @forbzie22_0253

The use case you're describing (i.e. using drop paths as an intermediate to publish packages to ProGet) is not uncommon. It's fine, and can be simpler in many cases.

But since you asked... the downsides are that require file share access, new developers may be confused by the process (since the typical workflow is publishing directly), and that there is a lack of immediate feedback mechanism (just because file copy is success doesn't mean package is accepted). I guess those are all obvious.

As to your other questions...

Can ProGet handle situations where multiple computers are trying to install a module from the feed while at the same time that module is being updated by the dropPath feature?

Package versions are immutable and not meant to be overwritten. Instead you are supposed to publish new versions. If your workflow involves continuously overwriting the same version while continuously consuming it, you will get errors. Regardless of using drop paths or publishing.

Could these be potential file locking situations where a package is copied to the drop path and ProGet is trying to access it in the dropPath?

If the file in a drop path is locked for reading (since it's currently being written), ProGet will just try again later.

Hi @scott-wright_8356 ,

Can you try downgrading to 2023.30? I see some Maven changes, and any time there's a possibility of a regression. That will at least tell us where the issue might be.

Thanks,
Alana

Hi @scott-wright_8356 ,

You can change if you'd like, I'm not sure if it makes any difference with .NET6+ applications.

Cheers,
Alana

Hi @appplat_4310 ,

In 2023.31, we made made introduced various database performance improvements (deadlock, timeout reductions) via PG-2606, so it's possibly related to that. The query you identified most definitely helped, and it there's only one place it's called.

WE would be super-grateful if you could try running this?

ALTER PROCEDURE [NuGet_GetPackage]
(
	@Feed_Id INT,
	@Package_Id VARCHAR(255),
	@Version_Text VARCHAR(255) = NULL
)
AS
BEGIN

	SET NOCOUNT ON

	SELECT *
	  FROM [NuGetFeedPackageVersions_Extended] 
	 WHERE [Feed_Id] = @Feed_Id
	   AND [PackageGroup_Name] IS NULL
	   AND [PackageType_Name] = 'nuget'
	   AND [Package_Name_Lower] = LOWER(@Package_Id)
	   AND (@Version_Text IS NULL OR [Package_Version] = @Version_Text)
END

We believe that, for some reason, your query analyzer is following a different plan, but the new conditionals should force it to use the right index.

Let us know, and we'll get it fixed right away.

Thanks,
Alana

Hi @rick-kramer_9238 , you could certainly try to enable those and see if they help or make a change.

@scott-wright_8356 the Inedo Hub will upgrade the database code (stored procs, views) when doing an installation. So from the time you upgrade the first server to the last server of a cluster, the "database code" and "server code" will be "out of sync", yielding errors like this.

For most users it's okay, and they will typically (manually) upgrade all nodes over the course of a few minutes during a mtaintenance period etc.

Hi @jw , thanks for pointing this out! I added a note to review/investigate this in our run-up to finalizing ProGet 2024.

Hi @philippe-camelio_3885,

Without looking closer or testing it's hard to say, but that error sounds like the property isn't being set? You can see rom the the source code of Git::Checkout-Code, the BranchOrCommit argument (called Objectish in the code) will use DefaultValue of $commit unless a property is specified .

I guess just as a test, does set $commit = master make any difference?

Thanks,
Alana

Hi @rick-kramer_9238 ,

It's possible that the IIS App-pool is automatically shutting down, which can lead to a "warm up" time that's required on a first request. However, this is typically measured in seconds in slow cases, not minutes. So it shouldn't take that long to "warm up".

Unfortunately this isn't easy to troubleshoot, and it seems to be related to some kind of strange IIS configuration. We've heard of users solving it lots of different ways, from switching servers to uninstalling IIS, to switching to the Integrated Web Server. But no idea what actually works.

The easiest thing to do, you may want to consider switching to the Integrated Web Server. Microsoft recommends that over IIS these days as well. The quickest way to do this is to uninstall ProGet (this does not delete database or packages), then reinstall to use the integrated web server. Make sure to point to same database at install time. You can configure HTTPS later.

Best,
Alana

Hi @gurdip-sira_1271 ,

That message means that the Login for GLOBAL\GLOPROGET01$ does not exist at the SQL Server level; you can add this in SQL Server Management Stuiod, under Security > Logins.

Cheers,
Alana

Hello,

I am copy-pasting the same reply made to EDO-10211 -- feel free to reply to one or the other.

We'll do our best to help, but based on the information you provided, it sounds like your server might be "overloaded" and have more traffic than it can handle. I would recommend trying out ProGet 2023.31 (releasing later today) which has a few more performance tweaks, but if that doesn't solve the issue than it will involve looking at ways to expand server/clsuter capacity or reduce/throttle traffic.

The error you shared is unfortunately unrelated, and is just a generic "client disconnected" error - it will occur if a a client (web browser, etc) disconnects prior a request being received. These should not be logged, but due to a known issue in .NET6 they are. This should be fixed in .NET8 (ProGet 2024).

Best place to check is ProGet Diagnostic Center, under Admin. That will have the most relavent errors.

Best,
Alana

Hi @forbzie22_0253 ,

(Sorry this is basically a copy/paste of the same answer to @philippe-camelio_3885 's question)

We haven't run into any of these issues in our testing, but this error is a "generic database timeout", which is implying that "something" is going on with the database. That's the only information we have, "something".

To troubleshoot this, you'll need to use some of SQL Server's performance/activity monitoring tools to spot what's going on at the time. It could be anything from outdated statistics to a missing index to a bad query. Or who knows.

With ProGet, timeouts can happen during extremely high usage of feeds/server. We have also seen instances where SQL Server's auto-statistics aren't working, and rebooting the server will help -- you can try running sp_UpdateStatistics as well. I have never seen that personally, but two users reported this already.

Best,
Alana

Hi @philippe-camelio_3885 ,

We haven't run into any of these issues in our testing, but this error is a "generic database timeout", which is implying that "something" is going on with the database. That's the only information we have, "something".

To troubleshoot this, you'll need to use some of SQL Server's performance/activity monitoring tools to spot what's going on at the time. It could be anything from outdated statistics to a missing index to a bad query. Or who knows.

Best,
Alana

Hi @philippe-camelio_3885,

We do not support "sub-path content/URL Rewriting" in any of our products and strongly discourage even trying. It won't work and will just lead to lots of wasted time and headaches, since everything from javascript ajax requests to CSS background images to cookies assumes a well-known root path of / for the application.

Best,
Alana

Hi @philippe-camelio_3885 ,

In BuildMaster 2022, we redesigned the Git and Source Control experience, which required significant changes to the various Git extensions (Git, GitHub, GitLab, and AzureDevOps).

With these changes, instead of using service-specific operations like GitHub::Get-Source, you can simply use operations like Git::Checkout-Code. Behind the scenes, BuildMaster wires everything up using build variables like $Repository and $Commit.

In Otter, you'll need to specify those variables or To and BranchOrCommit properties, similar to this:

Git::Checkout-Code
(
    To: $DossierVisHab,
    From: gitlab-vishab
    BranchOrCommit: master,
 );

The extensions are indeed the same between BuildMAster and Otter, but they do different things than before. Now, the GitLab extension mostly just provides integration into Issue Tracking, and information intended for the BuildMaster UI, such as a list of organizations, repositories, etc.

Best,
Alana

Hi @philippe-camelio_3885,

I didn't realize it was possible to create those in Otter

I thought only Generic Git repositories work. There's no GitLab/Github/etc.-specific functionality available in Otter, those are primarily intended for BuildMaster's integrations.

I would just use Generic Git repository, since the Git-base doperations will all be the same.

Cheers,
Alana

Hi @mahdy-merry_6333 ,

It's possible, and you can do this by creating a second feed that effectively proxies the first feed and uses disk storage. You could use a Connector on the two feeds or REplication.

Cheers,
Alana

Hi @matt-wood_5559 ,

Looks like your account was added to My Inedo incorrectly due to a human error (someone manually added it on our end since you had been talking to sales)... but it looks like its already been corrected, and you have a working key to use now. You shouldn't have any issues going forward.

Cheers,
Alana

@appplat_4310 yes please :)

Once I get confirmation it works, i'll commit the changes so it goes in the next maintenance release

It's really hard to say, but that's probably it. We'd really need to analyze it further to tell. How many packages are in the release?

FYI - this is all getting a "total rewrite" in ProGet 2024, and you can preview some of the features in the latest release of PRoGet

Hi @norm-ross_1437 ,

I'm not sure what version of ProGet you're using, but this sounds familiar and I'm almost certain we fixed this specific case a while ago. So I would try to upgrade.

I don't realy recommend playing around with data in the tables....

Best,
Alana

Hi @v-makkenze_6348 ,

I'm not sure what version you upgraded from, but there hasn't really been many changes that would have caused this I think.

This is a general database timeout error, which could be happening under heavy load (so trying again should work), or it could be a result of needing to update statistics, etc.

Thanks,
Alana

@lucas-almeida_8120 in that case, I would use a import path - https://docs.inedo.com/docs/proget-bulk-import-with-droppath

Hi @scott-wright_8356 ,

Thanks for sharing that; I think I see where the issue is.

This is recent regression... can you run the 1.FeedPackageVersions_ExtendedWithMaven.sql script that's attached to PG-2604? I think that will fix the issue.

Thanks,
Alana

Hi @scott-wright_8356 ,

That's strange; I'm not sure how that's possible. There's clearly some bad/unexpected data in the maven tables, and I'm not sure how that happened. Maybe an error/bad API call? I don't know.

Can you help us identify this by running EXEC Packages_GetPackages or SELECT * FROM FeedPackageVersions_ExtendedWithMaven, and seeing if you can spot any rows with with a null Package_Name`?

Specifically, I'm looking for something that would cause this below function to crash. The only thing I can think of is a null Package_Name.

        public static string BuildUrl(FeedId feedId, string group, string name, string version)
        {
            var builder = new StringBuilder();
            builder.Append("/feeds/");
            builder.Append(Uri.EscapeDataString(feedId.Name));
            builder.Append('/');
            if (!string.IsNullOrWhiteSpace(group))
            {
                builder.Append(Uri.EscapeUriString(group));
                builder.Append('/');
            }

            builder.Append(Uri.EscapeDataString(name));

            if (!string.IsNullOrEmpty(version))
            {
                builder.Append('/');
                builder.Append(Uri.EscapeDataString(version));
            }

            return builder.ToString();
        }

Hi @forbzie22_0253 ,

We are aiming to release ProGet 2024 in early April; here are the upgrade notes in progress, which will give some idea of what's coming: https://docs.inedo.com/docs/proget-upgrade-2024

Best,
Alana

Hi @lucas-almeida_8120 ,

This error is indicating that your database not being upgraded along with the rest of your installation. Or is otherwise corrupt. I can't imagine how that could happen, so unfortunately you'll have to do some digging on this.

Here is where the connection string for the ProGet database is stored:
https://docs.inedo.com/docs/installation-configuration-files

If you review the installation logs in the Inedo Hub, you will see what database/connectionstring is being used.

Good luck, and let us know what you find!

Alana

Hi @scott-wright_8356,

This is usually due to a networking error; it may also be resolved by restarting. It's also just something you can ignore. The service messanger isn't very important in ProGet.

Best,
Alana

Hi @scott-wright_8356,

When installing ProGet with the Inedo Hub, you can specify the SQL Server and Database Name. If it's an existing ProGet database, then it will be upgraded/downgraded as needed. I'm don't know what the errors were, but typically the installing users has a lack of db_owner permissions.

To upgrade ProGet in an HA set-up, you can just use the Inedo Hub to upgrade each node. Usually it takes 1-2 minutes per node, so you can do it manually pretty easily. There shouldn't be any errors.

Best,
Alana

Hi @scott-wright_8356 ,

This error means that "something" that is taking an exclusive lock handle on files within your SMBShare. Unfortunately, the operating system does not provide information about what that "something" is, but it's typically an anti-virus, search-indexing, or back-up tool.

ProGet uses shared read handles, but obviously needs a lock handle when writing. There are a few race conditions where this could occur, but they are extremely rare and require multiple requests on different servers in a cluster that attempt to add the same package at the same time.

However those are very rare, so I suspect "something else" is locking those files. Usually it's an anti-virus tool - since it "looks like" a zip file, it's being opened? You'd need to use a tool to monitor openarting system handles, like sysinternals procmon.

Best,
Alana

Hi @forbzie22_0253 ,

We do not currently have an API Key API, but this is something we are planning for ProGet 2024.

Today, you'd need to use the Native API. Since you're automating an installation, it's easeist to just use SQL to call those API/Stored Procs, since you'll also want to modify other properties like license key, etc.

Best,
Alana

Hi @proget-markus-koban_1462,

Thanks for clarifying; the "User directory" is outdated and I removed that from the documentation. In old versions of ProGet, you'd need to set up a separate directory. Now they are just added as Built-in users.

The Trial version should allow you to configure SAML. It sounds like the issue is that Keycloak is not redirecting to ProGet. I really wish I knew how to help troubleshoot... but after logging in, Keycloak is supposed to POST to /saml-acs-callback in the ProGet instance. I wonder if there's n additional thing that's not configured on Keycloak?

Best,
Alana

Hi @hashim-abu-gellban_3562 ,

This is just a user-friendly name for a URL within the Azure service, and isn't sent to ProGet.

You use practically anything I'm sure... we suggest proget-saml but anything is fine.

Alana

Hi @redbaron2_9872 ,

This error is unrelated to removing packages.

Based on the stack trace, it's coming from the Package Policies Preview Feature that was introduced in ProGet 2023.30. Not sure exactly how that's possible, but that code should not be running unless the feature was enabled under Admin > Policies.

Please let us know if that was the source of the error.

Best,
Alana

Hello,

I'm not familiar with Keycloak to give you a direct answer, but ProGet can integrate with LDAP and/or SAML protocols. Both of those will require configuration within ProGet, under Admin > Manage Security.

https://docs.inedo.com/docs/various-saml-overview
https://docs.inedo.com/docs/various-ldap-v4-advanced

Best,
Alana

Hi @v-makkenze_6348 ,

Can you try it a few times? Hopefully it will work eventually. It typically takes only 5-10 seconds, but it looks like that procedure is using the buggy SQL MERGE statement, which has issues on a variety of SQL Server versions...

We will rewrite that procedure in a future release.

Cheers,
Alana

Hi @moez-ul-hasan_3818 ,

ProGet only supports SAML-based authentication; as you noticed, we recommend and document using SAML for Azure. So I would just recommend creating a SAML-based Enterprise App for that.

OpenID is a totally different protocol, and our products do not currently have support for that.

Cheers,
Alana

Hi @forbzie22_0253 ,

Not really... we have tried it from time to time, and like your own testing, it seems to work okay?

In general, both the PowerShell Gallery and the client tools (PSResourceGet, PSGet) are "quirky", and we do our best to document this behavior for ProGet users. We link to several open GitHub issues in that documentation, and you've spotted several of other "quirks" logged in GitHub as well.

The issue isn't really about ProGet, but the fact that PowerShell tooling is designed to work exclusively with PowerShellGallery.com. Everything else is a a secondary use case, which is why you'll see a lot of issues logged with non-PowerShellGallery.com usage (ProGet, ADO, etc).

PowerShelllGallery.com is "based on" the NuGet Server API, but it's not documented how or where it diverges. Sometimes the PowerShell team will fix the "divergences", other times they won't. If we can work-around the "divergence" then we will, otherwise we just call it a "quirk" and document it.

Hope that helps,
Alana

Hi @mlorenzschleipen_5169 ,

Thank you for the reproduction information; I was able to reproduce this and we will fix via PG-2578.

It's a bit close to our maintenance release window (we ship later today), so it's scheduled for PRoGet 2023.30 (May 1). Let us know if you are interested ina pre-release, and we can get it as soon as available.

Thanks,
Alana

@forbzie22_0253 performance should be about the same, correct

Hi @forbzie22_0253 ,

SQL Server Express is the default database that we ship with ProGet, and is fine for many small/Basic installations. Once you go into a High-availability and Load-balancing configuration, higher editions of SQL Server are strongly recommended.

Thanks,
Alana

Hi @dan-brown_0128 ,

The latest version includes a migration tool, so I would recommend using that.

The URL has been changing throughout ProGet 2023, but you will be safe whitelisting cdn.inedo.com and security.inedo.com. This is only required for downloading updates, as it's an offline database that ships with ProGet.

Cheers,
Alana

Hi @forbzie22_0253 ,

ProGet Free edition may be used in commercial use. There is no limit on the number of instances, but a ProGet Free instance may not connect/network to another ProGet instance -- they need to be stand-alone instances.

ProGet Free instances can be connected to NuGet.org, just not other instances of ProGet.

Best,
Alana

Hi @scott-wright_8356 ,

Based on the stack trace, it looks like the timeout is occurring while running the Dashboards_GetLargestPackages procedure. We've tested that with feeds with 100K to 1M packages with no issues.

Suince you were digging into other SQL performance issues, can you try running that in SQL Server? Specifically Dashboards_GetLargestPackages 1, 10

Thanks,
Alana

Hi @scott-wright_8356,

We haven't seen any other issues with this procedure in particular, but it's something we can consider to update.

Have you considered that doing the commit with UPDLOCK and SERIALIZABLE could be causing issues with AlwaysOn and sycnchronsist commits going to the secondary?

One thing we faced with ProGet 2023's new database model was handling different bugs in different version of SQL Server's analysis engine. Without this pessimistic lock, some versions of SQL Server 2019 will deadlock while updating totally unrelated indexed view.

That said, you shouldn't need to use synchronous commits with ProGet in AlwaysOn. That's going to slow things down a lot to begin with, and that level of data-integrity is so important in ProGet.

Thanks,
Alana

Hi @yogeshshines_9136 ,

This should be resolved by downloading the latest Inedo Hub from https://my.inedo.com/downloads

Thanks,
Alana

Hi @Justinvolved ,

I've never seen that error and it makes no sense. It's a random Windows COM error. It's not BuildMaster-specific, it's happening when invoking the MWA libraries to save IIS configuration. This would happen if you performed the identical action from with IIS Manager as well, wrote a PowerShell script, etc.

I searched "a specified logon session does not exist. it may have already been terminated" and the advice is all over the place. It has something to do with permissions, I guess? Adding "IIS" adds more specific results, so maybe that will help.

So my advice from here is to just search and try random things that people said work. Feel free to share what you found !

Best,
Alana

Hi @sbaeurle,

I'm afraid we'll be sticking to following SemVer2 for the foreseeable future. That's a very well-documented standard that is machine readable and predicable. Microsoft's versioning scheme is anything but that.

That said, Microsoft's container tagging isn't at all consistent, as you can see: https://hub.docker.com/_/microsoft-windows

The operating system versioning is even more bizarre, and Microsoft abandoned any sensible Major/Minor scheme in the late 1990's. For example, Windows 7 was 6.1, Windows 8 was 6.2, Windows 10 is 10.0, and Windows 11 is.... 10.0. S Except when it's aliased as 23H2 something. Servers are.... no one really knows.

Then add to that patching, which may or may not impact the version number.

To clear up this confusion, we recommend you use SemVer to make versions of your internal base images, based on some Microsoft build.

This is the mapping scheme we recommend:

• Major is the product number (19 for Server 2019, 22 for 2022)
• Minor is the "service pack" version (0 for first version, 1 for 1903, etc)
• Patch is your internal revision number of that image (when you install patches, etc)

Another "tip" is overloading digits. For example, if you see a case where you will want to "revise your patches", then just increment your patch version by 10. So 0, 10, 20, 30, 40. Then you can "patch" 10 by going to 11, 12, 13.

Hope that helps

Cheers,
Alana

Hi @PhilipWhite ,

Thanks for clarifying!

Long story short, you probably want to just use $PipelineStageName. In your case, it would be which would be "Test" (based on the screenshot that says Test Stage).

Overall, it's a little confusing but Pipeline Stages and Environments are orthogonal concepts:

• Environments are used to group servers and scope permissions
• Stages define a phase in the deployment/release process (i.e. a pipeline)

A Deployment Target (e.g. "Deploy to XXXXX") is part of a stage, and may be associated with an Environment. Just click "edit", then check "Environment-specific permissions", then select an environment.

That would put an environment in context (and thus $EnvironmentName would return what you expect), but more practically it would:

• throw an error if any of the servers used in the execution are not a part of that environment
• not allow someone to press the "deploy" or "force" button to that stage , unless they have permissions specific to that environment

It's a little confusing at first, which is why we "kind of" hide environments from the user.

This is most definitely a common point of confusion, so we are really open to feedback if you can think of how to improve documentation/user experience!!

Alana