Posts made by dean-houston

Great news, thanks for the update!

Hi @sneh-patel_0294 ,

That error message is coming from the operating system; it doesn't necessarily mean a permissions issue.

Does it happen every time for every package, consistently?

If that's the case, then it's certainly some kind of permission configuration. The user running the ProGet Web Service (or IIS App pool) may not have the appropriate permissions to the folder.... or it could be something related to network access? I don't really know.

The operating system is opaque with the error message, and you might have to use a tool like procmon to see exactly what's going on. That will show you what programs/processes request file handles.

If this is sporadic, then it means the file is locked. It's possible for ProGet to lock the file, but it's unlikely and would require basically two processes trying to write to the same file at the same time. We've only seen that with misconfigured build servers that publish same build twice.

More likely the file locking is coming from like backup, index scanning, or malware that's masquerading as "security software". Procmon will also advise this, if you can catch it.

-- Dean

Hi @forbzie22_0253 ,

Tags are a field in the nuspec file, which is embedded within the NuGet package:
https://learn.microsoft.com/en-us/nuget/reference/nuspec#tags

As such, we do not recommend using tags in NuGet, because they are "permanent" and you can't "untag" packages. It makes it hard to think of a tagging system that will be useful for the long-term: https://blog.inedo.com/nuget/best-practices-internal-nuget-packages/

-- Dean

Hi @arkady-karasin_6391 ,

A 403 error means Not Authorized; since those are publicly-available repositories, my guess is that you have a Proxy or Content Filter that's blocking that URL, and the ProGet Server is getting a 403 response from that intermediate server.

-- Dean

@joel-shuman_8427 thanks for the heads up!

I just updated it
https://github.com/Inedo/inedo-docs/blob/master/CONTRIBUTING.md

HI @parthu-reddy ,

Thanks for the feedback; I update the docs to mention that the feeds/permissiosn would also need to be updated as well.

We hope to handle an in-place migration in a future version, but didn't want to delay shipping the feed. Please let us know if you have any issues/feedback.

Note we are adding several improvements to the Maven (New) feeds in ProGet 2024.15:

• PG-2798 Add Direct Download (Artifact Import) Support for Maven (New) Feeds
• PG-2797 Add OSS Metadata Caching Support to Maven (New) Feeds
• PG-2796 Improve MavenIndex Download Visibility for New Feed Connectors
• PG-2792 Feed Management API Returns Empty Type on Maven2 and new feeds
• PG-2794 Add Simulated Directory Browsing to Maven (New) Feeds

-- Dean

Hi @parthu-reddy ,

I'm afraid we don't have enough information to help with this; it appears to be an error the tool you're using (packet), and there's not enough information in the screenshot to see what error packet is encountering. It just says "Packaged failed with could not download..."

We aren't familiar enough with packet to know how to follow their Stack Trace.

If you can't find a clear error message, I would use an HTTP Proxy tool like Fiddler Classic to inspect the traffic that packet is making, and see if you can spot an issue there.

I would also try downgrading packet , as it's very possible there's a regression in the tool.

-- Dean

Hi @kichikawa_2913,

We see multiple connectors pretty often, and it rarely presents a problem.

The main downside comes in the overhead of aggregation; for some queries like "list all package versions", each connector will need to be queried and have the results aggregated. So it could cause performance issues for for very high-traffic feeds - at least that's what we see on the support side of things.

However, if you plan on using a package-approval workflow, then it won't be a problem, as your approved-npm feed wouldn't have any connectors.

Hope that gives some insight,

Dean

Hi @stuart-houston_1512,

ProGet will download nightly updates of the vulnerability database from an inedo.com server.

If you're running in a totally air-gapped environment, then you obviously won't get these updates. However, each version of ProGet includes an up-to-date database, so upgrading will get you the updates.

--Dean

Hi @steviecoaster ,

The Native API can be a little finicky, especially since you can invoke with JSON, forum-encoded values, querystring, and I think even XML. But it sounds like you're on the right track.

Let me share the C# code that ProGet uses to set the password:

    using (var rfc2898 = new Rfc2898DeriveBytes(password ?? string.Empty, 10, 10000, HashAlgorithmName.SHA1))
    {
        var bytes = rfc2898.GetBytes(20);
        DB.Users_SetPassword(userName, bytes, rfc2898.Salt);
    }

... it looks a little different than the code you're using, so hopefully that will help!

-- Dean

Hi @russell_8876,

Thanks for all the additional details!

Actually, this is definitely a bug in pgutil, since it's sending a negative offset to ProGet, and then ProGet is responding with a 400 error ("Missing or invalid "offset" parameter.)

Asset uploaf capability is just a few weeks old, and I'm not sure if we tested with such a large file. Anyway, we'll get it looked at and let you know once fixed; just a small tweak needed here I think:

https://github.com/Inedo/pgutil/blob/thousand/Inedo.ProGet/AssetDirectories/AssetDirectoryClient.cs#L240

-- Dean

Hi @noam-linux-2022_8869 ,

Here is documentation on how to enable HTTPS in our products on Windows and Linux:

• https://docs.inedo.com/docs/installation/installing-on-iis/installation-windows-https-support
• https://docs.inedo.com/docs/installation/linux/https-support

-- Dean

Hi @steviecoaster,

The Native API isn't hacky, just harder to used. Here is the documentation on the Native API:
https://docs.inedo.com/docs/proget/reference-api/proget-api-http#native-api-endpoints

We don't have any articles/guidance on how to call the Native API beyond what's there.

The Users_* procs have not changed in years and are very safe to use. There are a few forums posts here and there with "hints" on work with the User_ procs, like this:
https://forums.inedo.com/topic/4198/reset-proget-admin-password-via-api/2

-- Dean

Hi @steviecoaster ,

I agree it'd be nice, but it's also not trivial as you noticed. I believe it's possible to configure security with the Native API, but obviously not as easy. So I would explore that, it's probably close to what you wnat.

A first-class Built-in Users/Groups API has been requested over the years (and is now something that makes sense with pgutil) - however if we made it, the API would be paid-editions only. We haven't had any interest from paid users in such a feature, as they generally use LDAP or don't mind non-API configuration.

-- Dean

@steviecoaster great, thanks for sharing!

I added this to our documentation (https://github.com/Inedo/inedo-docs/pull/253), but apparently I don't have ability to merge PRs in that repo so it'll go live sometime later I'm sure

@steviecoaster glad you were able to figure it out :)

Easy typo and we should consider a validator on that "Update SSL Certificate" page as well to save a headache like this!

Hi @v-makkenze_6348 ,

If you download (i.e. cache) the package, then you shouldn't see the compliance issue anymore. The reason is that ProGet does not have information about the package unless it's cached/local, or if you're viewing it on the package overview page.

When ProGet runs a build analysis (first screenshot), it only uses local/cached package data. This is for performance reasons, as users will have 100's of builds with 1000's of packages in each build, and that much traffic to each connector is problematic.

However, we are working on building a "remote metadata cache" that will fetch this data in a more performant manner.

-- Dean

Hi @steviecoaster ,

Sorry about giving the bad advice there -- I did not realize that the offline installer does not include the hub.exe program. It looks like the Offline Installer Creation Process must strip that from the offline installer we provide.

As you discovered, the advice from before wouldn't really work. Without doing a deep dive in the code, I don't know how to make it work. This isn't a use case we designed for, and I'd hate to send you down a wild goose chase.

How about just using our standard silent installation approach, which I shared before:

# create working directories
mkdir C:\InedoHub
cd C:\InedoHub

# download and extract file to working directory
Invoke-WebRequest "https://proget.inedo.com/upack/Products/download/InedoReleases/DesktopHub?contentOnly=zip&latest" -OutFile C:\InedoHub\InedoHub.zip
Expand-Archive -Path InedoHub.zip -DestinationPath C:\InedoHub

# perform silent installation
hub.exe install ProGet:5.2.3 --ConnectionString="Data Source=localhost; Integrated Security=True;"

This will basically install the desired version and it's likely "good enough" for the time being.

-- Dean

@MY_9476 I'm afraid not; what you're seeing is a diff of the OtterScript, which is what's stored in the database. The issue is that the "serialization to code" order must have changed between major versions.

We try to not have that happen, but fortunately this only happens the first time you edit such a script. Next time, only your changes will be preserved.

Hi @jw,

There's no problem deleting the deprecated licenses and adding their SPDX identifiers to the new licenses. When you delete a license, it will remove the association with packages.

However, the compliance analysis scheduled job will reassociate them. This runs nightly, or you can manually run it. Also, if you visit the package page or download the package, it should associate with the new license.

We'll definitely reconsider the design/approach if there's more demand, but we need to really make sure there's a value to the user -- there's a relatively high cost to change things and then there's a chance of regression/bugs, which is really frustrating to users.

-- Dean

Hi @jw ,

[1] is somewhat expected behavior, as some older versions of ProGet allowed non-normalized URLs to be added; newer versions should not allow this. We do not plan to "clean-up" the data at this time, but if you're "brave" you could could like do it with some API/DB calls

[2] This is probably some quirk related to older data, but you can probably just cut the items to clipboard, save the license, the edit again, and it should work-around the quirk

FYI, the URLs that we have in our database for LGPL-3.0-only are: as follows

gnu.org/licenses/lgpl+gpl-3.0.txt
gnu.org/licenses/lgpl-3.0-standalone.html
opensource.org/licenses/LGPL-3.0

I'm guessing you have the www from an older version

-- Dean

Hi @MY_9476 ,

I'm afraid this isn't technically feasible.

When you edit in Visual Mode, the statement is basically "serialized" to OtterScript, and the property order is determined by however the operation's code (i.e. C#) is laid out. Fortunately this is really consistent, and really only is a problem when you go back-and-forth.

But in this case, there must have been some change to the C# / order of properties (refactoring perhaps?) so it's inconsistent across versions of Otter? If you edit it again, the order will be the same.

-- Dean

Hi @steviecoaster,

We're currently working on the pgutil feeds commands, and expected create and update to be finished in a week or so. In the mean time, I just published pgutil-1.1.2, so please check that out - it will have a basic version of pgutil feeds create.

As for the PowerShell command, you'll need to pass the JSON structure that's documented in that article. You would want to use something like ConvertTo-Json to convert a Hash table to JSON.

-- Dean

Hi @steviecoaster,

If you want to use the offline installer in a script (which I think is best for a Chocolatey script), then just unzip it after downloading. I know it's an .exe file, but you can just unzip it like any other zip file (e.g. with Expand-Archive or something in PowerShell).

Once you've done that, you can just run the hub.exe per the silent installation instructions.

Alternatively, you can write a script like this, which will instruct hub.exe to download a version:

# create working directories
mkdir C:\InedoHub
cd C:\InedoHub

# download and extract file to working directory
Invoke-WebRequest "https://proget.inedo.com/upack/Products/download/InedoReleases/DesktopHub?contentOnly=zip&latest" -OutFile C:\InedoHub\InedoHub.zip
Expand-Archive -Path InedoHub.zip -DestinationPath C:\InedoHub

# perform silent installation
hub.exe install ProGet:5.2.3 --ConnectionString="Data Source=localhost; Integrated Security=True;"

The only downside to this approach is that it's not really version controlled (i.e. it's always using latest Inedo Hub) and the package can't be realizably internalized, since hub.exe will download files from the internet.

-- Dean

Hi @cstekelenburg_4169 ,

If you created a connector with the proper URL, username, and password, then you've set it up correctly.

Please note, you will not be able to list or search remote packages in the ProGet UI. This is a limitation, since ADO is rudimentary and does not support listing/searching.

Instead, you will need to use the NuGet API (Visual Studio, Nuget.exe) to pull packages. Once package has been pulled through ProGet, you will "see it" in the UI since it's been cached.

I'm not trying to be pedantic with terminology, but please note it's not a "sync" -- it's more of a "proxy" or "pass through". When you make a request via the NuGet API, that API request is forwarded to your connector (I.e. Azure DevOps).

-- Dean

Hi @cstekelenburg_4169,

Azure Artifacts does not support the NuGet Search API, so this means you won't see packages on the feed/search page in ProGet. This is expected behavior with rudimentary NuGet repositories that don't support listing/searching.

In the past, I'm guessing that you either:

• used the Azure Feed Import Wizard, which will perform a one-time download using a special (non-NuGet API)
• pulled packages via the NuGet API (e.g. with nuget.exe, visual studio, etc), which will then would cache them in ProGet

-- Dean

Hi @cstekelenburg_4169 ,

ProGet does not "sync" remote repositories, but instead displays and caches results through a "connector" you configure. If there are any connector errors, they will be logged when you do a query.

Azure artifacts do not support searching or listing packages, so you won't "see" a package unless you type in the exact package name.

-- Dean

Hi @johnsen_7555 ,

Thanks for sharing the additional details; I was able to reproduce this, and its a regression/bug.

We fixed this via PG-2723 (FIX: Noncompliant Pypi packages can be downloaded when blocking is enabled on trial licenses), which is going to be shipped in the next maintenance release.

However, in the mean time, you can try a patch/prerelease version that contains the fix; since you mentioned Docker, the tag to pull would be 24.0.9-ci.1.

Hopefully that will solve the issue!

-- Dean

Hi @johnsen_7555 ,

Thanks for sharing all of the configuration details; can you navigate to one of the GPL packages that you're trying to install, and see what it says on the package page within ProGet?

On that page, you can also ReAnalyze the package (using the dropdown button ) and get a log of which policies/rules were applied.

-- Dean

Hi @carl-westman_8110,

Thank you so much for the kind words, and it sounds like you're on the right track for setup/configuration!

ProGet is designed for the scenario you described (i.e. a multiple instances that point to the same database/files), and we call it a server cluster:
https://docs.inedo.com/docs/installation-high-availability-load-balancing

The above instructions are for Windows, but the same principle would apply for Azure Web Apps, Kubernetes, etc. We just don't have documentation or the ability to support the underlying platform. What I mean by that, if the "scale out" feature in Azure doesn't do what it's supposed to, you would need to contact Azure support.

All that said, do note that a ProGet clustered installation requires a ProGet Enterprise license:
https://docs.inedo.com/docs/proget-administration-license

Hope that helps,

-- Dean

Hi @scott-wright_8356 ,

That procedure is ancient and hasn't caused problems before. I suspect the issue might be the size of the data in that able - you can see what that looks like on the connector cache management page.

You could also try to optimize the query like so:

ALTER PROCEDURE [dbo].[Connectors_GetCachedResponse]
(
  @Connector_Id INT,
  @Request_Hash BINARY(32)
)
AS BEGIN

SELECT * INTO #ConnectorResponses
   FROM [ConnectorResponses] WITH (READUNCOMMITTED)
   WHERE [Connector_Id] = @Connector_Id
   AND [Request_Hash] = @Request_Hash

UPDATE [ConnectorResponses]
   SET [LastUsed_Date] = GETUTCDATE(),
       [Request_Count] = CR.[Request_Count] + 1
 FROM  [ConnectorResponses] CR 
  JOIN #ConnectorResponses _CR ON CR.[Connector_Id] = _CR.[Connector_Id]

SELECT * FROM #ConnectorResponses
  
END

Let us know if that works, we can update the code accordingly if so.

-- Dean

Hi @philippe-camelio_3885,

Troubleshooting the database can be a pain, but there are a lot of tools that can help. Here is a nice guide from Microsoft, which includes a script that shows you how to use the sys.dm_exec_query_stats and sys.Dm_exec_sql_text views:
https://learn.microsoft.com/en-us/troubleshoot/sql/database-engine/performance/troubleshoot-slow-running-queries

Just make sure to remove WHERE t.text like '<Your Query>%' from the sample, if you want to see all queries. Some red flags on that query will a disproportional total_elapsed_time , etc.

That said, there are two different jobs:

• "Server Checker" attempts to connect to each server and make sure the agent is up to date; when this fails, a server will go into an Agent Error state
• "Routine Configuration" attempts to check desired configuration against each server using OtterScript; this is what puts a server in Failed or Drift state

A "Server Checker" is run frequently and should be very fast. There is very little database activity (just Servers_GetServers to get all servers, then Servers_UpdateServerStatus per server).

A "Routine Configuration" is obviously more complicated, and may involve a lot more database activity.

-- Dean

Hi @scott-wright_8356 ,

Thanks for sharing a lot of the details; it's not really clear what objects the locks are on, can you look at the XML view of the deadlock report to find out?

Since you're "already there", you're welcome to try doing what we would do --- just add WITH (READUNCOMMITTED) to the problematic queries (i.e. the ones where object locks are occurring on).

For example, like this: SELECT * INTO #PgvdPackageNames FROM [PgvdPackageNames_Extended] WITH (READUNCOMMITTED) WHERE [PackageName_Id] = @PackageName_Id

As an FYI, this should not be an issue in ProGet 2024, since package analysis is cached for a short while. Also, our internal development practices (going forward) are to READUNCOMMITTED in "hot" queries this anyway.

-- Dean

hi @andreas-unverdorben_1551 ,

The timeout message you shared is for a NuGet feed - so the issue isn't the 727 packages that you're installing (which often results in 1400-2100 requests being fired off simultaneously) , but the NuGet build(s) that are also going on at the same time and hammering the server.

The server cluster will definitely help with this peak load.

FYI --- that error message you shared is is from NuGet v2 API, so something is still calling it.

Cheers,
Dean

Hi @andreas-unverdorben_1551 ,

What's happening here is your server is being overloaded due to lots and lots of traffic -- effectively you are doing a Denial of Service attack on you server. As I mentioned in npm install slow on proxy feed, ProGet is not a static file server and there is a lot of processing required for each request.

The best way to handle this is to run a load-balanced ProGet server cluster.

Alternatively, you will need to reduce or throttle traffic.

• ProGet has a built-in traffic throttle under Advanced Settings (concurrent web request limit)
• Stop using the NuGet v2 API, that's very inefficient
• Stop or reduce connector usage, so ProGet doesn't have to contact nuget.org etc.
• Do not use multiple sources in your nuget configuration, since NuGet will issue all of those requests simultaneously
• Disable parallel restore on NuGet and npm

Best,
Steve

Hi @andreas-unverdorben_1551 ,

npmjs.org primarily serves static content and runs on massive server farms running in Microsoft's datacenters.

Your ProGet server is much less powerful and does not serve static content. Not only is every request is dynamic (authentication, authorization, vulnerability checking, license checking, etc), but most requests (such as "what is the latest version of package X") need to be forwarded to npmjs.org and aggregated with local data.

So, a much less powerful server doing a lot more processing is going to be a little slower ;)

Running ProGet in a server cluster will certainly help.

Cheers,
Dean

Hi @devops-services_2004 ,

In this case, can you open a ticket and send us that tar.gz file (attach on the ticket)?

From there we will load it in a debug version of ProGet and identify where the problem is occurring. Not something that will be easy to spot by inspecting a file :)

Thanks,
Dean

Hi @devops-services_2004 ,

This means that the file you uploaded contained unexpected data; like maybe something in the package.json file was wrong, or it was in the wrong format, etc.

How did you create this npm package? Or did you download it from somewhere? We could probably look at the file and say what's wrong with it.

Best,
Dean

Hi @henh-lieu_7061 ,

This error indicates that you don't have access to the c:\ProgramData\Romp directory, or there is otherwise a file error when writing to that folder.

You can try deleting that folder, then reinstalling again.

Best,
Dean

Hi @Ricardo_C_RST ,

For EC2 (a virtual server), you can just follow the ordinary Inedo Hub installation process. As for HTTPS and Domain, that's a setting probably easiest to do in AWS. I'm not familiar with AWS, but something like CloudFront might help?

Dean

Hi @Justinvolved,

I don't know if I'm totally tracking the structure, and what you mean by "referencing the latest builds" in the deployment scripts.

Are you looking to have three applications? One app per repo, and then a "controller" application?

Or, do you want to have one application that builds from two repos? And it will always build/deploy content from those at the same time?

Dean

Hi @MaxCascone ,

Thanks for pointing these out; we will get the RPM icon fixed (probably a minor CSS thing), and then then hide the option on the Manage Asset Directory page (via PG-2540).

This view is intended mostly for ProGet ISV Edition use cases, and we likely won't expand on this in ProGet 2024, since you're probably first person who commented on it

Cheers,
Dean

Hi @kigiwow570_6179,

I don't have enough information/specifics to answer the questions; it really depends on the API queries being made, etc. Some API calls result in errors, others do not. In general, doing a NuGet restore will not result in errors being logged.

Best,
Dean

Hi @kigiwow570_6179 ,

These errors are being "forwarded" from NuGet.org, and if you're seeing them it means that "something" on your network is making requests to a ProGet feed that has a connector to nuget.org.

This is typically an old, outdated tool or script. You'll need to track this down and update/disable it.

It could also be a self-connector that is connecting to a feed that also has a connector to ProGet using the V2 APi - so double check your self-connectors.

Best,
Dean

Hi @philippe-camelio_3885 ,

You mention the server is crashing? That's really strange, and we've never quite seen anything like this before. We routinely test on much less powerful hardware in nearly identical scenarios that you are doing now.

SQL Server / .NET will use as much ram as it can, so the usage isn't surprising or really concerning.

Otherwise we don't really any info info on the errors --- You'll need to track down error messages from the ProGet side of things; perhaps the Docker container error logs, the SQL Server error logs, etc. Maybe try putting the npm client (i.e. BuildMaster) on a different server, just to help isolate any potential issues.

We've seen a lot of weird things happen -- every now and then, there's a problem with the network controller/hardware/driver on the server, and it's having a hard time processing the 1000's of simultaneous "loopback" connections that npm is making from the BuildMaster container to the ProGet container, and the "outbound" connections that npm would be making to npmjs.org.

But until you find out what those errors are on the ProGet side, it's impossible to guess.

Thanks @philippe-camelio_3885 , roger that!

I thought you were reffering to the API :)

We'll get this fixed as BM-3912 in a future maintenance release, likely Dec 14 since it's a holiday week coming up here in the states.

Cheers,
Dean

Hi @philippe-camelio_3885 ,

Thanks for clarifying; so it sounds like you're talking about Variable Prompts (Templates), which are part of a pipeline?

These are not "variables" per se; instead, they are used to prompt users at certain points during a pipeline (creating build, deploying build, etc.) to input values. Those values will then be created as variables.

We don't plan to extend templates outside of pipelines. If you reaaaaly wanted to, you could create a BuildMaster "application that creates applications" using the API and then leverages the pipeline prompts to create the variables in the format you want.

It's probably just easier to clone an application, however.

Cheers,
Dean

Hi @philippe-camelio_3885 ,

I'm sorry it's not really clear what you're trying to accomplish.... you can define list-based variables at all levels. Just specify the value like @(a,b,c), for example.

Cheers,
Dean

Hi @philippe-camelio_3885 ,

Sorry - it's not really clear what the issue is; can you share more details? Ideally a way I can reproduce this in a new instance of BuildMaster, but also maybe a screenshot or the APi calls you're making?

Thanks,
Dean

Hi @pallavi-tarigonda_9617,

These errors are being "forwarded" from NuGet.org, and if you're seeing them it means that "something" on your network is making requests to a ProGet feed that has a connector to nuget.org.

This is typically an old, outdated tool or script. You'll need to track this down and update/disable it.

It could also be a self-connector that is connecting to a feed that also has a connector to ProGet using the V2 APi - so double check your self-connectors.

Best,
Dean