Posts made by stevedennis

@philippe-camelio_3885 this is related to SQL Server timeouts/performance issues

Unfortunately we don't have any other information available from the information provided, and about the only suggest I would have is to attempt to increase hardware/CPU on the SQL Server, perform database maintenance (see advanced maintenance) , and finally use SQL Server performance monitoring tools to find where there are problems

There may be some queries that can be optimized or missing indexes. If you can find any of those, please let us know and we can explore to add those optimizations in the product

@jw the script should also work in ProGet 2023

That error is implying that data in the FeedPackageVersion didn't get cleaned up, as it should have been done on this portion:
https://gist.github.com/apxltd/351d328023c1c32852c30c335952fabb#file-gistfile1-txt-L449

Unfortunately this difficult to debug on its own, and it will be impossible to debug w/o the database itself, but if you're comfortable with SQL feel free to modify or tweak. It's not an easy script :(

We haven't yet run this against all customer databases yet, but it's our list this week. If you haven't sent us your database already, please do :)

Hi @v-makkenze_6348 ,

Just an FYI, we do have a duplicates clean-up script available, and ran it against your database with no issues.

https://gist.github.com/apxltd/351d328023c1c32852c30c335952fabb

We haven't yet decided how to roll it out, but if you do run into more package analyzer issues like this (particularly on the SBOM side), it might help.

Thank you,
Steve

Hi @philippe-camelio_3885 ,

Looks like this is a bug when severs have list-type configuration variables; we'll get this fixed in an upcoming maintenance release via BM-3944

Otherwise I can't think of a work-around that would be any simpler then editing the impacted servers.

Thank you,
Steve

Hi @philippe-camelio_3885 ,

A Full Server Check will occur once per hour by default - does that also crash? You can also trigger by going to Admin > Service.

Also, can you give more detail about what you mean by crashing? There is no "app pool" on Linux/Docker (doesn't use IIS), and the web application isn't responsible for running the server check (that's the Service application). So unclear what the crash would be.

Thank you,
Steve

Hi @jw ,

So far as we can tell, these sort of errors are only occurring when there is a 4-part NuGet version ending in 0 that was added in a previous version. For example, if you added 4.2.1.0 in a previous version, it could yield an analysis error when trying to view the package as 4.2.1 (i.e. how NuGet requests it).

The duplicate package names were a problem in ProGet 2023 as well, but we haven't seen any crashes do to that yet. Just weird results in the SBOM analysis. But these would have also been a problem in ProGet 2023.

So long story short, I don't think you need to run the script.

However, if you upgrade and run into these issues, then you can run it:
https://gist.github.com/apxltd/351d328023c1c32852c30c335952fabb

If that still doesn't work, then rollback to ProGet 2023 and restore the database. And hopefully send it to us again so we can investigate the issue :)

Thank you,
Steve

Hi @jw ,

We have a data-cleanup script but did not ship it with ProGet 2024.1; we are considering it in ProGet 2023.2.

The above error occurred because of a 4-part version that ended in .0, but if you just delete that from the feed and add it back again, its should work. Can you try that?

We can provide you the data clean-up script if you want to try it. We ran it against several customer databases and it seems to work okay, and clean-up a lot of "bad" versions and casing.

Thank you,
Steve

Hi @dan-brown_0128 ,

Oh that's really weird --- I must have mis understood from the get-go. I'm not sure how that's scenario is possible. At least in ProGet 2023, 2024, I wasn't able to reproduce this under any circumstances

Perhaps it was a bug in an import from a long-while ago. But re-reading your first post, it sounds like you've worked around it?

Otherwise I think it would take a kind of data-cleanup script to handle.

Thanks,
Steve

hi @dan-brown_0128,

ProGet simply does not support case-sensitive npm package names. This is a 10+ year old design decision (back from ProGet v3), which we made because npm package were always supposed to be lower-cased: https://docs.npmjs.com/creating-a-package-json-file#required-name-and-version-fields

JSONPath is an invalid package that entered the public registry due to an npmjs.org bug. It's an old package -- latest version version 0.1.2, is 8+ years old -- and according to the package page says that it's "moved to jsonpath-plus to avoid npm problems in dealing with upper-case packages."

ProGet permits these packages, but there are just going to be quirks if you have them in your feed. For example, if you have both jsonpath-1.1.1 and JSONPath-0.1.2 in your feed, then they will generally show up as jsonpath, but in some API queries you may see JSONPath.

We understand this is different than how the npmjs.org registry behaves, but as I mentioned, this was a "day 1" choice that we can't change today. No one's mentioned this as an issue until now.

If npm audit is crashing because sometimes JSONPath is returned in results, that sounds like a bug in npm audit to me? We obviously can't add support for case-sensitive npm packages nor change how ProGet returns data in npm API calls just to work-around an npm audit bug like this.

The easiest solution is to stop using JSONPath and remove from your feed, which it sounds like you already have. It's a really old package, and there are only a handful like this on the public registry.

Best,
Steve

@dan-brown_0128 I understand

So you'd either have to upgrade (where we fixed the code) or reassess it to Low so npm audit won't crash. I suppose you could also patch npm audit so it doesn't crash.

Hi @dan-brown_0128 ,

Thanks for the detailed analysis; we'll get this fixed via PG-2636 in 2024.1 (due later today).

You should be able to assess it as "Low" to work-around it in the meantime. FYI, here is the code that is used to map a vulnerability to an npm vulnerability:

//assessed
if (vuln.Severity_Code != null)
{
    return vuln.Severity_Code switch
    {
        Domains.AssessmentSeverityCodes.Error => npmAuditSeverity.critical,
        Domains.AssessmentSeverityCodes.Warning => npmAuditSeverity.high,
        Domains.AssessmentSeverityCodes.NotApplicable => npmAuditSeverity.info,
        Domains.AssessmentSeverityCodes.Custom => npmAuditSeverity.info,
        _ => npmAuditSeverity.info
    };
}
// unassessed and has a Severity Score
else if (!string.IsNullOrWhiteSpace(vuln.ScoreSeverity_Text))
{
    return vuln.ScoreSeverity_Text switch
    {
        nameof(CVSSRange.Low) => npmAuditSeverity.low,
        nameof(CVSSRange.Medium) => npmAuditSeverity.moderate,
        nameof(CVSSRange.High) => npmAuditSeverity.high,
        nameof(CVSSRange.Critical) => npmAuditSeverity.critical,
        _ => npmAuditSeverity.info,
    };
}
return npmAuditSeverity.none;

Thank you,
Steve

Hi @dan-brown_0128,

I understand that npm is currently case insensitive, but there was a brief period that it wasn't. This led to several duplicate packages being published:

• https://www.npmjs.com/package/JSONPath
• https://www.npmjs.com/package/jsonpath

I don't believe this is possible anymore? Either way, ProGet is case insensitive for npm packages, which means that JSONPath and jsonpath are considered the same package. If JSONPath came first, then that's what the package name is I guess? In either case, the package name should be returned in results as jsonpath

It doesn't seem to cause any issues, except with npm audit?

This is a rare case, so I would just not worry about it. We don't recommend npm audit and you should use pgutil audit anyway. More comprehensive, covers licenses, all that..

Hi @stefan-kiryazov_8753 ,

Thanks for the inquiry; I've updated our Other feed types docs with a link to this thread.

On a first glance, it looks like a CocoaPod itself is just a basic text file that acts as a "pointer" to a GitHub repository. The pod client uses the git client to "download" files and tags for versioning. In other words, there is no package file.

A CocoaPod repository is a Git repository, and the pod client seems to just use git client to commit/push files to the repo. In other words, there is no API for ProGet to implement.

Here is what a CocoPod repo looks like (note how it's just a bunch of files in a Git repo):
https://github.com/CocoaPods/Specs/tree/master/Specs

To make a "private repo", you basically just create a Git repository:
https://guides.cocoapods.org/making/private-cocoapods

This means that ultimately to implement a CocoPods repository, your only option is to create a private Git repository? That's my inital assessment at least.

And of course, we have no plans to add Git source code hosting to ProGet :)

From here, I recommend to ask the developers to research this a bit more, and maybe contribute their thoughts? It just doesn't seem like the iOS devs uses packages - it's all very open-source and just GitHub repositories.

Steve

@daniel-scati we'll also get this fixed via PG-2635 in an upcoming maintenance release (hopefully 2024.2), which is targeted for next Friday.

Hi @daniel-scati , thanks for the analysis!

We'll get this fixed via PG-2635 in an upcoming maintenance release (hopefully 2024.2), which is targeted for next Friday.

Hi @aharalambopoulos_3520 ,

ProGet works as a "Private Docker Registry" which seems to be different than a "Docker Hub Mirror".

Last time we researched Docker Hub Mirrors, they seemed to be primarily intended to provide image to certain geographic regions (like China) where Docker Hub content would otherwise be restricted. They could also be used to set up a "local mirror" of Docker Hub, but in all cases, it seemed to basically just redirect traffic from the default docker.io URL (or whatever) - so it wasn't intended to be used as "Private Docker Registries".

In any case, Mirrors don't seem to be a good fit for ProGet; instead, if you wish to use nginx, we would advise to "privatizing" and "lock" images using sematic tag, so that you can be assured that corp.local/images/nginx is a tested/safe image with tags you control.

Best,
Steve

@jw excellent, thanks for the finds! I also added this to our "final touches" for ProGet 2024 to address all these-- all pretty easy fixes I think

Hi @jw ,

I'm afraid this is a bit too granular for us now, but it's something we can consider re-evaluating down the line, especially as we will likely want to add specialize permissions for projects, policies, etc. We expect that will happen later in the year, after ProGet 2024's new features get more adoption. We'll see if anyone else requests package-level permissions, etc.

As for the Advanced, I put a note in ProGet 2024's final touches to address that. Honestly I thought those were only on Debug builds only, but clearly not... thanks for reporting!

Cheers,
Steve

Hi @proget-markus-koban_7308 ,

It's highly unlikely we would consider implementing anything Keycloak-specific, but if it's something that SAML supports - and something done by the major providers like Azure, Ping ID, Okta, etc -- we definitely consider it. We just don't know much about it.

We haven't done any further research since that post and we likely won't do any further research on our own, since only one user asked in a few years (and they ended up not needing it anyway).

If this is something you'd be interested in exploring, it'd be best to collaborate and help us bridge the gap between SAML and ProGet.

Here's some relevant questions/discussion from that topic:

I'm not so familiar with SAML behind the scenes... do you know how "SAML group claims" work? For example...

• Is it something that comes back in the XML response, or does it require a separate request?
• What do the "group claims" look like? Like a list of human-readable group names?

And them most importantly... what should ProGet do with such claims upon receipt? Treat the user as if they're in the group (kind of like LDAP groups), and allow permissions to be assigned against that group (like LDAp, but without searching)?

The hardest part is going to be figuring out how to set this up in a SAML provider, document it, etc.

Thanks,
Steve

Hi @philippe-camelio_3885 ,

No idea I'm afraid; there's clearly some issue with unexpected data coming from your ProGet server that's not being validated.

Can you share the results of /health (first API call) then /api/management/feeds/list (using the API token you specified)?

With that we can hopefully spot something.

Thanks,
Steve

Hi @bbalavikram ,

The framework option may not do what you think; first and foremost, 4.5 is not a valid option for framework. You need to use a "framework monifier" that's defined here:
https://learn.microsoft.com/en-us/dotnet/standard/frameworks

But keep in mind that framework monifier must also be in your project file. The framework argument for dotnet simply selects which of the frameworks in your project file to build. It's really only useful for multi-targeted builds, which you probably don't have.

It's possible that dotnet simply will not work with your project. This is unfortunately the case with many old projects. You can continue to try to "play" with your csproj files to try get it to work (note: you can run the same dotnet commands on your workstation).

If you can't get it to work, then you'll need to use MSBuild:Build-Project or DevEnv::Build. We do not have script templates from these, but you can convert your script template to OtterScript and then try modifying the script that way.

Here is some information on build scripts:
https://docs.inedo.com/docs/buildmaster-platforms-dotnet#creating-build-scripts

Best,
Steve

Hi @jw ,

We'll add this as a "if time" on our ProGet 2024 roadmap... and hopefully it's as simple as just setting that flag. We'll update as we get closer to the final release.

Cheers,
Steve

Hi @bbalavikram ,

This error message is coming from the dotnet command-line tool, and I think it has something to with an old/legacy project file format. If you were to run the same command on your workstation, you would get the same rror.

From here, I would compare/contrast the .csproj files in the broken projects, and see if you can figure out what's the difference.

Note that if you search "root element is visualstudioproject expected project", you'll see a lot of people have a similar error, but their solution is also to do similar things - i.e. edit the file and fix the format.

Once you fix the project file, if you check the code back into Git, it shoudl work the next time.

Best,
Steve

Hi @dan-brown_0128 , @gdivis ,

We received a support ticket that seems to be a similar topic...

I am using a ProGet Debian feed to mirror http://deb.debian.org/debian. I am using it on Debian containers running on my desktop. I have noticed that packages that I had installed already once are no longer available for a second installation. I have attached a screenshot that uses the command "apt update && apt install -y git" as example. According to the error message, the package "git" is unknown.
However when looking it up in the ProGet feed, it is listed as a cached package as it was installed in an earlier execution of the container. I have then set a retention rule on the feed to delete absolutely all packages and run the job manually to clean all packages cached on the feed. When re-executing the exact same command, the package git and its dependencies are found and installed.

It appears that cached packages are no longer returned to the Linux instance when executing "apt update". I can force a complete cleanup of the ProGet feed prior to an installation as a workaround but this is a bit tedious.

So I wonder if this has something to do with how we're generating indexes, or something to that effect?

I also asked the other user for input - but we're currently stuck trying to reproduce this, so any other insight would be helpful.

Steve

Hi @dan-brown_0128 ,

The error that you discovered is about all that would be logged on ProGet; you'll need to use a tool like ProcMon to monitor file system activity for the processes.

Best,
Steve

Hi @dan-brown_0128 ,

I'm afraid that's not really possible; the sqllite databases are stored in the package store, which needs to be shared/common across servers.

Cheers,
Steve

Hi @andy222 ,

This was fixed in 2023.11 via BM-3933; as a work-around, you can see the global pipelines by clicking "Shared (Global)" under any application > settings > pipelines.

Cheers,
Steve

Hi @dan-brown_0128,

Hmm, that's strange, and I can't imagine anything in ProGet that would yield the behavior that you're describing. We've been following this pattern for a few years now and haven't had any issues like this. Behind the scenes, here is what's happening:

1. If the index.sqllite3 file exists, then ProGet attempts to opens it
2. If the file cannot be opened (corrupt, old schema version, etc), then it is deleted
3. ProGet then instructs SqlLite library to open a database with the file;
4. If the file does not exist, SQL Lite will create it

There's really no difference between writing to a disk path or a UNC path - that's all handled at the operating system level. Few things to note...

• A health check for a Debian2 feed simply downloads and updates the index file - i.e. it's the same thing that happens when you browse the feed the UI.
• Connector Health Check is performed by the ProGet Service, which is a separate program than the ProGet Web application

Cheers,
Steve

Hi @dan-brown_0128,

ProGet uses SqlLite databases as a cache for a variety of operations. I would first try to delete and the recreate the connector, to see if it makes any difference. That will create a new disk path.

If ProGet does indeed have full control over that directory, you'd need to use some sort of file system monitoring tool like ProcMon to see what activity is happening when attempting to create/open the database file.

It wouldn't surprise me if there is some kind of "malware" or "rootkit" installed on your server that is interfering with ProGet's operation. We see these cause all sorts of strange problems, especially when they start "quarantining" package files that users upload because they contain "dangerous executable code" like .dll files

Best,
Steve

Hi @dan-brown_0128 ,

Thanks for the stack trace and additional information; that definitely helps pinpoint the issue.

Debian require that the entire remote repository is downloaded (which can sometimes take more than 10 seconds on slow connections), and this repository information is cached in a disk-based SQL Lite database. And that error message is coming from SqlLite, which is implying some kind of file access issue.

The database is stored in «Storage.DebianPackagesLibrary»\C«connector-id»\index.sqlite3", and you can find the values under Admin > Advanced Settings.

Do you see that index.sqllite3 file on disk? Any reason that ProGet wouldn't be able to write to that directory?

Thanks,
Steve

Hi @v-makkenze_6348,

thanks for the report, I was able to reproduce it with that particular package. We plan to resolve this via PG-2587 in the next maintenance release scheduled for next Friday.

Hi @dan-brown_0128 ,

Yes, but only once you've enabled ProGet 2024 Vulnerability Preview features (available in ProGet 2023.29+).

Thanks,
Steve

Hi @bbalavikram ,

Sorry about that! How strange..... it looks like there was a weird issue with your MyInedo account, and it was missing some internal data. This lead to an error in generating the key with your email address.

I corrected your account, so please try again (etiher from BuildMaster or MyInedo).

And let us know if you have ay qeustions about BuildMaster too -- happy to help!

Best,
Steve

Hi @v-makkenze_6348,

We've rewritten this to not use MERGE, which should help; we plan to ship PG-2583 in the upcoming maintenance release on Mar 1.

Cheers,
Steve

Hi @davidroberts63 ,

That sounds like a neat use case :)

The ProGet Health API will report the version of a ProGet instance, and you can use the Universal Package API - List Packages Endpoint to query our products feed for the ProGet package.

Cheers,
Steve

Hi @c-schuette_7781 ,

This error is occurring on the remote server (i.e. nuget.devexpress.com). This error can happen when a server is overloaded... so you're basically doing a DoS on DevExpress's server. You'll need to try again or contact DevExpress for help,.

I believe that DevExpress wrote their own, custom NuGet server. We've had several issues with it in the past. While talking to them, you should also suggest they switch to ProGet ISV Edition like some other component vendors

Best,
Steve

Hi @c-schuette_7781,

I have a NuGet "default" feed that is connected to the standard nuget.org feed and also includes connectors to three "local" NuGet feeds. I

So basically, you're doing a Denial of Service attack against your ProGet server ;)

When the NuGet client makes that FindPackagesById() request, ProGet needs to now make four separate web requests (nuget.org plus the three other feeds). Considering that NuGet client makes 100's of simultaneous requests for different packages, you're going to run into errors like this. Especially with multiple builds (multiple sets of 100's of requests / second).

If you want to handle this level of traffic, you need to use load balancing. See How to Prevent Server Overload in ProGet to learn more.

Otherwise, you need to reduce traffic. Switch to the NuGet v3 API, use connector metadata caching, reduce the number of connectors, set a Web.ConcurrentRequestLimit in the admin > advanced, etc.

Hi @pallavi-tarigonda_9617,

Adding an entry to the hosts file should not cause a "blocked" connection, so it sounds like there's definitely something strange going with your machine's configuration. I'm not sure how to troubleshoot this further. Self-connectors work fine in testing Free edition, and other users don't have an issue.

If it helps, here's the code that ProGet uses to determine if a connection is local:

public bool IsLocal
{
    get
    {
        var connection = this.NativeRequest.HttpContext.Connection;
        if (connection.RemoteIpAddress != null)
        {
            if (connection.LocalIpAddress != null)
                return connection.RemoteIpAddress.Equals(connection.LocalIpAddress);
            else
                return IPAddress.IsLoopback(connection.RemoteIpAddress);
        }

        if (connection.RemoteIpAddress == null && connection.LocalIpAddress == null)
            return true;

        return false;
    }
}

I would explore the hosts file issue; the fact that a loopback (127.0.0.1) entry wouldn't work sounds like there was some kind of data entry error/typo in your hosts error, but hard to say.

Best,
Steve

Hi @philippe-camelio_3885 ,

This behavior is expected, though a little confusing. The blue info box explains it a little bit, but if you add a second PGVC vulnerability source, then you'll see two entries for PGVC in your list. Those are separate sources that point to the same database. It's not recommended, and only acts as a work-around to allow for different aassessments for different feeds..

What are you trying to accomplish? If it's just basic vulnerability scanning, then I recommend doing the following:

1. remove everything
2. Disable PGVC
3. Enable PVGC

Hope that helps,

Steve

Hi @m-webster_0049 ,

The first thing I would try is to troubleshoot this is to switch to a very basic API key like hello. That just eliminates any typos, spacing, etc.

Next, I would try specifying the API Key via X-ApiKey header (see docs) - just to see if you get a different error. It's possible there is a regression somewhere.

Best,
Steve

Hi @philippe-camelio_3885 ,

Can you share a screenshot of your Admin > Vulnerability Sources screen? It looks like you have three vulnerability sources configured.

Note that we no longer recommend using OSS Index, and instead just having (one) PGVC enabled.

Thanks,
Steve

Hello,

That makes sense; there's a few threads with this similar issue, so you may want to search and follow some of the troubleshooting steps.

But basically, ProGet checks for local requests using HttpRequest.IsLocal, which basically just looks for 127.0.0.1. If it's not local, then a license violation is recorded.

Try using 127.0.0.1 for your connectors, or if that's not possible, and your server doesn't resolve proget.xxxx.com as 127.0.0.1, you may need to add a /etc/hosts entry for proget.xxxx.com 127.0.0.1so that it will come across as a local.

Cheers,
Steve

Hi @jw ,

There is one other setting, under SCA > Vulnerabilities > Download Blocking. Try setting that, then maybe you'll also need to run Package Analysis again.

Let us know -- we can try to add a few more hints/clues in the UI to make this less confusing, at least as a temporary measure before tying this together better in the back-end.

Thanks,
Steve

Hi @jw ,

One thing to check --- is "vulnerability blocking" enabled on the nuget-proxy feed? That's currently how the SCA Projects know to pick up if a vulnerability issue is desired.

Thanks,
Steve

Hi @hashim-abu-gellban_3562,

I see a few issues here...

First, the URL you're using is not correct; the easiest way to to find the url by clicking the "download package link" in the ProGet UI. It will look like this: /nuget/your-feed-name/package/your-package-name/your-version-number

Second, you're downloading a file - so you want to use a powershell command like this:

$url = "https://myprogetserver/feeds/mynuggets/package/mypackage/1.0.0"
$destination = "c:\mypackages\mypackage-1.0.0.zip"
Invoke-WebRequest -Uri $url -OutFile $destination

Best,
Steve

@jw thanks for the bug report!

We'll get this fixed in the an upcoming maintenance release via PG-2491 :)

Hi @carl-westman_8110 ,

This is likely due to some authentication or other configuration with Azure Blob. You will see the specific error on the ProGet, logged under Admin > Diagnostic Center.

Best,
Steve

Hi @avoisin_5738 ,

The error message means that an invalid zip file was received in the request; so the file can't even be opened. I don't know how Klondike works.

If you're totally sure that you're uploading the nupkg file, I would try opening it as a zip file (like rename it to .zip, use 7zip, etc.). I would expect a similar error.

If it's a valid zip file, I would upload it via the UI; if that works, it means your script has some kind of issue in corrupting the stream, or not sending the complete file, etc.

Steve

Hi @avoisin_5738 ,

The error "End of Central Directory record could not be found" basically means that the file is not a valid ZIP file. The most common case for this is pushing the wrong file (.nuspec instead of .nupkg, or a dll or .psm file). There are some other rare cases where the stream can be corrupted on the way to ProGet, but that's not common.

Hope that helps,
Steve