Questions about the new ProGet Vulnerability Central (PGVC)

[atripp]

@sebastian no problem :)

Actually... I just realized that I have a backup of your database that you sent to us for analysis/review. And your vulnerability assessments were one of our test datasets to see how migration goes.

So we can give you a very accurate and specific answer once we get to that phase of the ProGet 2023 release

[sebastian...]

@atripp We are currently testing new ProGet 2023 release. Are there any news about the migration from OSS Index to PGVC? Is there a tool, a script or some kind of documentation available?

[stevedennis]

Hi @sebastian ,

We actually used your set to analyze this - and we just couldn't come up with a solution.

The PGVC found more total vulnerabilities than OSS Index did, but without doing some really complex code or machine learning something, we couldn't figure out an simple way to reconcile the two datasets.

Many of the OSS Index vulnerabilities didn't list a CVE number, and the titles and descriptions were different - but it seemed like they were talking about the same problem in the same package.

We gave up after that. Open to ideas for sure!

Thanks,
Steve

[sebastian...]

Hi @stevedennis,

I'm not looking for a 100% perfect migration. Basically, I have a certain number of automatically and manually assessed vulnerabilities, and I would like to keep those assessments, wherever the same vulnerability exists in PGVC. In detail: I'm fine with "loosing" all OSS Index vulnerabilities. I'm also fine with seeing a larger number of new PGVC vulnerabilities after switching from OSS Index to PGVC, even if those aren't really "new", but already existed as an OSS Index vulnerability. What I would expect from a migration is this:

1. Auto assess all new PGVC vulnerabilities based on the "Auto assess" option configured for assessment types (I guess this should work anyway and have nothing to do with migrating from one source to another).
2. If a PGVC vulnerability can be matched to an OSS Index vulnerability (based on a CVE number), apply the assessment of the old vulnerability to the new one. If it can't be matched, rule 1 applies.

However, as I was writing this, I was having a look at our current list of vulnerabilities and it looks like we have lost a larger number of them. I currently only have 182 vulnerabilities, and only 5 of those are manually assessed. Is there any mechanism that deletes old vulnerabilities? I know for a fact that I have manually assessed way more vulnerabilities than that to unblock the corresponding packages. None of our assessment types have an expiration, but those should only remove the assessment, not the vulnerability itself, right?

[stevedennis]

Hi @sebastian,

Thanks for clarifying that :)

[1] auto-assess should work just fine; that would be our recommendation anyway

[2] this is what we thought too, but there were just so few that this would have worked on that we gave up

So in that case, you can just enable PGVC, enabled download blocking on the feed, and then you'll get all the new PGVC vulnerabilities added to the system after running a Vulnerability Download scheduld job.

If you delete the OSSIndex source, then all the vulnerabilities/assessments will be deleted.

There was a very long-standing bug where ProGet wouldn't update or delete a vulnerability if it was updated/deleted at the source. We fixed that in 2023.

Now I can't say for certain if that's what happened here... but we noticed that some similar erroneous vulnerabilities -- like a vulnerability with a mangled title or some other data entry problem -- disappeared after a nightly scan.

FYI - expiration dates won't delete the assessments, it'll just consider them invalid

Hope that helps!

[sebastian...]

Hi @stevedennis,

thanks for the quick reply. I was just confused, because all of our vulnerabilities seem to have CVE IDs, so I thought that all of them should be suitable for matching between different sources. But given that there seem to be only 5 manually assessed vulnerabilities left, I guess there isn't really much to actually migrate. We'll try simply switching from OSS Index to PGVC and see what happens...

[sebastian...]

Hi @stevedennis,

I have enabled PGVC, but the outcome was not exactly as expected:

1. I activated PGVC for our Nuget feed, but I did not delete the OSS Index source, because I wanted a side-by-side comparison of the vulnerabilities. Yet, all my OSS Index vulnerabilities are gone (even the ones for our npm feed).

2. Almost half (274 out of 566) of the vulnerabilities reported by PGVC do not have a CVSS score (OSS Index vulnerabilities almost always have a CVSS score). As a consequence, those vulnerabilities are not automatically assessed.

3. PGVC vulnerabilities do not display the CVE number or a CWE, and in many cases do not provide links to external sources like the NVD.

Is all of that intended and by design?

[stevedennis]

Hi @sebastian ,

[1] That definitely doesn't sound right; that didn't happen when we tested, so we'll have to check that out, it could be a bug...

[2] 274/566 seems awfully high; several do not have scores, but since we have to compute the score ourselves with equations like these, it's very possible that the underlying data isn't formatted perfect or there's a bug somewhere -- can you share the examples you found so we can investigate?

[3] This is expected; it seems that many (or most) vulnerabilities in the database do not have a conspicuous CVE number (perhaps they're not CVEs??), and in those cases, the descriptions are very thorough... it's a huge dataset so we're still learning what's in it.

Cheers,
Steve

[sebastian...]

@stevedennis said in Questions about the new ProGet Vulnerability Central (PGVC):

[2] 274/566 seems awfully high; several do not have scores, but since we have to compute the score ourselves with equations like these, it's very possible that the underlying data isn't formatted perfect or there's a bug somewhere -- can you share the examples you found so we can investigate?

Sure, what do you need? Here is a little snippet from the Vulnerabilities_Extended view, filtered on Score is null:

External_Id	Package_Name	Package_Versions	Title_Text

GHSA-23cv-jh4v-vffm	Microsoft.AspNetCore.App.Runtime.win-x64	>=3.1.0 <3.1.1	Denial of service in ASP.NET Core
GHSA-23cv-jh4v-vffm	Microsoft.AspNetCore.App.Runtime.win-x86	>=3.1.0 <3.1.1	Denial of service in ASP.NET Core
GHSA-23cv-jh4v-vffm	Microsoft.AspNetCore.Http.Connections	>=1.0.0 <1.0.15	Denial of service in ASP.NET Core
GHSA-2c7v-qcjp-4mg2	Microsoft.WindowsDesktop.App.Runtime.win-x64	>=7.0.0 <7.0.1	.NET Remote Code Execution Vulnerability
GHSA-2c7v-qcjp-4mg2	Microsoft.WindowsDesktop.App.Runtime.win-x86	>=7.0.0 <7.0.1	.NET Remote Code Execution Vulnerability
GHSA-2xjx-v99w-gqf3	Microsoft.NETCore.App	>=2.2.0 <2.2.1	Exposure of Sensitive Information in System.Net.Http
GHSA-35hc-x2cw-2j4v	System.Security.Cryptography.Xml	<4.4.2	Denial of service vulnerability exists when .NET and .NET Core improperly process XML documents
GHSA-365p-96qv-xr7g	Microsoft.AspNetCore.HttpOverrides	>=2.0.0 <2.0.2	ASP.NET Core allow an elevation of privilege
GHSA-365p-96qv-xr7g	Microsoft.AspNetCore.Server.Kestrel.Core	>=2.0.0 <2.0.2	ASP.NET Core allow an elevation of privilege
GHSA-3m2r-q8x3-xmf7	Microsoft.AspNetCore.Server.Kestrel.Core	>=2.0.0 <2.0.3	Moderate severity vulnerability that affects Microsoft.AspNetCore.All, Microsoft.AspNetCore.Server.Kestrel.Core, Microsoft.AspNetCore.Server.Kestrel.Transport.Abstractions, and Microsoft.AspNetCore.Se
GHSA-3m2r-q8x3-xmf7	Microsoft.AspNetCore.Server.Kestrel.Transport.Abstractions	>=2.0.0 <2.0.3	Moderate severity vulnerability that affects Microsoft.AspNetCore.All, Microsoft.AspNetCore.Server.Kestrel.Core, Microsoft.AspNetCore.Server.Kestrel.Transport.Abstractions, and Microsoft.AspNetCore.Se
GHSA-3rp6-rjw4-cq39	Microsoft.AspNetCore.Mvc.Core	>=1.1.0 <1.1.6	Cross-origin Resource Sharing bypass in ASP.NET Core
GHSA-3rp6-rjw4-cq39	Microsoft.AspNetCore.Mvc.Cors	>=1.1.0 <1.1.6	Cross-origin Resource Sharing bypass in ASP.NET Core
GHSA-3wcj-rg8q-9cqv	Microsoft.AspNetCore.Mvc.Core	>=2.0.0 <2.0.1	Open redirect in ASP.NET Core
GHSA-5633-f33j-c6f7	Microsoft.NETCore.App	>=2.1.0 <2.1.7	Tampering vulnerability in .NET Core
GHSA-5f2m-466j-3848	System.Private.Uri	>=4.3.0 <4.3.2	Denial of service in ASP.NET Core
GHSA-655q-9gvg-q4cm	Microsoft.AspNetCore.App.Runtime.win-x64	>=3.1.0 <3.1.1	Remote code execution in ASP.NET Core
GHSA-655q-9gvg-q4cm	Microsoft.AspNetCore.App.Runtime.win-x86	>=3.1.0 <3.1.1	Remote code execution in ASP.NET Core
GHSA-655q-9gvg-q4cm	Microsoft.AspNetCore.Http.Connections	>=1.0.0 <1.0.15	Remote code execution in ASP.NET Core
GHSA-6px8-22w5-w334	Microsoft.AspNetCore.Server.Kestrel.Core	>=2.1.0 <2.1.7	Denial of service in ASP.NET Core

Googling the External_Id of the first entry (GHSA-23cv-jh4v-vffm) leads me to this GitHub Advisories entry, which links to this NVD entry, which mentions a score of 7.5.
If you need more data, I can submit a support ticket and upload an export there.

[gdivis]

Hi @sebastian ,

Regarding [1], I can't reproduce this behavior. When I add OSS Index I see vulnerabilities from it and also from PGVC, though this is only on the latest v2023 build, so maybe this works differently on another version?

For [2], you are correct. There are a lot more of these in the dataset than we initially thought. We plan to resolve this by filling in the missing data from NVD, though I can't give an estimate on when we will have that (maybe @apxltd can give you a rough idea of our schedule).

Finally, for [3], I think we will end up storing a CVE for most of these to help populate missing CVSS scores if for no other reason. I think this would be part of updates we do to address [2].

-Greg

[apxltd]

Hi @sebastian,

As @gdivis mentioned, it will be a bit of an undertaking to fill-in the missing data (in particular the ID/Scores) using the NVD datasets. We are also exploring aggregating some additional datasets as well, in particular ones for system packages (Debian, RPM), so we can incorporate container image scanning directly in ProGet.

It seems our userbase (and the community at large) is slowly starting to "grok" vulnerabilities, but it's quite a ways off.

We've got some other work ahead of this, so I'm going to say perhaps late June / early July is when we can consider picking up on this again. Looking forward to any feedback you have in the mean time :)

Thanks,
Alex

[sebastian...]

Hi @apxltd,

I think having scores is crucial. Otherwise we would have to manually assess a lot of vulnerabilities by hand.

CVE-IDs aren't really that important. I guess we would only use them to find more information on the issue by Googling them. What would be nice: having links to sources, like a GitHub advisory entry, an NVD entry, ...

CWEs might become interesting at some point, if those would become available as a separate field. One could think of some kind of statistics on categories of vulnerabilities.

One more thing we noticed, but this is a general problem, not related to PGVC per se: I can't see information on vulnerabilities or deprecated packaged in Visual Studio. Nuget added support for vulnerabilities about 2 years ago. As a result, the dotnet client or Visual Studio can display on vulnerabilities of packages.

When I look at the same solution and list of packages using our Nuget proxy feed in ProGet, I don't seen any of that info:

I would assume that Nuget expanded their API to support this at some point and ProGet does not support that functionality yet. Are their any plans to support this?

[apxltd]

Hi @sebastian ,

Thanks for the feedback; that's my understanding of the scores. The main value is time savings in manual assessment. I wish it was easier to get in, and it's too bad our datasets don't have it. We'll need to review this in the coming months, since it involves a lot more work on our end to refactor things.

As for Visual Studio, one of our engineers here just prototyped that, and I think we'll do it. That's much simpler and easy to sneak in a maintenance release :)

In the past, it wasn't possible in Visual Studio before (they used a different API that was nuget.org-only), and it wasn't feasible in ProGet due to the indexing. But now it's much easier.

I think this will be one of those "freemium preview" features. Not that it impacts you, but we're thinking that free users will get a link to a page in ProGet that's like "vulnerabilities are paid feature, here are the benefits, etc. To see which vulns the linked package has, navigate to feed, click here, etc. Paid users can assess, block, etc."

Paid users would just link to the package or vulnerability or something. Just our idea figured I'd share ;)

Cheers,
Alex

[sebastian...]

@apxltd said in Questions about the new ProGet Vulnerability Central (PGVC):

As for Visual Studio, one of our engineers here just prototyped that, and I think we'll do it. That's much simpler and easy to sneak in a maintenance release :)

Sounds great, looking forward to it

[sebastian...]

Hi @apxltd,

Wow, that was fast... I noticed that this was apparently already implemented in 2023.5 (PG-2359). Just wanted to let you know that I tested this with 2023.6 and it works like a charm