Questions about the new ProGet Vulnerability Central (PGVC)

[sebastian...]

ProGet Vulnerability Central (PGVC) was announced for ProGet 2023 and is available as a preview feature since 2022.20. Would you mind sharing some insights on this new feature? I already have a few questions about it:

1. What is the advantage of using PGVC over OSS Index? Two things I could figure out so far are a) you don't need an API key from OSS (they could stop their free service any time) and b) it's updated with each version update of ProGet (which really only is of any significance if the ProGet server wasn't connected to the internet, in which case I would wonder what feeds there would be to scan...?). What would be other advantages? Does PGVC use more sources? Is it updated faster or more frequently than OSS? Is it more accurate, more reliable, ...?

2. The blog post states that PGVC data is available "instantly" while OSS Index needs to be updated over night. What does that mean? It's my understanding that both databases are updated daily. What is the actual difference here?

3. What about migrating from OSS Index to PGVC? We have a lot of assessed vulnerabilities from OSS. Will ProGet match those to the ones from PGVC? Or will we have to re-asses all of them?

4. What about using both services (OSS Index and PGVC) at the same time? Is that possible (it seems possible in the feeds' settings)? Would it make sense to do that, maybe because both services use different sources or maybe as a form of redundancy/backup?

[atripp]

Hello @sebastian ,

Great questions, and I'll do my best to help. This is a little complicated I think :)

[1] PGVC vs OSS Index
From a technical standpoint, PGVC is implemented as an "offline database", which offers a lot of performance benefits - namely ProGet can know about vulnerabilities in packages you're not yet using, and display those on remote packages. ProGet will download updates on a nightly basis.

Regarding the "Quality of data", it's really hard to say. I think everyone just aggregates from the same sources like NVD:

• PGVC leverages the Open Source Vulnerability (OSV) platform developed by Google and backed by Microsoft, etc. It’s an open platform.
• OSS Index is just Sonatype, and it’s closed (proprietary).

We decided to invest in PGVC because OSS Index has been rate limiting more and more, and the quality of results have been declining over the years. We believe PGVC (and the underlying OSV platform) will ultimately be superior.

[2] Instant Availability & Overnight Scanning
As I mentioned above, PGVC is an offline database. This means ProGet can immediately query that database to show you vulnerabilities on packages you may want to use or are currently using. This is not possible with OSS Index due to rate limiting.

The "vulnerability scan job" (which both OSS Index and PGVC scan do) will basically compare all packages you have in ProGet (local/cached) against the vulnerability source. This is to show you about vulnerabilities discovered in pakcages you're using.

[3] Migration
We are planning on some guidance about this. In theory, its should be possible because both the PGVC and OSS Index use CVE-ID. But the OSS Index sometimes uses their own ID instead of a CVE-ID.

We'll study some datasets and see what we can bring over. It might be a SQL Script or a tool inside of PRoGet.

[4] Using Both
I want to say, that you should just pick one source. Otherwise you’ll get a lot of duplicate vulnerabilities. Either one should be sufficient for package scanning, as they both aggregate the same publicly-available data sources.

However, it wouldn't hurt to try using both... just to see what comes up for vulnerabilities. If you delete a vulnerability source, it will delete all the assessments -- so that is a quick way to at least test (you can delete the PGVC vulnerability source).

[sebastian...]

Hi @atripp, thanks for the quick feedback and the insights!

As for the migration part: great to hear that there is something planned. It seems that it would be best for us to wait until some guidance or migration tool is available.

[atripp]

@sebastian no problem :)

Actually... I just realized that I have a backup of your database that you sent to us for analysis/review. And your vulnerability assessments were one of our test datasets to see how migration goes.

So we can give you a very accurate and specific answer once we get to that phase of the ProGet 2023 release

[sebastian...]

@atripp We are currently testing new ProGet 2023 release. Are there any news about the migration from OSS Index to PGVC? Is there a tool, a script or some kind of documentation available?

[stevedennis]

Hi @sebastian ,

We actually used your set to analyze this - and we just couldn't come up with a solution.

The PGVC found more total vulnerabilities than OSS Index did, but without doing some really complex code or machine learning something, we couldn't figure out an simple way to reconcile the two datasets.

Many of the OSS Index vulnerabilities didn't list a CVE number, and the titles and descriptions were different - but it seemed like they were talking about the same problem in the same package.

We gave up after that. Open to ideas for sure!

Thanks,
Steve

[sebastian...]

Hi @stevedennis,

I'm not looking for a 100% perfect migration. Basically, I have a certain number of automatically and manually assessed vulnerabilities, and I would like to keep those assessments, wherever the same vulnerability exists in PGVC. In detail: I'm fine with "loosing" all OSS Index vulnerabilities. I'm also fine with seeing a larger number of new PGVC vulnerabilities after switching from OSS Index to PGVC, even if those aren't really "new", but already existed as an OSS Index vulnerability. What I would expect from a migration is this:

1. Auto assess all new PGVC vulnerabilities based on the "Auto assess" option configured for assessment types (I guess this should work anyway and have nothing to do with migrating from one source to another).
2. If a PGVC vulnerability can be matched to an OSS Index vulnerability (based on a CVE number), apply the assessment of the old vulnerability to the new one. If it can't be matched, rule 1 applies.

However, as I was writing this, I was having a look at our current list of vulnerabilities and it looks like we have lost a larger number of them. I currently only have 182 vulnerabilities, and only 5 of those are manually assessed. Is there any mechanism that deletes old vulnerabilities? I know for a fact that I have manually assessed way more vulnerabilities than that to unblock the corresponding packages. None of our assessment types have an expiration, but those should only remove the assessment, not the vulnerability itself, right?

[stevedennis]

Hi @sebastian,

Thanks for clarifying that :)

[1] auto-assess should work just fine; that would be our recommendation anyway

[2] this is what we thought too, but there were just so few that this would have worked on that we gave up

So in that case, you can just enable PGVC, enabled download blocking on the feed, and then you'll get all the new PGVC vulnerabilities added to the system after running a Vulnerability Download scheduld job.

If you delete the OSSIndex source, then all the vulnerabilities/assessments will be deleted.

There was a very long-standing bug where ProGet wouldn't update or delete a vulnerability if it was updated/deleted at the source. We fixed that in 2023.

Now I can't say for certain if that's what happened here... but we noticed that some similar erroneous vulnerabilities -- like a vulnerability with a mangled title or some other data entry problem -- disappeared after a nightly scan.

FYI - expiration dates won't delete the assessments, it'll just consider them invalid

Hope that helps!

[sebastian...]

Hi @stevedennis,

thanks for the quick reply. I was just confused, because all of our vulnerabilities seem to have CVE IDs, so I thought that all of them should be suitable for matching between different sources. But given that there seem to be only 5 manually assessed vulnerabilities left, I guess there isn't really much to actually migrate. We'll try simply switching from OSS Index to PGVC and see what happens...

[sebastian...]

Hi @stevedennis,

I have enabled PGVC, but the outcome was not exactly as expected:

1. I activated PGVC for our Nuget feed, but I did not delete the OSS Index source, because I wanted a side-by-side comparison of the vulnerabilities. Yet, all my OSS Index vulnerabilities are gone (even the ones for our npm feed).

2. Almost half (274 out of 566) of the vulnerabilities reported by PGVC do not have a CVSS score (OSS Index vulnerabilities almost always have a CVSS score). As a consequence, those vulnerabilities are not automatically assessed.

3. PGVC vulnerabilities do not display the CVE number or a CWE, and in many cases do not provide links to external sources like the NVD.

Is all of that intended and by design?

[stevedennis]

Hi @sebastian ,

[1] That definitely doesn't sound right; that didn't happen when we tested, so we'll have to check that out, it could be a bug...

[2] 274/566 seems awfully high; several do not have scores, but since we have to compute the score ourselves with equations like these, it's very possible that the underlying data isn't formatted perfect or there's a bug somewhere -- can you share the examples you found so we can investigate?

[3] This is expected; it seems that many (or most) vulnerabilities in the database do not have a conspicuous CVE number (perhaps they're not CVEs??), and in those cases, the descriptions are very thorough... it's a huge dataset so we're still learning what's in it.

Cheers,
Steve

[sebastian...]

@stevedennis said in Questions about the new ProGet Vulnerability Central (PGVC):

[2] 274/566 seems awfully high; several do not have scores, but since we have to compute the score ourselves with equations like these, it's very possible that the underlying data isn't formatted perfect or there's a bug somewhere -- can you share the examples you found so we can investigate?

Sure, what do you need? Here is a little snippet from the Vulnerabilities_Extended view, filtered on Score is null:

External_Id	Package_Name	Package_Versions	Title_Text

GHSA-23cv-jh4v-vffm	Microsoft.AspNetCore.App.Runtime.win-x64	>=3.1.0 <3.1.1	Denial of service in ASP.NET Core
GHSA-23cv-jh4v-vffm	Microsoft.AspNetCore.App.Runtime.win-x86	>=3.1.0 <3.1.1	Denial of service in ASP.NET Core
GHSA-23cv-jh4v-vffm	Microsoft.AspNetCore.Http.Connections	>=1.0.0 <1.0.15	Denial of service in ASP.NET Core
GHSA-2c7v-qcjp-4mg2	Microsoft.WindowsDesktop.App.Runtime.win-x64	>=7.0.0 <7.0.1	.NET Remote Code Execution Vulnerability
GHSA-2c7v-qcjp-4mg2	Microsoft.WindowsDesktop.App.Runtime.win-x86	>=7.0.0 <7.0.1	.NET Remote Code Execution Vulnerability
GHSA-2xjx-v99w-gqf3	Microsoft.NETCore.App	>=2.2.0 <2.2.1	Exposure of Sensitive Information in System.Net.Http
GHSA-35hc-x2cw-2j4v	System.Security.Cryptography.Xml	<4.4.2	Denial of service vulnerability exists when .NET and .NET Core improperly process XML documents
GHSA-365p-96qv-xr7g	Microsoft.AspNetCore.HttpOverrides	>=2.0.0 <2.0.2	ASP.NET Core allow an elevation of privilege
GHSA-365p-96qv-xr7g	Microsoft.AspNetCore.Server.Kestrel.Core	>=2.0.0 <2.0.2	ASP.NET Core allow an elevation of privilege
GHSA-3m2r-q8x3-xmf7	Microsoft.AspNetCore.Server.Kestrel.Core	>=2.0.0 <2.0.3	Moderate severity vulnerability that affects Microsoft.AspNetCore.All, Microsoft.AspNetCore.Server.Kestrel.Core, Microsoft.AspNetCore.Server.Kestrel.Transport.Abstractions, and Microsoft.AspNetCore.Se
GHSA-3m2r-q8x3-xmf7	Microsoft.AspNetCore.Server.Kestrel.Transport.Abstractions	>=2.0.0 <2.0.3	Moderate severity vulnerability that affects Microsoft.AspNetCore.All, Microsoft.AspNetCore.Server.Kestrel.Core, Microsoft.AspNetCore.Server.Kestrel.Transport.Abstractions, and Microsoft.AspNetCore.Se
GHSA-3rp6-rjw4-cq39	Microsoft.AspNetCore.Mvc.Core	>=1.1.0 <1.1.6	Cross-origin Resource Sharing bypass in ASP.NET Core
GHSA-3rp6-rjw4-cq39	Microsoft.AspNetCore.Mvc.Cors	>=1.1.0 <1.1.6	Cross-origin Resource Sharing bypass in ASP.NET Core
GHSA-3wcj-rg8q-9cqv	Microsoft.AspNetCore.Mvc.Core	>=2.0.0 <2.0.1	Open redirect in ASP.NET Core
GHSA-5633-f33j-c6f7	Microsoft.NETCore.App	>=2.1.0 <2.1.7	Tampering vulnerability in .NET Core
GHSA-5f2m-466j-3848	System.Private.Uri	>=4.3.0 <4.3.2	Denial of service in ASP.NET Core
GHSA-655q-9gvg-q4cm	Microsoft.AspNetCore.App.Runtime.win-x64	>=3.1.0 <3.1.1	Remote code execution in ASP.NET Core
GHSA-655q-9gvg-q4cm	Microsoft.AspNetCore.App.Runtime.win-x86	>=3.1.0 <3.1.1	Remote code execution in ASP.NET Core
GHSA-655q-9gvg-q4cm	Microsoft.AspNetCore.Http.Connections	>=1.0.0 <1.0.15	Remote code execution in ASP.NET Core
GHSA-6px8-22w5-w334	Microsoft.AspNetCore.Server.Kestrel.Core	>=2.1.0 <2.1.7	Denial of service in ASP.NET Core

Googling the External_Id of the first entry (GHSA-23cv-jh4v-vffm) leads me to this GitHub Advisories entry, which links to this NVD entry, which mentions a score of 7.5.
If you need more data, I can submit a support ticket and upload an export there.

[gdivis]

Hi @sebastian ,

Regarding [1], I can't reproduce this behavior. When I add OSS Index I see vulnerabilities from it and also from PGVC, though this is only on the latest v2023 build, so maybe this works differently on another version?

For [2], you are correct. There are a lot more of these in the dataset than we initially thought. We plan to resolve this by filling in the missing data from NVD, though I can't give an estimate on when we will have that (maybe @apxltd can give you a rough idea of our schedule).

Finally, for [3], I think we will end up storing a CVE for most of these to help populate missing CVSS scores if for no other reason. I think this would be part of updates we do to address [2].

-Greg

[apxltd]

Hi @sebastian,

As @gdivis mentioned, it will be a bit of an undertaking to fill-in the missing data (in particular the ID/Scores) using the NVD datasets. We are also exploring aggregating some additional datasets as well, in particular ones for system packages (Debian, RPM), so we can incorporate container image scanning directly in ProGet.

It seems our userbase (and the community at large) is slowly starting to "grok" vulnerabilities, but it's quite a ways off.

We've got some other work ahead of this, so I'm going to say perhaps late June / early July is when we can consider picking up on this again. Looking forward to any feedback you have in the mean time :)

Thanks,
Alex

[sebastian...]

Hi @apxltd,

I think having scores is crucial. Otherwise we would have to manually assess a lot of vulnerabilities by hand.

CVE-IDs aren't really that important. I guess we would only use them to find more information on the issue by Googling them. What would be nice: having links to sources, like a GitHub advisory entry, an NVD entry, ...

CWEs might become interesting at some point, if those would become available as a separate field. One could think of some kind of statistics on categories of vulnerabilities.

One more thing we noticed, but this is a general problem, not related to PGVC per se: I can't see information on vulnerabilities or deprecated packaged in Visual Studio. Nuget added support for vulnerabilities about 2 years ago. As a result, the dotnet client or Visual Studio can display on vulnerabilities of packages.

When I look at the same solution and list of packages using our Nuget proxy feed in ProGet, I don't seen any of that info:

I would assume that Nuget expanded their API to support this at some point and ProGet does not support that functionality yet. Are their any plans to support this?

[apxltd]

Hi @sebastian ,

Thanks for the feedback; that's my understanding of the scores. The main value is time savings in manual assessment. I wish it was easier to get in, and it's too bad our datasets don't have it. We'll need to review this in the coming months, since it involves a lot more work on our end to refactor things.

As for Visual Studio, one of our engineers here just prototyped that, and I think we'll do it. That's much simpler and easy to sneak in a maintenance release :)

In the past, it wasn't possible in Visual Studio before (they used a different API that was nuget.org-only), and it wasn't feasible in ProGet due to the indexing. But now it's much easier.

I think this will be one of those "freemium preview" features. Not that it impacts you, but we're thinking that free users will get a link to a page in ProGet that's like "vulnerabilities are paid feature, here are the benefits, etc. To see which vulns the linked package has, navigate to feed, click here, etc. Paid users can assess, block, etc."

Paid users would just link to the package or vulnerability or something. Just our idea figured I'd share ;)

Cheers,
Alex

[sebastian...]

@apxltd said in Questions about the new ProGet Vulnerability Central (PGVC):

As for Visual Studio, one of our engineers here just prototyped that, and I think we'll do it. That's much simpler and easy to sneak in a maintenance release :)

Sounds great, looking forward to it

[sebastian...]

Hi @apxltd,

Wow, that was fast... I noticed that this was apparently already implemented in 2023.5 (PG-2359). Just wanted to let you know that I tested this with 2023.6 and it works like a charm