Unverified/not approved chocolatey package categorized with Vulnerabilities:None

[svc-4x9p2a_6341]

We tested the download of a "flagged" or at least "not approved" package from Chocolatey, but proget does not flag it as vulnerable and it is not clearly visible, that there are issues related to this package:

choco install crystalreports2008runtime

Chocolatey Report:
Some Checks Have Failed or Are Not Yet Complete
Not All Tests Have Passed
• Validation Testing Unknown
• Verification Testing Failed
• Details

Scan Testing Resulted in Flagged:
This package was submitted (and approved) prior to automated virus scanning integration into the package moderation processs.
We recommend clicking the "Details" link to make your own decision on installing this package.

The Chocolatey API returns the following information:

<d:IsApproved m:type="Edm.Boolean">false</d:IsApproved>
<d:PackageValidationResultStatus>Unknown</d:PackageValidationResultStatus>
<d:PackageScanStatus>Flagged</d:PackageScanStatus>
<d:PackageScanFlagResult>Unknown</d:PackageScanFlagResult>

In such cases, we would expect a vulnerability alert in Proget and a blocked download. Instead, Proget downloads this package and doesn't flag it at all.

We kindly ask Inedo for confirmation on whether this behavior is a bug or a known limitation in the current version and if it will be addressed.

[atripp]

Hi @svc-4x9p2a_6341 ,

First and foremost, Chocolatey does not incorporate "Vulnerabilities" (i.e. centrally aggregated reports of vendor-reported weaknesses in software) into the package ecosystem. This is just not something that's a part of the Windows ecosystem as a whole, unlike the Linux ecosystem (e.g. Ubuntu OVALs).

Chocolatey does, however, perform automated malware/virus scanning on packages. That's a totally different thing... please read our How Virus Scanning in Chocolatey Works article to learn more.

From a technical standpoint, ProGet will use (abuse?) the vulnerability subsystem to treat "flagged" packages as vulnerable. This was a "quick and dirty" way for us to experiment with exposing this data through ProGet without having to build an entirely new subsystem just for Chocolatey packages.

As for crystalreports2008runtime, it did not fail the virus/malware checking, so it's not going to be seen as "vulnerable" by ProGet. Instead, it hasn't been "validated" by Chocolatey's automated system. That's a different feature altogether (i.e. unrelated to virus checking) - and that ancient crystal reports package long predates the moderation feature in Chocolatey I believe.

In any case, ProGet does not expose nor allow users to "filter" on this validation status, and it's highly unlikely such a capability would add much value to users - especially considering no one has asked for it, and the cost of developing an entirely new, Chocolatey-only feature is nontrivial.

The reason is that everyone internalizes their packages; see Why You Should Privatize and Internalize your Chocolatey Packages
to learn more

Hope that helps, maybe @steviecoaster can assist more.

Cheers,
Alana

[steviecoaster]

Hi,

Fashionably late here, but what @atripp said is accurate. Chocolatey's moderation process on the Chocolatey Community Repository is a completely separate process to the vulnerability management features available in ProGet.

Our moderation process was put in place shortly after the Chocolatey Community Repository went live 15 years ago, and some packages were added to the feed and haven't received updates since then.

While our moderation process is really good, and we've never seen malicious content make it to approved status, you should always detonate-test a package in your own environment before promoting it to a production repository.