Problem with Vulnerabilities in docker with Clair

[w.repinski_1472]

I have a problem with handling and categorizing (score/assessment/severity) vulnerabilities in feeds with containers.

Despite the set thresholds for assessment, it does not work.

Tested from Clair 2.1.2, 2.1.7, and currently 2.1.8
ProGet versions 2022.26 and 2023.13
Docker Compose ecosystem.

The expectation is to automatically assign the behavior and color label depending on the scores levels.

Additionally, Clair seems to have issues with some blobs.

[rhessinger]

Hi @w-repinski_1472,

Thanks for all the information. It looks like this can be broken down into a couple of issues.

1. The auto-assessment is not currently working with Docker images. This looks like it may be due to a bug in the [Vulnerabilities_UpdateExternalVulnerabilities] stored procedure that is causing the auto-assessment from being saved, but if not, it may be an issue with how the vulnerability score is calculated for Clair vulnerabilities. I created a ticket, PG-2443, to track this fix and this should be addressed in the next maintenance release of ProGet.
2. The Vulnerabilities Overview page is not properly showing related Docker image layers. I have created a ticket, PG-2444, to fix this issue. This will have to wait until the next maintenance release.

As for the Clair could not process layer... messages. These are expected. When Clair scans the layers, it looks at all layers including the metadata layers. Any metadata layer or other data-based layer will show this message. These are safe to ignore as long as you see request was valid at the end of the layer process.

When you manually assess the vulnerability, do the assessments show?

Thanks,
Rich

[w.repinski_1472]

1. I think that the problem concerns discrepancies in the assessment, some solutions are assessed in points, and then it is defined in words Low/Medium/Hight/Critical etc.
   I noticed that when scanning containers you get verbal information (Severity) and therefore it is not properly mapped.
   In my DevSecOps experience, score presentation is not the most important thing, labels are important.

2. Can you expand the Overview to display the Assessment tag information in a new column. Maybe it's worth asking an external CVE database for the actual Score for the docker position? Or maybe within the existing ProGet solution to maintain such a mapping of the CVE number to Score?

3. Thanks for the clarification from Clair. It's logical but hard to figure out.
   However, I'm not entirely sure if it works as you describe.
   I will check and come back with a separate post. It seems to me that we also have the same entry for some resources that are a set of files / e.g. an Alpine image.

4. Handmade assessments are visible but....
   Additional problem, manually set assessment.
   As you can see in the image, I have duplicated vulnerabilities for the same image and event, the same CVE ID is listed once as a marked assessment, and the second time as an unmarked assessment.
   This seems to be a mistake from the point of business logic. If I mark any specific CVE in the assessment as, for example: Critical, it should appear as critical everywhere. Possibly a checkbox to check.

[rhessinger]

Hi @w-repinski_1472,

Thanks for the additional input. To expand this a bit more:

1. Our auto-assessment uses the CVSS score from the vulnerability to determine which assessment to use automatically. The assessment type is then displayed as the label. It looks like there is a bug in both the CVSS score returned from the Clair extension as well as the stored procedure that saves that information back to ProGet. If you look at the execution log from the original ticket you submitted to us, at the end you can see an error occurred in the merge statement. Those two issues together are causing this.

2. We will look into what is returned from Clair to get this score set properly. When it comes to the vulnerability overview page, we only show the latest unassessed vulnerabilities. If you click the "view all" link at the top of that table, that table will show the assessment information and the score.

3. We have not seen Clair return this message for a layer that is not a manifest/configuration layer (sorry, I call these metadata layers which can be confusing). If there is an issue with how the layer is parsed, you will need to submit an issue to Clair for this. ProGet is only showing the information that Clair has returned to ProGet for the parsing. This is how our Clair implementation works:

   1. Send a list of image layers to Clair
   2. Clair then will pull each layer from ProGet and scan it.
   3. ProGet will then pull the result for each layer from Clair
   4. ProGet will then auto-assess and save the vulnerability back to ProGet.

4. I think this is also related to the stored procedure I was referring to earlier.

We hope to have these issues resolved soon. I hope this clarifies this for you. Please let me know if I missed anything or if you have more questions.

Thanks,
Rich

[w.repinski_1472]

Hello,

In terms of the Clair error, the indication is "start using Clair v4", and this one is not supported by your software ... Maybe it would be possible to install an optional separate from v2 plugin for Clairv4? Have they already achieved the appropriate stability of the program itself as well as the API.

I will wait for corrections on other topics...

[stevedennis]

Hi @w-repinski_1472,

Unfortunately integrating with Clair v4 cannot be done with only a new plug-in / extension. It requires substantial changes to the vulnerability feature/module in ProGet, so it's something we would have to consider in a major version like ProGet 2024.

Thanks,
Steve

[w.repinski_1472]

I upgraded to version 2023.14 (build 17).

1. Automatic vulnerability assignment (score/severity) still doesn't work.

2. Still no information in the Score column.
   
   Wouldn't it be better if this column was named Score/Severity?

3. After some time of operation (several manual scans, changes in the repository, new packages) I get this error.

[rhessinger]

Hi @w-repinski_1472,

Can you please ensure that your Clair extension is updated to 2.0.1 as well? Some of the fixes required changes to the Clair extension directly. The two issues that were fixed in the extensions:

1. A separate API call is needed to get the CVSS score
2. Duplicate vulnerabilities were being returned by Clair

Looking at your screenshots, it seems those issues will be fixed with an extension update.

Thanks,
Rich

[w.repinski_1472]

@rhessinger said in Problem with Vulnerabilities in docker with Clair:

Can you please ensure that your Clair extension is updated to 2.0.1 as well? Some of the fixes required changes to the Clair extension directly. The two issues that were fixed in the extensions:

A separate API call is needed to get the CVSS score
Duplicate vulnerabilities were being returned by Clair

Looking at your screenshots, it seems those issues will be fixed with an extension update.
Thanks,
Rich

Thanks for the info, I'm checking.
How can I make the information about a new extension update more visible?

[rhessinger]

Hi @w-repinski_1472,

You can see extension updates by navigating to Administration > Extensions and then an information block at the top of the page will display that there are extension updates. For the actual changes that were made in the 2.0.1 version of the Clair extension, you can view the 2.0.1 milestone in GitHub for our Clair extension.

Thanks,
Rich

[w.repinski_1472]

@rhessinger said in Problem with Vulnerabilities in docker with Clair:

You can see extension updates by navigating to Administration > Extensions and then an information block at the top of the page will display that there are extension updates. For the actual changes that were made in the 2.0.1 version of the Clair extension, you can view the 2.0.1 milestone in GitHub for our Clair extension

And how will I know that I have to go to the extension menu to update the extension?
Can ProGet send an email to the "administrator" or display a notification on the main screen like it does when updating ProGet itself?

[rhessinger]

Hi @w-repinski_1472,

We currently do not have any mechanism for alerting the user when an extension update is available. Our guidance around this is to check extensions for updates after a product is upgraded or when instructed by support. In this case, it was my fault for not alerting you to upgrade the extension to fix this issue. I'm sorry for that and I will make sure this does not happen again.

Also, many extensions are included in the install package. These extensions are updated automatically when the product is upgraded. So this won't be a problem with most extensions, it just so happens that the clair extension is not an included one.

Please let me know if you have any other questions for us.

Thanks,
Rich