Forums

rhessinger · 3 Jan 2022, 14:46

Ah, thanks for the extra details here. I was finally able to recreate this issue. I'm tracking the fix in BM-3770 which is expected to release in the next version of BuildMaster, which is due out this week.

Thanks,
Rich

rhessinger · 28 Dec 2021, 21:09

Hi @cronventis,

I set up an example in our test environment here and everything is working as expected. I did verify that the Image_Id that is in the database should match the Image Digest in the Image Data table as well.

Is this image a local image or a cached image? Currently, usage will only show for a local image.

If it is a local image, I think the next steps will be to run a few different SQL queries to identify why the usage is not matching to the image.

Thanks,
Rich

rhessinger · 22 Dec 2021, 21:00

Hi @cronventis,

Thank you for checking that for me. I dug into this further and it looks like we only actually use the Image_Id to find the image within a feed. We expect Kubernetes to return the Config Digest and that is what is stored in the Image_Id column. I'm going to work on creating this scenario and verify what is being pulled from Kubernetes. With the holidays coming up, I will not be able to look into this until next week. You should hear back from us by Tuesday or Wednesday of next week.

Thanks,
Rich

rhessinger · 22 Dec 2021, 14:35

Hi @marc-ledent_9164,

So far, I'm unable to recreate this issue. Do you have a reverse-proxy (like nginx, apache, traefik, etc...) in front of it? If so, can you try to bypass that and see if the issue still happens?

Thanks,
Rich

rhessinger · 22 Dec 2021, 14:31

Hi @cronventis,

Thanks for checking that for us. That does mean the Container Scanner is working. Did you happen to configure the container usage on your Docker Registry in ProGet? You can check this by navigating to Docker Registry -> Manage Feed -> Detection & Blocking and then you should see your container scanner under Container Usage Scanning.

If you have configured your feed to use that, the next step would be to look at what the Kubernetes scanner identified for your images and how ProGet has these images stored. If you look at your ContainerUsage table, you will need to look at two columns; Container_Name and Image_Id. The Container_Name is your repository's name and the Image_Id is the Digest for your image. Can you confirm that you see the container names you are expecting in that table?

Thanks,
Rich

rhessinger · 21 Dec 2021, 13:36

Hi @marc-ledent_9164,

Thanks! I have received it. I'm going to look it over and test a few more cases and get back to you shortly.

Thanks,
Rich

rhessinger · 20 Dec 2021, 17:26

Hi @marc-ledent_9164,

Would you be able to send me over the contents of your Configuration table on your BuildMaster database? I'm having trouble recreating this and I'm guessing it has to do with one or more of those settings. You can send it to support@inedo.com and include [QA-723] in the subject. I will keep an eye out for it that way and review it.

Thanks,
Rich

rhessinger · 17 Dec 2021, 19:32

Hi @marc-ledent_9164,

Just to confirm, on your test site, is it a brand new BuildMaster database also?

Thanks,
Rich

rhessinger · 16 Dec 2021, 17:35

Hi @cronventis,

The fix will only prevent new duplicates from being created. Mainly this is because I cannot ensure that the first vulnerability is always the properly assessed vulnerability. For now, the best option will be to run a SQL query directly against SQL Server ProGet database after you upgrade to 6.0.5.

I have created a SQL query that will delete all the duplicates excluding the first vulnerability that was added to ProGet. If that criteria works for you, this query should be good enough.

BEGIN TRANSACTION

DELETE FROM [Vulnerabilities]
WHERE [Vulnerability_Id] in (
    SELECT v.[Vulnerability_Id]
    FROM [Vulnerabilities] v
        INNER JOIN (
            SELECT [External_Id]
                ,[FeedType_Name]
                ,[VulnerabilitySource_Id]
                ,COUNT([External_Id]) as [NumberOfDuplicates]
                ,MIN([Vulnerability_Id]) as [FirstVulnerability]
                ,MAX([Vulnerability_Id]) as [LastVulnerability]
            FROM [Vulnerabilities_Extended]
            GROUP BY External_Id, FeedType_Name, VulnerabilitySource_Id
            HAVING count(External_Id) > 1
        ) duplicates on v.External_Id = duplicates.External_Id
        WHERE v.Vulnerability_Id != duplicates.[FirstVulnerability]
)

ROLLBACK

Currently, I have the script set to rollback at the end (meaning it won't actually delete the duplicates). If this works for you, you can simply change ROLLBACK to COMMIT and rerun the query and it will remove the duplicates.

Please let me know if you have any questions!

Thanks,
Rich

rhessinger · 16 Dec 2021, 15:35

Hi @cronventis,

Thanks for sending this over. I have found the issue, PG-2064, and have fixed it. It will be released tomorrow in ProGet 6.0.5.

Thanks,
Rich

rhessinger · 16 Dec 2021, 12:48

Hi @cronventis,

Thanks for confirming thsi for me. I already have the fix in for ProGet 6.0.5 which is expected to release on Friday.

Thanks,
Rich

rhessinger · 15 Dec 2021, 15:25

Hi @philippe-camelio_3885,

In my experience you should only need to add ` in front of all the $ and that should work. Can you give this version a try?

# ACL LOG4J  https://www.haproxy.com/blog/december-2021-log4shell-mitigation/
   option http-buffer-request
   acl log4shell url,url_dec -m reg \`${[^}]*\`${
   acl log4shell url,url_dec -m reg \`${jndi:(?:ldaps?|iiop|dns|rmi)://
   acl log4shell url,url_dec -i -m reg \`${[\w`${}\-:]*j[\w`${}\-:]*n[\w`${}\-:]*d[\w`${}\-:]*i[\w`${}\-:]*:.*}
   acl log4shell req.hdrs -m reg \`${[^}]*\`${
   acl log4shell req.hdrs -m reg \`${jndi:(?:ldaps?|iiop|dns|rmi)://
   acl log4shell req.hdrs -i -m reg \`${[\w`${}\-:]*j[\w`${}\-:]*n[\w`${}\-:]*d[\w`${}\-:]*i[\w`${}\-:]*:.*}
   acl log4shell_form req.body,url_dec -m reg \`${[^}]*\`${
   acl log4shell_form req.body,url_dec -m reg \`${jndi:(?:ldaps?|iiop|dns|rmi)://
   acl log4shell_form req.body,url_dec -i -m reg \`${[\w`${}\-:]*j[\w`${}\-:]*n[\w`${}\-:]*d[\w`${}\-:]*i[\w`${}\-:]*:.*}
   http-request deny if log4shell
   http-request deny if { req.fhdr(content-type) -m str application/x-www-form-urlencoded } log4shell_form

Thanks,
Rich

rhessinger · 15 Dec 2021, 14:39

Hi @cronventis,

I was able to recreate this issue and we should be able to get this corrected in the next release of ProGet. Can you please confirm which version of ProGet you are using?

Thanks,
Rich

rhessinger · 15 Dec 2021, 13:39

Hi @cronventis,

Do you have multiple vulnerability sources configured in your ProGet instance? Are you able to provide screenshots of the actual vulnerabilities in ProGet?

Thanks,
Rich

rhessinger · 15 Dec 2021, 12:39

Hi @Stephen-Schaff,

How many tags does the build/ops-dotnet-6.0 have? Is 1.0.1 the only tag it has or is there another? Does the base-prerelease registry also have strict versioning enabled?

The error you are seeing with vulnerability scanning will not affect the setting of the tag on Docker images. Vulnerability scanning is a scheduled job that happens in the background and not automatically upon upload of a new image so that should not affect this image. I am curious about that error though. Do you have your Sonatype vulnerability source attempting to scan your Docker registry? If so, Sonatype does not support Docker registries. We only support using Clair for scanning vulnerabilities in Docker images.

Thanks,
Rich

rhessinger · 15 Dec 2021, 03:09

Hi @furqan-ahmad_4378,

So 5.3.27 definitely has the bug. The bug affected Docker Registries not using shared blob storage. Beginning in ProGet 5.3, all new Docker Registries will use common blob by default, but any existing registries that were upgraded do not. This caused the vulnerability scanner to not find the layers during the scan when looking at images in a registry not using common blob storage.

I'm currently testing with the quay.io/coreos/clair:v2.1.7 image. We have not implemented Clair's newest API yet because it does not seem to be stable as of yet. They are still making too many changes too frequently.

Hope this helps!

Thanks,
Rich

rhessinger · 14 Dec 2021, 21:15

Hi @furqan-ahmad_4378,

ProGet 5.3.39 and later include the fixes for these issues. We also improved the performance of the list repositories page in ProGet 6.0.3+. What version of ProGet are you currently running? Also, is this happening for every layer?

Thanks,
Rich

Posts made by rhessinger