Posts made by Dan_Woolf

Hi @MellowOak,

A quick background on how ProGet handles malicious packages. Malicious packages are treated as vulnerabilities in ProGet. That means that a malicious package will show up as an unassessed vulnerability (since they rarely have a CVSS score) and can be assessed, analyzed, and blocked like any other package with a vulnerability. With that said, most of the time, this blocking is not needed because as soon as they are identified as a malicious package, the public feed will have already removed the package. The only time they are really caught in ProGet is when they have already been downloaded and cached. In ProGet 2026, we are working on a better way to store and distinguish malicious packages separate from vulnerabilities.

To answer your questions:

1. All paid editions support the ability to block malicious packages via policies. Please refer to our License Restrictions documentation for edition limitations.
2. The best configuration for blocking malicious packages is to block all unassessed vulnerabilities. This requires an administrator to review unassessed vulnerabilities regularly on the Reporting & SCA -> Vulnerabilities tab.
3. Yes a block can be overriden by:
   1. Add an exception in the policy for that package and version
   2. A ProGet admin can set the package status to Always Allow
   3. It could be manually downloaded from the ProGet UI
4. Inedo's aggregator runs mutliple times a day to pull from all the vulnerability and malicious package sources and creates a custom compresssed database file. As long as ProGet has access to cdn.inedo.com, ProGet will then download that file nightly update it's database. It can be updated under Scheduled Jobs -> Vulnerability Database Updater.
5. There are multiple ways to gain visibility, but the easiest is to use ProGet's notifier feature to be alerted when a vulnerability to a pacakge.

Hope that helps to answer your questions! Please let us know if you have any other questions.

Thanks,
Dan

Hi @misael-esperanzate_7550,

What version of ProGet do you have installed? This may be an issue that we have already fixed. I remember this happening in some older versions of ProGet.

Thanks,
Dan

Hi @michal-roszak_0767,

That icon indicates it's a remote package. That means it can be remote or cached. If you use the Pull to ProGet feature, then it will pull all the files locally and converts them to an internal/local package and those icons go away. If you combine that icon with the error "File not found on disk", this means the artifact was cached, but something deleted the file from disk. That is what Dean was trying to explain.

@misael-esperanzate_5668,

I think the next step from here is to navigate to Manage Feed -> Storage & Retention, and then run the re-index feed job (found to the right of storage) with the "Remove database entries for missing package files" option selected. That will remove any Maven artifacts from the database that are missing local files and allow them to be re-cached again. If you want to review the Maven artifacts that would be removed, run the re-index without selecting the remove database entries option. This will warn you of any Maven artifacts that a missing first, then you can run it again to remove them. Hope this helps!

Thanks,
Dan

Hi @lukas-christel_6718,

You mentioned that your version had an underscore in it, but the command you referenced does not have an underscore. Is this something that once you add a package with a bad version, then any package fails to install, any version of a package with the same group and name fails to install, or just the specific package with the bad version fails to install? Also, can you please send us the bad version that you uploaded to the feed? Once we have that information, we should be able to reproduce your issue and get a fix out for you.

Thanks,
Dan

Hi @si-peham_2672,

As far as I can tell, it looks like there is still a permission issue with the working directory that is preventing file listing. The workaround to use the --input argument seems to get past that issue, so I would continue to use that. For your reference, I did test this locally and on a test server, and both worked without issue. If you are interested, here is a link to our code that is searching for the file (https://github.com/Inedo/pgutil/blob/5001174a4c786bb238e8546d18d82fed71cfcc58/Inedo.ProGet/Scan/DependencyScanner.cs#L114). The SourceFileSystem.Default.FindFilesAsync just calls new DirectoryInfo(folder).EnumerateFiles and then matches against the name.

For the Specified argument was out of the range of valid values error, that will be resolved in ProGet 2024.33, which will be released this Friday.

If you have any other questions, please let us know.

Thanks,
Dan

Hi @si-peham_2672,

I think this is related to a recent Audit API bug, PG-2941. We will be releasing that fix in ProGet 2024.33 due out on Friday. Can you please try running that with the --noaudit parameter? That will bypass the audit portion in the scan. That will at least scan the "requirements.txt" file and create the build, then the audit will then be performed later by ProGet.

Thanks,
Dan

Hi @c4buildmasters_2588,

Thanks for the PR! We are currently reviewing your PR and we will be in touch.

Thanks,
Dan

Hi @guilherme-silva_2532,

The warnings in the blob layer scanner are to be expected in most images. Not every layer includes a file system. The layer scanner scans each layer for an application database. This will vary from layer to layer.

There was also a bug released in ProGet 2024.32 that was not recording certain layer scanner results properly. This will be fixed in PG-2943 and will be released in the maintenance release this Friday. This may be why you are not seeing any packages found in this container.

Thanks,
Dan

Hi @si-peham_2672,

When you say owner, you mean you have full control on the folder correct? When the --input parameter is omitted, it will attempt to scan the directory for the requirments.txt file. I'm guessing that is causing the issue. A workaround you can try is to specify the path for the requirements.txt file for the --input parameter and see if that works.

Thanks,
Dan

Hi @itadmin_9894,

Would you be able to share the entire JSON response and exactly what information is missing that you are expecting to see?

Thanks,
Dan

Hi @rhowell_8827,

Thanks for reaching out. I took a look at the promotion code for Debian-based package promotion and it looks like this is due to a recent regression in our promotion process. I have created a new ticket, PG-2942, to track the fix. This is expected to be released next week in ProGet 2024.33.

Thanks,
Dan

Hi @v-makkenze_6348,

I apologize for the delay on our side. After testing these two packages (Microsoft.Web.Infrastructure.1.0.0 and System.Net.Http.Formatting.Extension.5.2.3), I'm unable to recreate your error. They download fine for my tests. Can you please try to delete these two cached packages and try it again?

Thanks,
Dan

Hi @caterina,

I'm sorry about this. There looks like there was a conflict with another notifier change. I have created a new ticket, PG-2933, to fix this and it will be released in the next maintenance release.

Thanks,
Dan

Hi @layfield_8963,

NFS storage is supported and used by many users. I'm surprised changing that would fix it, especially since the CLI worked and the backend storage would not be different between the two. I will do some testing in our labs on this as well.

Thanks,
Dan

Hi @layfield_8963,

Based on your console output, it looks like there may be something manipulating the response back. I performed a few quick checks on ProGet 2024.26 and I was able to upload to an Asset Directory without issue. The CLI and the UI use a slightly different mechanism for uploading files. The biggest difference is that the UI uses a specific multi-part upload API specific to our upload control. Do you have a proxy or load balancer sitting between your computer and ProGet?

Thanks,
Dan

Hi @caterina,

We're a bit concerned that this is too specific for your usecase, and we're really struggling documenting it.

Instead of doing this, we're thinking it might just make more sense to run (Get-Item 'C:\path\to\file.exe').VersionInfo using PowerShell and export these into variables instead.

For Example:

$versionInfo = (Get-Item 'C:\path\to\file.exe').VersionInfo
$version = if($versionInfo.FileVersion) { $versionInfo.FileVersion} else { $versionInfo.ProductVersion }
$productName = $versionInfo.ProductName
pgutil builds scan --input=myApplication.csproj --project-name$productName" --version=$version

Thanks,
Dan

Hi Mike,

The Asset API does require the DownloadPackage permission. Currently, only the ProGet UI can be used with the ViewFeed only permission. I'll add this as a note to discuss in the next products meeting early next week.

Thanks,
Dan

Hi @sgardj_2482,

You can create a custom task under Administration -> Manage Security -> Tasks / Permissions then click Customize Tasks in the upper right corner. When you do that, you can create custom task that has the View Feed permission. That will allow a user to view an asset's metadata in the ProGet UI and not be able to download the package.

Thanks,
Dan

Hi @caterina,

This looks to be an issue with pgutil. I just pushed an update to pgutil, 2.0.3, that includes the fix. If you update to that version, you should be able to promote builds now.

Thanks,
Dan

Hi @forbzie22_0253,

The easiest solution is to change both the "ProGet Service" (INEDOPROGETSVC) service and the "World Wide Web Publishing Service" (W3svc) to Automatic (Delayed Start). This typically allows SQL Server to start before IIS and the ProGet Service start.

Thanks,
Dan

Hi @parthu-reddy,

This is usually an indication of an outside process accessing this file. To verify this, if you restart ProGet, access to that file will be restored. If it is still locked, then something external to ProGet is accessing it. The most likely culprit is an antivirus scanning that file when ProGet attempts to access it.

Can you restart ProGet and see if there is still a file-locking issue? If the file is still locked, can you verify that you do not have an antivirus solution that is constantly scanning that file and locking it?

Thanks,
Dan

Hi @sonali-prasad17_0018,

Are you able to see the packages from the mirror in your Debian feed within ProGet? Also, do you get any errors when trying to update your source in Ubuntu?

Thanks,
Dan

Hi @maciej-luszczynski_5271,

Thanks for bringing this to our attention. I have corrected the example as suggested.

Thanks,
Dan

Hi @parthu-reddy,

Can you tell me what version of ProGet you are using? It looks like this issue was fixed recently, PG-2688, in ProGet 2024.5. I believe if you upgrade to 2024.5, that should fix your issue.

Thanks,
Dan

Hi @forbzie22_0253,

ProGet will connect to https://my.inedo.com/services/v1/activate for activation.

Thanks,
Dan

Hi @lucas-almeida_8120,

This a known issue that we are currently working through. It seems to be an issue with some bad/duplicate data in your database. The current workaround is to downgrade to ProGet 2023. We are hoping to have a fix ready soon.

Thanks,
Dan

Hi @forbzie22_0253,

You can do this using our Bulk Import via a drop path feature. We have a helpful How To Bulk Import Packages Into A Feed From Disk article in our documentation.

Please let us know if you have any other questions!

Thanks,
Dan

Hi @itops_6398,

I'm very happy to hear that you got this working. You typically do not need the UPN because it will be automatically appended when searching for the user in your domain, but it doesn't hurt to be there. There are typically only two scenarios when the UPN is needed; when that username is already in the built-in directory or you have a multi-domain forest setup in Active Directory, then the UPN will tell it specifically where to look.

With all that said, if it is working for you now, then I would leave it as is because SAML and AD/LDAP are difficult enough as it is. We are going to add the transform information to our SAML docs in hopes of helping others later.

Thanks,
Dan

Hi @itops_6398,

Are you still using the defaults for the LDAP Overrides? Unfortunately, AD LDAP queries give very little log output. Right now you are seeing the amount of logging we receive from the AD server. The best way to see more details is to log into your Active Directory server and look in the Windows Event Log to see the errors found in the connection.

On a separate note, when it comes to configuring an Azure Enterprise Application SAML provider to link to your active directory. I have found that changing the nameidentifier claim to use a transformation of ExtractMailPrefix() on the attribute user.userprincipalname will extract the username and typically link well to Active Directory usernames.

Thanks,
Dan

Hi @carl-westman_8110,

Thanks for clarifying that for me. So that changes things a bit. When it comes to the LDAPS issue, the original suggestion for adding the certificate will not work. The LDAPS certificate needs to be added directly to the trusted certificates in the Container, not the app service. We don't have much advice on this as of yet, but let me talk internally on this and see what my colleagues say.

As for the actual SSL certificate in ProGet, you will need to follow the Configuring HTTPS without a Reverse Proxy guide in the Linux HTTPS Support. That will show you how to configure the SSL certificate within the container itself and not the app service. We also have more guidance on converting certificate files to different formats at the bottom of our HTTPS support on Windows documentation.

Thanks,
Dan

Hi @carl-westman_8110,

I didn't realize how you were running ProGet. Are you currently running the ProGet Docker Image using an Azure App Service?

Thanks,
Dan

Hi @carl-westman_8110,

The rejected certificate was from your Active Directory provider, not the Azure Web App itself. This is normally due to Active Directory generating its own certificates. I'm not exactly sure how to trust them in Azure, but it looks like you can register CA certificates on an Azure Web App using Microsoft's documentation: https://learn.microsoft.com/en-us/azure/app-service/configure-ssl-certificate-in-code

As an alternative, you can edit your User Directory in ProGet and on the Connection tab, you can change to "Use LDAPS and bypass the certificate errors". That will allow ProGet to bypass the certificate validation process.

Thanks,
Dan

Hi @k_2363,

We found an issue in our Native API related to our .NET 6 upgrade that was preventing certain values from converting properly. That has been resolved and will be released on Friday in ProGet 2023.18.

Thanks,
Dan

Hi @Jon,

Thanks for sending this over to us. It looks like there is an issue with the configuration API and it's causing it to always return a 404 error. I have added a ticket, OT-497, to fix this issue, and it should be released on Friday this week in Otter 2022.13.

Thanks,
Dan

Hi @caterina,

This has been an issue we see from time to time and it is related to NuGet's request caching. One thing to try first is to make sure you do not have IIS request caching enabled. To do this:

1. Navigate to your ProGet site in IIS
2. Double click on "Output caching"
3. On the right, click "Edit Feature Settings..."
4. Uncheck both "Enable cache" and "Enable Kernel Cache"
5. Click OK

The next thing to try is to clear NuGet's HTTP cache. You can do that by running the following NuGet command:

nuget locals http-cache -clear

Typically the combination of these will resolve the issue. Can you please try these next time you have the issue?

Thanks,
Dan

Hi @guyk,

After some research, it looks like scope-mapped tokens require a custom authentication procedure. It looks like if you use an Admin User, it will then use the standard Docker authentication procedure. Would you be able to try that and see if it works with ProGet?

Thanks,
Dan

Hi @guyk,

Thanks for confirming that for me. On that same email you sent me, can you please send me a repository name that is available to that token?

Thanks,
Dan

Hi @guyk,

The error I'm getting is a 401 unauthorized exception. Can you confirm that the token you sent me is a token password? Following Micorosft's documentation https://learn.microsoft.com/en-us/azure/container-registry/container-registry-repository-scoped-permissions, you need to create a token password for the token. Then the docker login username is the name of the token and the password is the token password.

Thanks,
Dan

Hi @guyk,

Thanks, I have it and I'm testing now. I'll reach out shortly and let you know what I find.

Thanks,
Dan

Hi @chrisblankde,

Can you please tell me what metadata you are trying to update? If it is just the version, you can use the Repackage API. If it is something other than the version, in ProGet 2023, we added the ability to edit the upack metadata directly in ProGet. You can do that by navigating to the package version in your universal package feed and then click "Edit Package" in the upper right corner.

If neither of those options works for you, then what you suggested (download/extract/edit/repack/upload) would be the way to handle this.

Thanks,
Dan

Hi @guyk,

You are correct, the proxy settings are global, not at a feed level.

Would you be able to provide us with a URL and a scope-mapped token we can debug with? You can email these to support@inedo.com with the subject of [QA-1148]. If so, we can easily debug this and see what is going on. Once we complete the fix, we can let you know and you can revoke access from us.

Thanks,
Dan

Hi @guyk,

Glad to hear that fixed your issue with the Docker hub connector. When it comes to "Attempt to search", this options does work on Microsoft's container registry, GitLab's container registry, and Google Cloud (gcr/k8s). It only actually affects searching for images though. When it is unchecked, it just requires exact image name searches.

The manifest error is definitely annoying, unfortunately, Docker itself filters out all returned errors and just displays that "not found: manifest unknown: manifest unknown" message. Along with that, when it comes to querying other Docker registries, they throw a lot of errors instead of just returning an empty dataset when an image is not found. That makes it hard to produce meaningful entries into our logs.

As for the Azure container registry. Are you using a public or private registry? One thing we have noticed is the Azure container registry can have multiple package types mixed into it. When you mix package types, ProGet has a hard time using that registry. If you can provide us with an example, we can quickly debug it and let you know what is wrong with the connection.

Thanks,
Dan

Hi @guyk,

Can you try using https://registry.hub.docker.com for your connector URL and see if you have the same issue?

Thanks,
Dan

Hi @guyk,

I have a few questions for you. Can you please share the connector URL you use for your Docker connector? Also, on one of the images that failed with this message, when you look at them in ProGet, do you see if the image is cached? If so, can you try to delete the cached image and pull it again?

Thanks,
Dan

Hi @v-makkenze_6348,

When you enable the preview feature for ProGet Vulnerability Central, ProGet will add a vulnerability source name PGVC automatically, but it will not show under the vulnerability sources. It looks like after you enabled that, you added a new ProGet Vulnerability Center (which will default the name to "ProGet Vulnerability Central"). So the ProGet Vulnerability Center source should be left off and can probably be removed. That is definitely what was causing a duplicate vulnerability.

Thanks,
Dan

Hi @hashim-abu-gellban_3562,

I think the issue here is that the link for the sample config in our docs is now pointing to the wrong sample configuration file. You will want to download the config sample from: https://raw.githubusercontent.com/quay/clair/v2.1.8/config.example.yaml

As Alana mentioned they are on Clair v4 now and it is a different product than version 2 and v4 has a completely different configuration file structure. If you use this older config example, you'll see in the first section has the database connection information. Use this configuration file base should fix your issue.

Thanks,
Dan

Hi @arozanski_1087,

We talked it over in our team meeting yesterday and decided to change ProGet to always create a new release when one doesn't exist and add new dependencies when the release does exist. This way that parameter will not be needed. So you will just need to upgrade ProGet to v2022.23 when it is released later today.

As for the Yarn support and scanning multiple package types in the initial scan, we plan to work on that early next week, so we should have something for you soon.

Thanks,
Dan

Hi @arozanski_1087,

We were able to get the fix in for the appending dependencies, PG-2294. This will be released tomorrow in ProGet 2022.23 and we will also be releasing a new pgscan, v1.4.2, that includes a new parameter, --append-dependencies, that will allow you to append dependencies from multiple scans.

Thanks,
Dan

Hi @arozanski_1087,

I'm sorry I missed this earlier, but it looks like there was a typo on the GitHub page for the identify documentation, it previously used --consumer-product-version. That parameter should have been --version. If you run the pgscan help identify command, you will see it is appropriately listed there. My colleague updated the documentation to include the proper parameter. That should get you passed the error above.

I also noticed another issue. If you run the command twice, the dependencies that were found on the second run will not be added to the project. I'm working on getting this fixed, but as a workaround, you can do the following:

1. Navigate to your Project & Release in ProGet
2. Navigate to the "Imported SBOMs" tab
3. Download the latest SBOM (please note that viewing the XML in the browser removes the XML namespace which will prevent future uploads)
4. Click the "Upload SBOM" button
5. Copy and paste the contents from your downloaded SBOM into the "SBOM file" field
6. Check the "Overwrite release" option.
7. Click Import

That will then add the missing packages to your release. To fix this, we will need to make a change to both pgscan and ProGet. I will do my bet to get this fix into tomorrow's release.

Thanks,
Dan

Hi @arozanski_1087,

Thanks for the additional information. I'll need to research this a little further because nothing is jumping out at me looking through the code. Please stay tuned!

Thanks,
Dan