Request for Creation of API for Package Auditing Before Dependency Restoration

fabrice.mejean

Hello,

I would like to submit a request for the creation of an API dedicated to auditing packages prior to the restoration of dependencies within our software factory. Currently, we utilize Native APIs to validate the dependencies used by an application before restoration. However, this approach has certain limitations that lead us to seek a more efficient solution.

Below are the key functionalities we would like to see implemented in this new API:

Verification of Package Promotion:

Currently, the existing API verifies if the package has been promoted, but it does not confirm whether the package is still present in the target. We would like a more thorough method that utilizes the package API to verify the presence of the package.

Status of Obsolete Packages:

There is currently only a "set package status" API, but no "get package status." We need functionality that allows us to retrieve the status of packages to identify those that are obsolete or blocked from downloading.

Vulnerability Checks:

We need to verify package vulnerabilities and indicate those that are blocked from downloading due to critical vulnerabilities, along with their severity ratings.
The goal of this API would be to perform a complete audit of dependencies before the build process, preventing issues during package restoration. This would enable developers to quickly diagnose and resolve installation problems, ultimately saving them a significant amount of time.

Additionally, I want to highlight that the current audit API requires the registration of a build to function, which is not feasible in our context, especially for builds that have not yet been initiated.

Thank you for considering this request. I am available for any further information or to discuss the details related to this proposal.

Best regards

Fabrice MÉJEAN

stevedennis

Hi @fabrice-mejean_5174 ,

Thanks for the suggestion; to help move this forward, we thought about this from a technical standpoint. We're a little hesitant to call the command pgutil packages audit since there's already an audit command for builds and vulnerabilities.

One idea is to deprecate pgutil vulnerabilities audit and call it pgutil packages audit instead.

In any case, this new command is a combination of pgutil builds audit and pgutil vulns audit.

The pgutil packages audit command would input a project, something like this:

pgutil packages audit --project=c:\projects\MyProject.csproj

Behind the scenes, pgutil would parse and POST a packageset:

POST /api/sca/audit-packages
[ 
    {
        "name": myPackage
        "version": 1.2.3
        "type": "nuget"
    },
    {
        "name": myPackage
        "version": 1.2.3
        "type": "nuget"
    }
]

The API would return an array I suppose (we don't have any samples for that, but it's a serialized BuildInfo.cs), and the end result would look like this:

$> pgutil packages audit --project=c:\projects\MyProject.csproj

Parsing MyProject.csproj... found X packages.

Azure.Core-1.35.0
 Compliance      : Compliant
 License         : MIT
 Vulnerabilities : None

Microsoft.Extensions.Configuration.EnvironmentVariables 8.0.0 
 Compliance      : Noncompliant
 License         : MIT, Apache-2.0
 Vulnerabilities : PG-123456 (High)
   the vulnerability title of this vulnerability goes here

The API could obviously contain more info. It's documented via serialized .cs classes here:
https://github.com/Inedo/pgutil/tree/thousand/Inedo.ProGet

Thanks,
Steve

fabrice.mejean

Hello @stevedennis ,

Thank you very much for taking the time to respond to my request. Your proposal aligns well with my needs, but I would like to point out that it lacks the retrieval of the "package status" (PackageStatus object).

Regarding the pgutil vulnerabilities audit command, I understand that there is a suggestion it might be redundant. However, I don't believe it would actually create a duplication, especially if it can execute more quickly and meet specific requirements.

Additionally, I’ve noticed that there isn’t a command to check if a package has been promoted, even though the API for it exists (https://docs.inedo.com/docs/proget/api/packages/promote/promotion-query). Would it be possible to add this functionality? Also, if we could verify that the package is still present in the destination repository, that would be a significant advantage.

Thanks again for your help!

Best regards,

Fabrice MEJEAN

stevedennis

Hi @fabrice-mejean,

Good point, it would not have PackageStatus. So let's jus say we add that in, like this:

Azure.Core-1.35.0 (Deprecated)
 Compliance      : Compliant
 License         : MIT
 Vulnerabilities : None

Obviously the API would return it in a structure manner, and it could also show downloads etc.

However, all this data (including package status) means we need a feed in context. So the command would have to look like:

pgutil packages audit --project=c:\projects\MyProject.csproj --feed myFeed

I'm not sure how I feel about that. Packages can be in multiple feeds, and often times users build from multiple feeds.

Anyway.... I have a new idea... what do you think about this:

pgutil packages metadata --feed=myNugetFeed --package=myNugetPackage --version=1.2.3

That could return the normal metadata, but also server-side metadata (package status, downloads, etc).

I guess it would require multiple API calls, though I doubt that's going to cause any more performance overhead than a "bulk" kind of API call that just "looped" over packages and looked them up in the same manner.

Let me know your thoughts.

Thanks,
Steve

fabrice.mejean

Hello @stevedennis ,

You raise a good point. Indeed, we should consider adding the 'feed' parameter, as the concept of obsolescence is tied to it. This begs the question of whether it should remain this way or if it should be managed more globally, similar to how vulnerabilities are handled. However, that would go beyond a simple API addition.

On the other hand, I am absolutely fine with making multiple calls to achieve my goals, as long as ProGet can handle the load.

Additionally, I would like to ask about the use of 'pgutil packages audit' and whether it indicates compliant or non-compliant. Does this status utilize all the rules (license, vulnerability, and others), and is there a plan to provide insights into the reasons behind the result? If it takes into account all the rules, then the 'other rules' are dependent on the feed, making the 'feed' parameter essential.

Thank you for your thoughts on this matter. Looking forward to your feedback!

Best regards,

Fabrice MEJEAN

stevedennis

Hi @fabrice-mejean,

This begs the question of whether [obsolescence] should remain this way or if it should be managed more globally, similar to how vulnerabilities are handled. However, that would go beyond a simple API addition.

I could see this making sense for OSS packages hosted at their official source (e.g. NuGet.org, npmjs, etc) -- but you're right, much more than an API change and we should limit scope for now :)

Additionally, I would like to ask about the use of 'pgutil packages audit' and whether it indicates compliant or non-compliant. Does this status utilize all the rules (license, vulnerability, and others), and is there a plan to provide insights into the reasons behind the result? If it takes into account all the rules, then the 'other rules' are dependent on the feed, making the 'feed' parameter essential.

Good point -- the feed parameter would be important if you wanted feed-scoped policies. Otherwise, only the global policy could apply... which is limiting.

As for "reasons behind the result", there is a short text available in the compliance system that is displayed in the UI and error messages for non-compliant results. So I think we could bring that into the pgutil output like this:

$> pgutil packages metadata --feed=myNugetFeed --package=Junk.Package --version=1.2.3

Junk.Packge-1.2.3 (Deprecated, Unlisted)
 Compliance      : Warn (Deprecated; Unknown License)
 License         : RSGPL
 Vulnerabilities : None

So to summarize my understanding... here's what I'm thinking.

The pgutil packages metadata command will provide package metadata (i.e. from the manifest file), server metadata (listed, unlistd), vulnerabilities, and compliance -- similar to what the ProGet UI shows
It will work like the other packages commands, which require a feed parameter
We will not do the pgutil packages audit command afterall, since we don't have an easy way to work on a "set of packages" with regards to feeds

And of course, the new API call would return the data in a structured manner. We can share that as we get closer to implementation.

How's this sound?

Thanks,
Steve

stevedennis

In retrospect, I don't think pgutil would display the output like that exactly, but you get the idea. It'd basically be a bunch of label/text pairs -- basically not that different from what's in the ProGet UI

pmsensi

@stevedennis

Re: Get package license with ProGetClient

the pgutils packages metadata + API end point to be used in Inedo.Proget lib is what I need :)

Do you have a timeline for the implementation ?

Best Regards,
Pedro

fabrice.mejean

Hi @stevedennis,

I agree with your proposal; it should allow me to meet my current needs.

Regarding obsolescence management, unfortunately, the majority of open-source components do not manage obsolescence, let alone end-of-life dates. That is why we mark all packages older than 3 years as obsolete. This allows us to verify that the open-source project is still alive, and that in case of a vulnerability, we are still able to carry out a corrective version upgrade. I will certainly create another post about this soon, but it gives you something to think about.

I would appreciate a YouTrack reference to track this feature.

Thank you again.

Fabrice MÉJEAN

atripp

Hi @fabrice-mejean @pmsensi ,

We've got this spec'd out and on the roadmap now as PG-3126! It'll come through a maintenance release, along with pgutil security commands for configuring users, groups, and tasks.

The target is 2025.13, which is planned for October 24. I don't know if we'll hit that target, but that's what we're aiming for.

Please check out the specs on PG-3126; I think it captures what you're looking for, which is basically an expanded metadata object that includes compliance data, detected licenses, and vulnerabilities.

Thanks,
Alana

pmsensi

@atripp Looks good!

For my clarification, the property DependencyGroups, will be a list of all the (main) dependencies of the package ?

Best Regards,
Pedro

atripp

Hi @pmsensi,

Correct -- it'll be whatever data is on the "Dependencies" tab in ProGet, which is basically whatever is in the manifest file (.nuspec, etc).

Thanks,
Alana

pmsensi

Hello @atripp

Sorry to open an old topic.

While checking the content of the result of GetPackageMetadataAsync (PackageMetadata), I don't see any reference to the Dependencies.

The public APi has changed ? Or there is a new way to get the dependencies information ?

Best Regards,
Pedro

atripp

Hi @pmsensi,

I don't believe that was ever in the PackageMetadata object; that information is stored in ecosystem-specific formats inside of the manifest file (e.g. .nuspec) in the package and it's not something we can easily generalize. The UI shows a "simplified" version but it's not always correct or suitable for metadata/api results.

You'd have to parse those metadata files to get that information.

hope that helps,

Alana

pmsensi

Hello @atripp ,

oh ok, I thought that will be available, reading your last reply from last year.

Anyway, that information is not even stored in database ?, so means, I need to use the Nuget api to get that information ?

Best regards,
Pedro