Hi @atripp,
I investigated a little bit further:
The NpmDependencyScanner does not treat the scope of a package as a 'Group'.
I added a little code for testing:
Afterwards the packages-list of my release looks like this:
But the url behind a scoped package is still not working:
Cheers,
Caterina
Hi @atripp,
thank you for the quick fix. We would have suggested just the same solution.
I have tried the version and scoped packages are now visible but I ended up with the following:
Cheers,
Caterina
Hi @atripp,
thanks for having a look at it.
Yes, we are using pgscan to create and upload our SBOMs.
Cheers,
Caterina
Hi @atripp,
I just created an angular test project with default dependencies and created the sbom.
I sent the sbom file via email to support@inedo.com.
If I upload this file to our ProGet server (using our npm-proxy-feed) I can see all packages without a scope, and the project is listed under usages. Packages with a scope are missing.
Cheers,
Caterina
Hello,
I am using pgscan to create and upload sbom files for our releases.
I noticed that our sbom files contain scoped npm packages (e.g. @angular/common, @babel/core) which is correct because we are using them.
But if I have a look at the packages of a specific release using scoped npm packages, those are not listed (Reporting & SCA -> Releases -> Specific Release -> Packages).
Vice versa, my release is not listed under "Usage & Statistic" of a used scoped npm package.
Has this behavior already been noticed?
Cheers,
Caterina
Hi Rich,
please take your time.
We used to have two different pgscan calls for nuget and npm and we ended up with two files on ProGet:
But did I get it right that if I export the sbom those two files are being merged into one? In this case we would have to think about separating those pgscan calls again.
Thank you
Caterina
Hi Rich,
to get back to my "initial problem".
If I would use pgscan with auto type I would run into the same problem. Because the dev dependencies within the package-lock.json would be ommited but the node_modules directory contains also dev dependencies and their package-lock.json files would be read as well leading to my initial problem (having dev dependencies in the sbom file). I think we are not able to distinguish between dev dependency and "real" dependency within the node_modules folder.
Of course I could explicitly specify only to scan the package-lock.json file with the npm type but I would have to make a second pgscan call for nuget packages and would end up with two sbom files. It is a lot more comfortable to have all dependencies in one sbom file.
Further, pgscan with auto type and pgscan with npm type would by default list different npm dependencies.
Or did I understand something wrong?
Thanks
Caterina
Hi Rich,
we talked about this options as well and we think that maybe a switch to exclude dev dependencies could be helpful.
If pgscan takes the argument --exclude-dev (e.g.) node_modules folders are ignored and only dependencies with "dev: false" in the package-lock.json file are written into the sbom file.
Otherwise all dependencies in package-lock.json are listed and node_modules folders are included.
I can't imagine a scenario where I want "package-lock-only" without "omit:dev". Because dev dependencies would be listed in the sbom but the node_modules would not be scanned which would lead to an incomplete output I guess.
Let me know how you think about it and what the thoughts of your team are.
Thanks
Caterina
Hi Rich,
I had to dive deeper into this topic to have a better understandig of it.
So basically what we are doing is calling "npm ci" followed by "ng build --configuration production".
The package-lock.json file contains all dependencies, production and dev dependencies. "npm ci" installs them all which means that the node_modules folder contains dev dependencies as well. But "ng build --configuration production" creates our production output which has no dev references.
I tried to call "npm ci --omit=dev". In this case only production dependencies are installed and part of node_modules. But unfortunately, "@angular/cli" is a dev dependency which is needed to call "ng build".
Therefore, I would say that the node_modules scan should still be removed since this folder could contain dev dependencies which are not part of the final product.
Further, I guess we should add a filter for the dev dependencies while parsing the package-lock.json in pgscan. Right now dev dependencies are part of the generated sbom file. But packages like "@angular/cli" for example are never being shipped with our product.
Hope it gets clear what I am trying to say.
Thanks
Caterina
Hi Rich,
thank you, I was looking for this conversation!
So our current problem is that ProGet determines a critical vulnerability in one of our projects:
Our affected project team contacted me and told me that the project is not referencing json-schema in any version. And this dependency is also not listed in the package-lock.json of the project.
But I noticed that our project references "minipass-sized" and "npm-normalize-package-bin" as developer dependencies. Both of them have "json-schema" as dependency in their package-lock.json files.
And I guess I just fixed my own problem here... DevDependencies should not be part of my production output and thus not part of my node_modules folder. I guess I will have to take a look at the buildprocess of the product again.
I will keep you updated.
Thanks
Caterina
Hi,
I noticed that the list of npm dependencies differs depending on which type is given.
If the input is a package-lock.json file with type "npm" only the dependencies of this file are being processed.
If the input is a .sln with no type, pgscan scans for nuget and npm dependencies. But for npm dependencies all package-lock.json files are being processed, also the ones under "node_modules". This results in different npm dependencies.
I don't think that package-lock.json files of node_modules should be processed since all necessary dependencies are part of my projects package-lock.json file.
Further, I would like to call pgscan on a .sln because all project dependencies (nuget and npm) are listed in one sbom file.
Anyone else have issues with this procedure? Or is there a valid reason why package-lock.json files of node_modules should be processed as well? I just think both pgscan calls should result in the same npm dependencies.
Thank you,
Caterina
Hi @Dan_Woolf,
so it seems to be a long known issue: https://github.com/NuGet/Home/issues/3116
Thank you for your help.
Caterina
Hi,
we noticed a delay between uploading a package and being able to do a nuget restore on a consumer of this package.
More specific:
We uploaded a new package "DAP.Common 7.0.13" to ProGet. Afterwards, we updated the dependency to this new version manually in a consuming project. We tried to call nuget restore but it failed with the following error: "Unable to find package DAP.Common with version (>= 7.0.13)". There were about 15 minutes between package upload and nuget restore. A few minutes later we tried it again and the package was found. But the package could be seen on ProGet right after the uplaod.
We were able to observe this behavior with several packages. If the dependency got updated manually in a consuming project, nuget restore failed for a certain time period.
Unfortunately, we can not say if the problem is still present if the nuget package manager in Visual Studio would have been used to update the package dependency.
Have you already seen such a behavior?
Hi @atripp
I think packages has to be iterated instead. I haven't seen dependencies beneath packages.
Further, the "empty" Key has to be ignored as it stands for the root project:
Maybe a little bit parsing would be necessary.
In lockfileVersion 2 a dependency was listed like this:
In lockfileVersion 3 it looks like this:
If desired, we can also upload package-lock.json files for testing via MyInedo.
We noticed that pgscan does not list any dependencies for one of our npm-projects.
After debugging into it and comparing it with other npm-projects we noticed that there is a difference in the lockfileVersion of the package-lock.json files. The "problem-project" has lockfileVersion 3 while the others have lockfileVersion 2.
pgscan tries to read the dependencies from the property "dependencies" which is a legacy-property from lockfileVersion 1. lockfileVerson 2 was downward compatible, but lockfileVersion 3 (used by npm v9) is not. The newest package-lock.json no longer has the property "dependencies" and all dependencies are part of the "packages"-property.
Here is the official documentation about it: https://docs.npmjs.com/cli/v9/configuring-npm/package-lock-json/#lockfileversion
Have you already noticed this breaking change in the package-lock.json files?
I already opened an issue for this topic on ghithub:
https://github.com/Inedo/pgscan/issues/33
Hi,
we have the assessment type "Manually Unblocked" which we have assigned to some vulnerabilities a while ago. Now we noticed that only 5-6 vulnerabilities are having this assessment even though we assigned it to more than that. How is this possible? ( ProGet 2022.29)
Further, after upgrading to ProGet 2023 on a test server (which is an exact copy of our live system) 0 vulnerabilities had this assessment. Somehow this information got lost. Is this already a known problem? Or is this behavior intentional?
Thanks
Looking at the Usage&Statistic of a package, we can see the Latest Version of a consumer.
We noticed that the Latest Version is not the highest version of a consumer, but the latest version uploaded.
E.g.: We uploaded a project in version 2.0.0 and 2.0.0 is shown as Latest Version in its dependencies. Afterwards we had to upload a fix from a branch in version 1.6.1. Now 1.6.1 is shown as Latest Versions in the same dependencies.
One could assume that only versions below 1.6.1 are affected if a vulnerability is found in the package.
Could anybody else observe this behavior? Is it supposed to work this way?