Scoped npm packages not listed in releases

caterina

Hello,

I am using pgscan to create and upload sbom files for our releases.
I noticed that our sbom files contain scoped npm packages (e.g. @angular/common, @babel/core) which is correct because we are using them.
But if I have a look at the packages of a specific release using scoped npm packages, those are not listed (Reporting & SCA -> Releases -> Specific Release -> Packages).
Vice versa, my release is not listed under "Usage & Statistic" of a used scoped npm package.

Has this behavior already been noticed?

Cheers,
Caterina

atripp

Hi @caterina ,

Can you share an SBOM that I can use to reproduce this? Doesn't need to be the whole one, just one that contains an example of a scoped package that you're not seeing.

Cheers,
Alana

caterina

Hi @atripp,

I just created an angular test project with default dependencies and created the sbom.
I sent the sbom file via email to support@inedo.com.

If I upload this file to our ProGet server (using our npm-proxy-feed) I can see all packages without a scope, and the project is listed under usages. Packages with a scope are missing.

Cheers,
Caterina

atripp

Thanks @caterina , confirming receipt! We will review this in the next week or so, please stay tuned.

atripp

Hi @caterina ,

Thanks for sending this; looking over the SBOM, it looks like the scope is incorrectly encoded in the package.

Correct: <purl>pkg:npm/%40ampproject/remapping@2.2.1</purl>
Incorrect: <purl>pkg:npm/@ampproject/remapping@2.2.1</purl>

Are you using pgscan to generate this? I think so, and in that case it seems to be a bug. We may wish to have ProGet also accept it, but I'm not sure.

FYI, here is a minimal SBOM that demonstrates the issue:

<?xml version="1.0" encoding="utf-8"?>
<bom serialNumber="urn:uuid:6109c18a7f7c403b9f9f11ff73c8fe34" version="1" xmlns="http://cyclonedx.org/schema/bom/1.2">
  <metadata>
    <timestamp>2024-01-17T08:23:58.4621303Z</timestamp>
    <component type="application">
      <name>Test</name>
      <version>1.0.0</version>
    </component>
  </metadata>
  <components>
    <component type="library">
      <name>@ampproject/remapping</name>
      <version>2.2.1</version>
      <purl>pkg:npm/@ampproject/remapping@2.2.1</purl>
    </component>
  </components>
</bom>

When npm/@ is replaced with npm/%40, the file is imported as expected.

Cheers,
Alana

caterina

Hi @atripp,

thanks for having a look at it.
Yes, we are using pgscan to create and upload our SBOMs.

Cheers,
Caterina

atripp

Hi @caterina ,

We just published pgscan 1.5.10, and I believe that it will solve this issue. Can you give it a shot?

Thanks,
Alana

caterina

Hi @atripp,

thank you for the quick fix. We would have suggested just the same solution.

I have tried the version and scoped packages are now visible but I ended up with the following:

Cheers,
Caterina

caterina

Hi @atripp,

I investigated a little bit further:
The NpmDependencyScanner does not treat the scope of a package as a 'Group'.
I added a little code for testing:

Afterwards the packages-list of my release looks like this:

But the url behind a scoped package is still not working:

Cheers,
Caterina

atripp

Hi @caterina ,

Sadly I wasn't able to fix w/ that last release (I didn't have a test case to try, so kind of a blind fix), and was going to give it proper shot this weekend,.... but it looks like you guys submitted a pull request? Did that releaae solve the issue?
https://github.com/Inedo/pgscan/pull/46

Thanks so much,

Alana

caterina

Hi @atripp,

our pull request contains a fix for the assignment of the scope to the group property of the dependency scanner.
But as already mentioned in my previous post this only fixes the appearance of the packages in the overview. The URL behind the package name is still broken.
Maybe this needs to be fixed on the ProGet-side?

Cheers,
Caterina

rhessinger

Hi @caterina,

I think I see what the issue is here. When it comes to the package purl for npm packages, the scope needs to be URI encoded. When it goes to parse the purl for scoped packages, it reads the @ in the scope as the character indicating a version and starts then fails to parse it as an invalid URI. I'll get a fix in pgscan to handle this shortly and reply back when I have an updated version.

Thanks,
Rich

sebastian...

@rhessinger @atripp
I don't think the remaining problem is a pgscan issue. The latest version of pgscan (v1.5.12 including commits eb61208, bc663c4 and 28e60ad) should handle the @ within scope names correctly. Here is a little sample from a generated SBOM file:

<component type="library">
  <group>@angular</group>
  <name>common</name>
  <version>17.0.9</version>
  <purl>pkg:npm/%40angular/common@17.0.9</purl>
</component>

<component type="library">
  <group>@angular</group>
  <name>compiler</name>
  <version>17.0.9</version>
  <purl>pkg:npm/%40angular/compiler@17.0.9</purl>
</component>

<component type="library">
  <group>@angular</group>
  <name>core</name>
  <version>17.0.9</version>
  <purl>pkg:npm/%40angular/core@17.0.9</purl>
</component>

The problem seems to be on the ProGet side. The %40 is correctly displayed as @ in the UI, and packages seem to be identified correctly as well.

The only remaining problem is that the /packages/from-purl API endpoint seems to have problems with two @ characters. Btw, both of them are apparently encoded as %40 within the URL. Here is an example URL: http://proget-host/packages/from-purl?pUrl=pkg%3Anpm%2F%40angular%2Fcore%4017.0.9. The endpoint seems to convert both %40parts back to the @ character, which results in the error message "pkg:npm/@angular/core@17.0.9 is an invalid purl".

Without knowing too much about the code within ProGet it's hard to speculate how to solve this, but my guess is that either the endpoint needs to be able to handle two @ characters within a purl, or the purl itself needs to be encoded differently when passed as a parameter with another URL. When I change the example URL of my previous example in a way that the % of the first %40 encoding is encoded as well (i.e. replacing %with %25, I get the following URL, which actually works: http://proget-host/packages/from-purl?pUrl=pkg%3Anpm%2F%2540angular%2Fcore%4017.0.9 (%40angular has been replaced with %2540angular).

Cheers,
Sebastian

rhessinger

Hi @sebastian & @caterina,

I'm sorry, I realized that after I sent the last response. I have already fixed it as part of ticket PG-2563 in ProGet 2023.28. That version is due out this Friday, but I can provide you with a pre-release version early if you want to fix this issue immediately.

Thanks,
Rich

sebastian...

Hi @rhessinger

Thanks, but I don't think we will need a pre-release. The remaining problem is basically a convenience issue (getting from the report to the package's info page with one click). Everything else seems to work as expected with the latest pgscan version.