@Dan_Woolf

Thanks for the update!

for the --append-dependencies flag do i only use that on the 2nd attempt?

@Dan_Woolf Thank you!

if it helps, my version is 2022.18 (Build 7)

@Dan_Woolf

Testing:
that test plan should work perfectly fine for me. We have 2 main scenarios for using this on yarn:

1. there are frontend-only builds where it's a project running on node.js and yarn
2. there are dotnet builds with SPA .csproj in them that downloads yarn in the background. if it was npm those calls would be done the same there.

Inspect:
I tried that command on a project that I haven't created an SCA for. I received a peculiar error testing it out that I'm not sure how i'd go about troubleshooting. It didn't create the project, and when I made one myself it still errored out with this message.

I have version 1.4.1 installed as a dotnet tool. Below is what I'm calling:

pgscan identify `
            --input=$pwd\PROJECT.sln `
            --proget-url=https://myprogeturl.mydomain `
            --consumer-package-version=0.1.0 `
            --project-name=PROJECT `
            --api-key=myApiKey `
            --report

Heyo @Dan_Woolf

1. Thanks for the clarification on the package.json parameter.
2. I'd be interested in helping work on it. What would be needed of me?
3. does Inspect require me to manually create a project for SCA or is there some manner of creating them programmatically?

I've got a few questions about how the pgscan utility works with regards to NPM packages.

• The code for the npm scanner indicates that it requires a package-lock.json file to be present in that directory. Is this intended to be used against the root solution, or a csproj? I have a project where this is used, but I consistently get a "pgscan : Unhandled exception. System.Collections.Generic.KeyNotFoundException: The given key was not present in the dictionary." error regardless of where that file is.
• If we're using Yarn, which produces a yarn.lock file instead, would I still be able to use this with the pgscan utility? Yarn uses the same packages as NPM, with different commands + a slightly different process for how it behaves.
• If I have a dotnet SPA, would I be required to report my packages twice: once to a nuget feed and once to a npm feed in order to get my package data uploaded? I think the answer is yes, but mostly because I don't understand what the Identify command is for, or why i should use it instead of publish.

@atripp that's extremely informative!

What I found this morning that was deleting and re-adding the connector for the IP feed after clearing the cache(because I'm paranoid) was what needed to happen for it to stop being cached by our Mainline feed.

My version of Proget is 6.0.7.

We created our own custom package and put it in a feed for custom packages made by my organization, called the IP feed. There's a secondary feed named Mainline, which has packages we store that we got from 3rd party sources, mainly Nuget.org. Mainline has a connector to IP that pulls packages in from said feed.

Recently I was told that a Nuget Package was being removed from IP as it was renamed, and thus I should remove it from the feed. I delete it from IP, but it still shows in mainline. I then try the following:

• unlist package before ensuring all versions are deleted from IP.
• go into Azure Blob storage (where the packages are kept) and delete it. I'll note that it was not in blob under the containers that had IP or Mainline when I verified.
• run the cleanup tasks in the management UI on both feeds.
• set cleanup of the feed under storage and Retention to Delete Cached connector Packages, and delete old versions of everything except the 10 most recent. It's been 24 hours and it's still there.
• clicked the "clear cache" button on the feed management page.

I see it still in Mainline, with the little radio tower symbol which indicates it's pulled from a connector. In the dropdown menu that allows me to download a package the option to delete it is not one of them. I'm out of ideas, sadly.

Who do I have to sacrifice to make the package go away? Albino goats are particularly expensive.

@atripp you're the best.

I'll give these a shot and come back if i run into something i can't work through.

@atripp

Are those APIs case sensitive? because I had originally tried to use npmFeeds_SetTagVersion and got the first error I posted

[16:07:11]Step 5/6: Proget: Apply tag to Latest Version (15s)
[16:07:11][Step 5/6] Step 1/1: Apply Latest Tag to Version (PowerShell) (15s)
[16:07:27][Step 1/1] Invoke-RestMethod : There is no API method named npmFeeds_SetTagVersion
[16:07:27][Step 1/1] Error message: Invoke-RestMethod : There is no API method named npmFeeds_SetTagVersion
[16:07:27][Step 1/1] At C:\TeamCity\BuildAgent\temp\buildTmp\powershell16555570498298526.ps1:25 char:13

I would have expected to get a 400 error from your API from my bad parameters making an invalid request instead of this.

@atripp

What I mean is that I want to try to tag npm packages via the API. I found this in a build that wasn't finished that I am now responsible for. The engineer previously responsible for it has left our organization and I don't know where he got this or how, and this was not documented by him to my knowledge in any ticket he worked on.

Here's the actual API call I saw in our build in a custom powershell runner in Team City. The %variable% references are Team City syntax for how that software inserts its own variables into script runners.

$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("Content-Type", "application/json")

$body = "{
`n`"Feed_Name`" : `"npm`",
`n`"FeedType_Name`" : `"npm`",
`n`"API_Key`" : '%svc.proget.apikey%',
`n`"Feed_Id`": 6,
`n`"Package_Name`": '%package.name%',
`n`"Scope_Name`": `"MyCompany`",
`n`"Tag_Name`": `"latest`",
`n`"Tag_Version`": '%app.version%'
`n}"

$response = Invoke-RestMethod "%svc.proget.host%/api/json/NugetFeeds_SetTagVersion" -Method 'POST' -Headers $headers -Body $body

Someone in my organization was using the NugetFeeds_SetTagVersion endpoint to attempt to tag nugets. I'd like to try the same for NPM packages, but my api calls return with an error: Invoke-RestMethod : There is no API method named npmFeeds_SetTagVersion

Does this exist for NPM? Does proget have a swagger or other query-able API that can tell me the definitions of what's there?

Good Morning @atripp ,

I'll try and experiment in fiddler with this to see if I can learn more about it. We don't have any cached responses that I am aware of internally but I'll investigate some sources that I think might potentially be the cause.

Is this the setting you were referring to called Configure Feed Caching?

Hello!

I currently run proget 5.3.28 (build 16). We run builds on our build server to update the nuget packages on our feed by method of a drop folder. Build operations go like this:

1 Restore solution
2 copy every .nuget file to a shared drive on the proget server
3 wait

I've watched the server run through it's processes, and it looks like it vacuums up all of those nuget files really quickly. I check the nuget page on the server, see the new package updated, then go try to run my other build that needed this new package. More often than not, the build fails because it can't find the new package version, citing the url for the feed that it cant find it in.

I normally have to wait anywhere from 10 minutes to an hour for a package to be searchable on proget through nuget. I occasionally get really lucky and I can use it right away. This has always been the case for us in the 2 or 3 years we have been using the software.

What factors determine how fast it reindexes packages to make them query-able in the system? I'd like to not have to tell my dev teams that they have to wait an hour for a build to rerun.

If there is a better method to updating the package feed, I'd love to do that instead. Please let me know.

Thank you @atripp! Using the utility multiple times isn't an issue for us. I'll find some way of making that work.

Thank you again for your assistance here.

@atripp Thank you for getting back to me! This is incredibly helpful.

When you point pgscan to a .sln file, it will parse the file and look for projects. Under each project, the tool will look for packages.config (which is the older style project format) and then project.assets.json (which is the newer style).

Does this register submodules and subprojects individually under the group name? Using my previous example: would all the nuget dependencies get registered under MyProject1, or would I see Myproject1\submodule1 as how it's registered? If it doesn't do the latter, what would you recommend me do in order to achieve that state?

This isn't currently supported it seems (I don't see a delete button in the UI), but if you don't mind going to the database, you can just do DELETE [PackageDependents] and then all the rows are cleared.

I don't mind going into the DB to fix my massive mistake. Is there a place I can feature request this as a UI addition?

I have proget build 5.3.25. On every page where I have registered the package dependencies the table for Package downloads cuts off what looks like the 3rd column.

If i hide the table I can see the missing information, and it lines up correctly.

I have several questions on how the PG Scan utility works that the documentation doesn't immediately answer for me.

suppose I am calling the pgscan utility....

& $pgscanDir\pgscan.exe Publish --type=$pkgType --input=$solutionName --package-feed=$feedName --proget-url=$progetURL --consumer-package-source=$sourceFeedName --consumer-package-name=$packageName --consumer-package-version=$releaseNumber --consumer-package-group=$devCodeName --api-key=$apiKey

1. What actually belongs in the consumer-package-name and consumer-package-version fields? I had assumed that this was supposed to be the package that I was using, such as Newtonsoft.Json. Is it actually supposed to represent my application?
2. How does pgscan handle subprojects and submodules when I call it from the .sln level? Say MyProject.sln has a subproject1.csproj, and uses submodule1.sln . How would pgscan report the nuget packages that are used?
3. How do I remove dependencies from packages once I register them? I'm fairly certain that I did all of these wrong and need to wipe the entire slate clean.

@atripp You bet. I'll see what I can do and let you know

@atripp I might have to go with trying a connector for this package. I got into this whole mess because I was unable to restore Owin at all after I had it in my feed after an update. I tried re-packaging it via the UI from within proget to calling it 1.0.0. This only made a package that we couldn't properly download or restore because other things depending on it were busted.

This is strangely not repeatable for me either.

I made a package on a new feed I named TestFeed, and created a nuget package called TestyMcBadVersion, gave it the semver of 1.0, and it uploads as 1.0.0 on the system. It was a completely empty package and I created it via CLI using the latest nuget.exe. I tried repackaging and deleting like we had reviewed above and... nothing. It works as intended. This was supposed to mimic the problems I was having with an older package. The official name of the package in question is Owin (I've had troubles with this in the past). the official Nuspec for it lists it as 1.0, in Proget it's listed as 1.0 because the software doesn't try to auto-fix it.

The problem seems relegated to my main feed where most of my developers pull from.