Packages with Noncanonical Names errors on internalized packages

jfullmer_7346

So we internalize the chocolatey package WinSCP from the chocolatey community repository along with the matching version of the Powershell module of WinSCP from the PSGallery.

In both cases, proget 2025 is giving us a feed error on our powershell and chocolatey repos that this package has a "Noncanonical Name."

It gets indexed in the repo as winscp but the nuspec/psd1/psgetmoduleinfo.xml of the package has the id and (file names for nuspec and psd1) as WinSCP.

I am pushing the package via a drop path after some internal processes repackage the nupkg to our internalized format.
The WinSCP.nupkg we put in the drop path has the WinSCP case in the nupkg file name and the same for the nuspec within.

I've tried fully deleting all versions of the package in the gui but re-pushing it has it still coming back as winscp in the proget side list of the package and giving this unhealthy feed error.

I feel like I'm just missing something simple to fix this.

jfullmer_7346

I have tried deleting the packages from my feeds completely, running the package re-index, and then pushing them again and the problem still persists.

stevedennis

@jfullmer_7346 this is a bit tricky behind the scenes but hopefully will be resolved with PG-3047 -- which we hope to get in the next or following maintenance release

jfullmer_7346

@stevedennis Is there anything extra that needs to be done on my side to fix this with the recent fix. I'm still getting the error for this package. I've tried forcing the case to be the same but no dice thus far.

atripp

Hi @jfullmer_7346,

unfortunately it looks like the fix didn't work and made things slightly worse

We are reverting that change in ProGet 2025.9 and will try a new approach via PG-3100 - hopefully in ProGet 2025.10

jfullmer_7346

@atripp If there's anything I can do to help with troubleshooting or debugging, I am happy to help.

atripp

@jfullmer_7346 thanks! As an FYI...

Names and Versions are centrally indexed, and were intended to be "write-only" by design
NuGet does not have case-sensitive names, but some earlier bugs allowed duplicate names to be created
we are adding Duplicate Names (e.g. winscp and WinSCP) and Duplicate Versions (e.g. winscp-4.0.0 and WinSCP-4.0.0 checking to feed integrity
When re-indexing a feed, you'll get an option to de-duplicate names/versions, which will fix it across all feeds
Pulling or Publishing a package will update the casing of a centrally-indexed name
Also we're getting rid of the concept of "Noncannonical Names" altogether, since we've discovered many NuGet packages "change casing" at some version

jfullmer_7346

@atripp Finally had some time to look at this again.
Updated to 2025.10 and gave the reindex a go.
Looked like it was gonna work but then it threw this error at the end of re-index

Performing reindex for PowerShell feed 2: ArrowPS
Deactivating feed.
There are 31 duplicated PowerShell name sets...
pkg:nuget/winscp (ID=562) is the canonical name in the set of 2...
Deduplicating pkg:nuget/WinSCP (ID=563)...
Re-activating feed.
Unhandled exception: Npgsql.PostgresException (0x80004005): P0001: Cannot deduplicate given nameid
   at Npgsql.Internal.NpgsqlConnector.ReadMessageLong(Boolean async, DataRowLoadingMode dataRowLoadingMode, Boolean readingNotifications, Boolean isReadingPrependedMessage)
   at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder`1.StateMachineBox`1.System.Threading.Tasks.Sources.IValueTaskSource<TResult>.GetResult(Int16 token)
   at Npgsql.NpgsqlDataReader.NextResult(Boolean async, Boolean isConsuming, CancellationToken cancellationToken)
   at Npgsql.NpgsqlDataReader.NextResult(Boolean async, Boolean isConsuming, CancellationToken cancellationToken)
   at Npgsql.NpgsqlDataReader.NextResult()
   at Npgsql.NpgsqlCommand.ExecuteReader(Boolean async, CommandBehavior behavior, CancellationToken cancellationToken)
   at Npgsql.NpgsqlCommand.ExecuteReader(Boolean async, CommandBehavior behavior, CancellationToken cancellationToken)
   at Inedo.ProGet.Data.PostgresDatabaseContext.PostgresCommand.ExecuteReader()
   at Inedo.Data.DatabaseContext.ExecuteInternal(String storedProcName, GenericDbParameter[] parameters, DatabaseCommandReturnType returnType)
   at Inedo.Data.DatabaseContext.ExecuteNonQuery(String storedProcName, GenericDbParameter[] parameters)
   at Inedo.ProGet.Data.DB.Context.PackageNameIds_DeduplicateId(Nullable`1 Canonical_PackageName_Id, Nullable`1 Duplicate_PackageName_Id)
   at Inedo.ProGet.Data.DB.PackageNameIds_DeduplicateId(Nullable`1 Canonical_PackageName_Id, Nullable`1 Duplicate_PackageName_Id)
   at Inedo.ProGet.Feeds.PackageFeed`5.ReindexFeedAsync(IReindexFeedExecutionContext context, CancellationToken cancellationToken)
   at Inedo.ProGet.Executions.ReindexFeedExecution.ExecuteAsync(IManualExecutionContext context)
   at Inedo.ProGet.Executions.ReindexFeedExecution.ExecuteAsync(IManualExecutionContext context)
   at Inedo.ProGet.Service.Executions.ActiveManualExecution.ExecuteAsync()
  Exception data:
    Severity: ERROR
    SqlState: P0001
    MessageText: Cannot deduplicate given nameid
    Where: PL/pgSQL function "PackageNameIds_DeduplicateId"(integer,integer) line 27 at RAISE
    File: pl_exec.c
    Line: 3925
    Routine: exec_stmt_raise

jfullmer_7346

I can re-index without the deduplicate option and it comes back as clean, though I imagine it will turn into a loop when I run the integrity check again.

jfullmer_7346

Also, should the integrity check be taking 30 minutes?
I figure it's because it's a chocolatey repo where the files for packages are embedded in the package, so there are some very large packages.
The powershell module repo runs through the check and the reindex in more reasonable time frames like 30 seconds.

atripp

@jfullmer_7346 thanks for giving it a shot, we'll take a closer look!

The "good news" is that the error message is a "sanity check" failure, so now have an idea of what's causing the error:

    -- Sanity Check (ensure there are no duplicate versions)
    IF EXISTS (
        SELECT * 
         FROM "PackageVersionIds" PV_D,
              "PackageVersionIds" PV_C
        WHERE PV_D."PackageName_Id" = "@Duplicate_PackageName_Id"
          AND PV_C."PackageName_Id" = "@Canonical_PackageName_Id"
          AND PV_D."Package_Version" = PV_C."Package_Version"
          AND (   (PV_D."Qualifier_Text" IS NULL AND PV_C."Qualifier_Text" IS NULL)
               OR (PV_D."Qualifier_Text" = PV_C."Qualifier_Text") )
       ) THEN RAISE EXCEPTION 'Cannot deduplicate given nameid'; RETURN; END IF;

In this case, it's saying that there are "duplicate versions" remaining (i.e. WinSCP-1.0.0 and winscp-1.0.0). Those should have been de-duplicated earlier. I wonder if the PackageVersionIds_GetDuplicates() function is not returning the right results.

I'm not sure what your experience w/ PostgreSQL is, but are you able to query the embedded database? If not, that's fine... it's not meant to be easy to query.

Also, should the integrity check be taking 30 minutes?

Maybe. The integrity check needs to verify file hashes, so that involves opening and streaming through all the files on disk. So when you have a lot of large packages, then it's gonna take a while.

jfullmer_7346

@atripp I'm fairly experienced with postgres and had originally tried troubleshooting at that level so I already have connections for running sql commands all setup.
Ran that stored procedure

SELECT public."PackageVersionIds_GetDuplicates"();
PackageVersionIds_GetDuplicates|
-------------------------------+
(8401,563,6.5.3,)              |
(8334,562,6.5.3,)              |
(8235,563,6.5.2,)              |
(8225,562,6.5.2,)              |
(8336,562,6.3.6,)              |
(6107,563,6.3.6,)              |
(629,563,5.21.7,)              |
(613,562,5.21.7,)              |
(630,563,5.21.8,)              |
(614,562,5.21.8,)              |

Also ran this one

SELECT public."PackageNameIds_GetDuplicates"();
PackageNameIds_GetDuplicates                                  |
--------------------------------------------------------------+
(562,nuget,,winscp,8336)                                      |
(563,nuget,,WinSCP,8401)                                      |
(1383,nuget,,jquery,)                                         |
(1591,nuget,,magick.net-q16-anycpu,)                          |
(1592,nuget,,magick.net-q16-hdri-anycpu,)                     |
(1593,nuget,,magick.net-q16-x64,)                             |
(1594,nuget,,magick.net-q8-anycpu,)                           |
(1595,nuget,,magick.net-q8-openmp-x64,)                       |
(1596,nuget,,magick.net-q8-x64,)                              |
(2822,nuget,,mongodb.driver,)                                 |
(3191,nuget,,Umbraco.CMS,)                                    |
(6487,nuget,,Microsoft.NetCore.App.Runtime.win-arm,)          |
(6488,nuget,,Microsoft.NetCore.App.Runtime.win-arm64,)        |
(6489,nuget,,Microsoft.NetCore.App.Runtime.win-x64,)          |
(6490,nuget,,Microsoft.NetCore.App.Runtime.win-x86,)          |
(6493,nuget,,Microsoft.NETCore.App.Runtime.linux-arm,)        |
(6494,nuget,,Microsoft.NETCore.App.Runtime.linux-arm64,)      |
(6495,nuget,,Microsoft.NETCore.App.Runtime.linux-musl-arm,)   |
(6496,nuget,,Microsoft.NETCore.App.Runtime.linux-musl-arm64,) |
(6497,nuget,,Microsoft.NETCore.App.Runtime.linux-musl-x64,)   |
(6498,nuget,,Microsoft.NETCore.App.Runtime.linux-x64,)        |
(6504,nuget,,Microsoft.NetCore.App.Runtime.osx-arm64,)        |
(6505,nuget,,Microsoft.NetCore.App.Runtime.osx-x64,)          |
(7315,nuget,,monero,)                                         |
(8613,nuget,,Umbraco.Cms.Web.BackOffice,)                     |
(9133,nuget,,SSCMS,)                                          |
(9404,nuget,,CefSharp.OffScreen.NETCore,)                     |
(9405,nuget,,CefSharp.WinForms.NETCore,)                      |
(9406,nuget,,CefSharp.Wpf.NETCore,)                           |
(9895,nuget,,UmbracoCms,)                                     |
(10001,nuget,,UmbracoCMS.Core,)                               |
(10906,nuget,,UmbracoCms.Web,)                                |
(28650,nuget,,jQuery,)                                        |
(28651,nuget,,Microsoft.NETCore.App.Runtime.osx-arm64,)       |
(28652,nuget,,Microsoft.NETCore.App.Runtime.osx-x64,)         |
(28653,nuget,,Microsoft.NETCore.App.Runtime.win-arm,)         |
(28654,nuget,,Microsoft.NETCore.App.Runtime.win-arm64,)       |
(28655,nuget,,Microsoft.NETCore.App.Runtime.win-x64,)         |
(28656,nuget,,Microsoft.NETCore.App.Runtime.win-x86,)         |
(28657,nuget,,UmbracoCMS,)                                    |
(28658,nuget,,Microsoft.NetCore.App.Runtime.linux-arm,)       |
(28659,nuget,,Microsoft.NetCore.App.Runtime.linux-arm64,)     |
(28660,nuget,,Microsoft.NetCore.App.Runtime.linux-musl-arm,)  |
(28661,nuget,,Microsoft.NetCore.App.Runtime.linux-musl-arm64,)|
(28662,nuget,,Microsoft.NetCore.App.Runtime.linux-musl-x64,)  |
(28663,nuget,,Microsoft.NetCore.App.Runtime.linux-x64,)       |
(28664,nuget,,sscms,)                                         |
(28665,nuget,,MongoDB.Driver,)                                |
(28666,nuget,,UmbracoCMS.Web,)                                |
(29225,nuget,,UmbracoCms.Core,)                               |
(31198,npm,,rambox,)                                          |
(39678,npm,,Rambox,)                                          |
(51673,nuget,,Monero,)                                        |
(55232,npm,,vvvebJs,)                                         |
(55420,nuget,,Umbraco.Cms,)                                   |
(55500,npm,,vvvebjs,)                                         |
(69073,cargo,,surrealdb,)                                     |
(69127,cargo,,deno,)                                          |
(69135,cargo,,tauri,)                                         |
(69283,cargo,,Deno,)                                          |
(69315,cargo,,Tauri,)                                         |
(69475,cargo,,mdbook,)                                        |
(69476,cargo,,mdBook,)                                        |
(75720,nuget,,Umbraco.Cms.Web.Backoffice,)                    |
(77034,nuget,,CefSharp.OffScreen.NetCore,)                    |
(77035,nuget,,CefSharp.WinForms.NetCore,)                     |
(77036,nuget,,CefSharp.Wpf.NetCore,)                          |
(19805812,nuget,,Magick.NET-Q16-AnyCPU,)                      |
(19805814,nuget,,Magick.NET-Q16-HDRI-AnyCPU,)                 |
(19805823,nuget,,Magick.NET-Q16-x64,)                         |
(19805825,nuget,,Magick.NET-Q8-AnyCPU,)                       |
(19805828,nuget,,Magick.NET-Q8-OpenMP-x64,)                   |
(19805829,nuget,,Magick.NET-Q8-x64,)                          |
(48480969,cargo,,SurrealDB,)                                  |

There are versions that I thought were deleted in that first result as I only have 1 version listed for each nuget repo, only keeping the latest one for internalized packages.

So maybe some old ids didn't get fully deleted from the database and they're caught as dupes? Maybe they're kept on purpose for historical data as that id isn't going to be reused? Maybe the duplicate check should also make sure it's an active version that is listed in the UI? Just spitballing, hopefully it helps

jfullmer_7346

Is it actually getting upset not because of canonical names and case differences, but because they're both nuget packages (in different repos) with the same name?

atripp

Hi @jfullmer_7346 ,

I haven't had a chance to look into more details, but thanks for providing the results of the query.

FYI - the PackageNameIds and PackageVersionIds are designed as a kind of "permanent, read-only record" -- once added they are not deleted or modified. Even if all packages are deleted (i.e. FeedPackageVersions). This is why this the "duplicate name" is such a headache to deal with.

That said, on a quick glance, we can see exactly where the error is coming from: there are duplicate versions (i.e. (ID=563)-v6.5.3 and (ID=562)-v6.5.3). So, when we try to deduplicate (ID=563) and (ID=562) (i.e. winscp and WinSCP), we get the error as expected.

What's not expected is that those versions were not de-duplicated in the earlier pass. My guess is that it's related to winscp being in one feed and WinSCP being in the other -- we tried to be conservative, and keep the de-duplication to packages related to the feed.

I'm thinking we just change that logic to "all packages of the feed type". Anyway, please stay tuned. We'll try to get it in the next maintencne release.

Thanks,
Alana

jfullmer_7346

@atripp What's odd is that 6.5.3 and 6.5.2 shouldn't be duplicate I think? The powershell module version (WinSCP) is still 6.3.6 so it shouldn't be seen as a duplicate version I would think.

Could it be because we have dev versions of these repos and we promote the packages? Though we do that for all our packages, so I imagine we'd see it way more often.

atripp

Hi @jfullmer_7346,

It's nothing you did.

The underlying issue is that a bug in ProGet allowed WinSCP (ID=563) to be added to the PackageNameIds table; that should never have happened since nuget is case insensitive. We've since fixed that bug.

However, once you have a duplicate name, weird things happen since querying for "give me the PackageID for nuget://winscp" returns two results instead of one. So now when you query "give me the VersionID for (ID=563)-v6.5.3", a new entry is created.

This has been a very long-standing issue, but there aren't any major consequences to these "weird things" except casing in the UI and the health check now fails.

But we're on it :)

Thanks,
Alana