NuGet no longer works after upgrading to 2024

v.makkenze_6348

Hi,

I upgraded to 2024 and now I get errors in our builds and Visual Studio

Visual Studio simply says "Object reference not set to an instance of an object" and I can't browse for packages.

NuGet during the build gives a big stacktrace.

##[error]The nuget command failed with exit code(1) and error(Failed to retrieve information about 'NeuronTenantAuthenticationHttp' from remote source 'https://packages.vicrea.nl/nuget/Vicrea/FindPackagesById()?id='NeuronTenantAuthenticationHttp'&semVerLevel=2.0.0'.
  Object reference not set to an instance of an object.
NuGet.Protocol.Core.Types.FatalProtocolException: Failed to retrieve information about 'NeuronTenantAuthenticationHttp' from remote source 'https://packages.vicrea.nl/nuget/Vicrea/FindPackagesById()?id='NeuronTenantAuthenticationHttp'&semVerLevel=2.0.0'. ---> System.NullReferenceException: Object reference not set to an instance of an object.
   at NuGet.Protocol.RemoteV2FindPackageByIdResource.BuildModel(String id, XElement element)
   at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
   at System.Linq.Enumerable.WhereEnumerableIterator`1.MoveNext()
   at System.Collections.Generic.List`1.InsertRange(Int32 index, IEnumerable`1 collection)
   at NuGet.Protocol.RemoteV2FindPackageByIdResource.<>c__DisplayClass23_2.<<FindPackagesByIdAsyncCore>b__0>d.MoveNext()
...

When I browse to the url I get a lot of info back, not sure why the query starts at version 2.0.0
https://packages.vicrea.nl/nuget/Vicrea/FindPackagesById()?id='NeuronTenantAuthenticationHttp'&semVerLevel=2.0.0

I tried to downgrade but the version dropdown is empty.

Kinda stuck for now

atripp

Hi @v-makkenze_6348 ,

We have a copy of your database, and we can look to reproduce this. Offhand, I believe it's a result of "bad data" (duplicate packages) that are in your database, and it's the same issue behind here: https://forums.inedo.com/post/15730

We need to write a clean-up script for that, but in the meantime you'll want to rollback.

The version should show up in the drop down, but if it's not, maybe try downloading the Inedo Hub again: https://my.inedo.com/downloads

And if that's still not working, maybe try downloading the Offline Installer for the latest 2023.x version.

Cheers,
Alana

v.makkenze_6348

I uninstalled and installed the previous version, so we can build again.
Think the downgrade dropdown was empty because we got an IIS error at the end of each upgrade, maybe that's solved now too.

jw

@atripp said in NuGet no longer works after upgrading to 2024:

Offhand, I believe it's a result of "bad data" (duplicate packages) that are in your database, and it's the same issue behind here: https://forums.inedo.com/post/15730

Hi Alana,

before we run into the same thing, is this by any chance the same "bad data" you mentioned in EDO-10311?

rhessinger

Hi @jw,

Yes we believe this is the same data issue. Our initial thoughts were that this only affected analysis, but it seems to be affecting the NuGet feed itself as well.

Thanks,
Rich

arlymac_7956

This post is deleted!

rhessinger

This post is deleted!

arlymac_7956

This post is deleted!

rhessinger

This post is deleted!

arlymac_7956

This post is deleted!

jw

Is this issue addressed in the 2024.1 release?

A clean-up script, that is supposed to remove the duplicate entries, was mentioned to be included in a post 2024.0 release, but I could not find anything in the changelog.

stevedennis

Hi @jw ,

We have a data-cleanup script but did not ship it with ProGet 2024.1; we are considering it in ProGet 2023.2.

The above error occurred because of a 4-part version that ended in .0, but if you just delete that from the feed and add it back again, its should work. Can you try that?

We can provide you the data clean-up script if you want to try it. We ran it against several customer databases and it seems to work okay, and clean-up a lot of "bad" versions and casing.

Thank you,
Steve

jw

The reason I'm asking is that I'm not quite sure how to proceed with our instance.

I want to avoid running into the same exceptions that were mentioned above. From the changelog it was not quite clear to me if these issues have been addressed in code or we really need to run a cleanup script before upgrading.

From what I've been told, our database has duplicates that only differ by casing of the package names. So, if there is any script to identify or clean these duplicates I'd be happy to give it a shot.

stevedennis

Hi @jw ,

So far as we can tell, these sort of errors are only occurring when there is a 4-part NuGet version ending in 0 that was added in a previous version. For example, if you added 4.2.1.0 in a previous version, it could yield an analysis error when trying to view the package as 4.2.1 (i.e. how NuGet requests it).

The duplicate package names were a problem in ProGet 2023 as well, but we haven't seen any crashes do to that yet. Just weird results in the SBOM analysis. But these would have also been a problem in ProGet 2023.

So long story short, I don't think you need to run the script.

However, if you upgrade and run into these issues, then you can run it:
https://gist.github.com/apxltd/351d328023c1c32852c30c335952fabb

If that still doesn't work, then rollback to ProGet 2023 and restore the database. And hopefully send it to us again so we can investigate the issue :)

Thank you,
Steve

jw

Thanks for the script, it does indeed find a few things.

I have an error all the way at the end of the script. Is this because I ran it on a 2023.32 database and it is only supposed to be run on a 2024.x database? (Only used DryMode so far)

Deleting Duplicate Ids...
Error: The DELETE statement conflicted with the REFERENCE constraint "FK__FeedPackageVersions__PackageVersionIds". The conflict occurred in database "ProGet", table "dbo.FeedPackageVersions", column 'PackageVersion_Id'.
[2024-04-29 17:57:00] 206 rows affected in 356 ms

stevedennis

@jw the script should also work in ProGet 2023

That error is implying that data in the FeedPackageVersion didn't get cleaned up, as it should have been done on this portion:
https://gist.github.com/apxltd/351d328023c1c32852c30c335952fabb#file-gistfile1-txt-L449

Unfortunately this difficult to debug on its own, and it will be impossible to debug w/o the database itself, but if you're comfortable with SQL feel free to modify or tweak. It's not an easy script :(

We haven't yet run this against all customer databases yet, but it's our list this week. If you haven't sent us your database already, please do :)

jw

Unfortunately this difficult to debug on its own, and it will be impossible to debug w/o the database itself, but if you're comfortable with SQL feel free to modify or tweak. It's not an easy script :(

Yea, the size and also the fact that it deletes things puts that out of my comfort zone. ;)

We haven't yet run this against all customer databases yet, but it's our list this week. If you haven't sent us your database already, please do :)

You can find our database in EDO-10311, I have also extended the expiration date in case you need to redownload.

Executing a revised version manually is perfectly fine for me, if you decide not putting it into the 2024.2 release. I'd appreciate clear communication what should or should not be done with the next update to get this fixed.

Thanks for the effort.

stevedennis

Hi @jw ,

Thanks - we modified the script and tested it against your database backup:
https://gist.github.com/apxltd/351d328023c1c32852c30c335952fabb

That said, your duplicate data should not cause any problems. The only feed packages this seems to modify in your database are Microsoft.NetCore.App.Runtime.win-x86-8.0.0 and Microsoft.NetCore.App.Runtime.win-x64-8.0.0. Both are cached.

That said, given that it had a bug, we probably won't put this script in 2024.2; we'd much rather "share the script on a case-by-case" basis to fix problems, until we're confident they will be resolved. Then decide how to have users repair the data.

Thank you,
Steve

jw

@stevedennis

Thanks for the updated version - it works smoothly now.