I ran it with the new stored procs, and I still got deadlocks, although far far fewer.

Looking at the xml, there was only 1 query deadlocked by the update instead of 6.

However, re-running the build (and the nuget restore, obviously) was able to resolve the deadlocks (I assume this is because it's deadlocking when caching packages, and with fewer deadlocking, more packages were cached).

So, good news so far. It's a definite improvement.

Now I just have to track down why 2 packages will restore directly from nuget, but not through proget's connector.... but that's a different issue.

Able to reproduce consistently (Brand new server. Transferred all internal nuget packages from old server, and any large build pulling cached, or caching external, nuget packages)

VM
Windows Server 2019 Standard 64 Bit
32 GB Ram
Intel(R) Xeon(R) CPU E5-4650 v3 @ 2.10GHz (2 Processors)

I will try out the attached queries and report back the results.

I'm not seeing how to upload the files. (And the plain text xml is too long to post itself.

The xml file (for one deadlock) had seven processes in it.

1-6 were:

SELECT * FROM [NuGetFeedPackageVersions_Extended] WHERE [Feed_Id] = @Feed_Id AND
[Package_Name_Lower] = LOWER(@Package_Id) AND (@Version_Text IS NULL OR
[Package_Version] = @Version_Text

#7 (And I suspect the deadlocker) was:

UPDATE [Feeds] SET [FeedState_Number] = COALESCE([FeedState_Number], 0) + 1
WHERE [Feed_Id] = @Feed_I

Microsoft.Data.SqlClient.SqlException (0x80131904): Transaction (Process ID 132) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
at Microsoft.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action1 wrapCloseInAction) at Microsoft.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose) at Microsoft.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady) at Microsoft.Data.SqlClient.SqlDataReader.TryHasMoreRows(Boolean& moreRows) at Microsoft.Data.SqlClient.SqlDataReader.TryReadInternal(Boolean setTimeout, Boolean& more) at Microsoft.Data.SqlClient.SqlDataReader.ReadAsyncExecute(Task task, Object state) at Microsoft.Data.SqlClient.SqlDataReader.InvokeAsyncCall[T](SqlDataReaderBaseAsyncCallContext1 context)
--- End of stack trace from previous location ---
at Inedo.Data.StrongDataReader.ReadAllAsync[TRow](IDbDataResult dbResult)
at Inedo.Data.DatabaseContext.ExecuteTableAsync[TRow](String storedProcName, GenericDbParameter[] parameters)
at Inedo.Data.DatabaseContext.ExecuteTableAsync[TRow](String storedProcName, GenericDbParameter[] parameters)
at Inedo.ProGet.Feeds.NuGet.NuGetFeed.GetPackageAsync(String id, NuGetVersion version)
at Inedo.ProGet.WebApplication.FeedEndpoints.NuGet.NuGetApi.GetHandler.ProcessGetRequestAsync(AhHttpContext context, WebApiContext apiContext, RequestData urlData, NuGetFeed feed)
at Inedo.ProGet.WebApplication.FeedEndpoints.NuGet.NuGetApi.ProcessRequestAsync(AhHttpContext context, WebApiContext apiContext, NuGetFeed feed, String relativeUrl)
at Inedo.ProGet.WebApplication.FeedEndpoints.NuGet.NuGetFeedHandler.ProcessRequestAsync(AhHttpContext context, WebApiContext apiContext, NuGetFeed feed, String relativeUrl)
at Inedo.ProGet.WebApplication.FeedEndpoints.FeedEndpointHandler.FeedRequestHandler.ProcessRequestAsync(AhHttpContext context)
ClientConnectionId:f2521b73-0e45-4796-bb52-faac892ad32f
Error Number:1205,State:51,Class:13