By migrating to ProGet to use as the NuGet server company wide, (as more and more developers switched to the new NuGet server) we were getting strange build errors relating to references coming from NuGet packages.
First, we tried to handle the issue by clearing all local NuGet caches, upgrading/downgrading nuget.exe, but later, it turned out that the source of the problem is, that some of the NuGet package folders in local cache do not contain the correct NuGet package, e.g. folder A.B.C/1.2.3 contains the content of X.Y.Z-1.1.0.nupkg?!?
At the end, we wrote a simple test application to download random .nupkg files parallely from ProGet server and verify the retrieved content of each .nupkg files, the result was disastrous: by trying to download 1000 random .nupkg parallely, only 89 packages were correct, the rest, 911 packages contained .nupkg of an other package being requested at the same time!
Obviously, there is a serious concurreny issue inside ProGet server, that by requesting A.B.C.nupkg, then X.Y.Z.nupkg is received.
This issue was first detected in an environment as:
OS = CentOS, ProGet = 5.2.7, Docker = 19.03.0, DB = MSSQL 2016 SP2, NuGet packages = 350 pieces and ~25K versions alltogether
And confirmed in following environment:
OS = CentOS, ProGet = 5.2.14, Docker = 19.03.0, DB = MSSQL 2016 SP2, NuGet packages = 350 pieces and ~25K versions alltogether
OS = Win10, ProGet = 5.2.14, Docker = 19.03.4, DB = MSSQL 2017 Docker, NuGet packages = 1K
OS = Win10, ProGet = 5.2.14 (both self-hosted and IIS hosted), DB = MSSQL 2017 Docker, NuGet packages = 1K