If for some reason the client has no packages locally (ie: Within a docker build where there are no local packages) it will query proget in order to obtain them. Rarely I am able to do a full dotnet restore without seeing a timeout error in Proget. The error seems to be coming from timeouts querying the DB, so even with enterprise version and load balancing this problem would probably be worse.
The server where it is hosted is more than enough to handle the load of a single client. SSD, 4 cores, and 32 GB RAM and with enterprise SQL Server too which has very few connections open when these errors occur.
I see many errors like the one below: (this package doesn't exist in proget, however since it is in NuGet.Config will still query ProGet.
Retrying 'FindPackagesByIdAsyncCore' for source 'https://proget.xxx.io/nuget/xxx-nuget/FindPackagesById()?id='System.Diagnostics.Process'&semVerLevel=2.0.0'.
Response status code does not indicate success: 504 (Gateway Timeout).
In proget web I see many errors like:
An error occurred processing a GET request to http://proget.xxx.io/nuget/xxx-nuget/v3/flatcontainer/system.text.json/index.json: Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.
stack trace:
System.Data.SqlClient.SqlException (0x80131904): Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.
---> System.ComponentModel.Win32Exception (258): Unknown error 258
at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action1 wrapCloseInAction) at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose) at System.Data.SqlClient.SqlCommand.EndExecuteNonQueryInternal(IAsyncResult asyncResult) at System.Data.SqlClient.SqlCommand.EndExecuteNonQuery(IAsyncResult asyncResult) at System.Threading.Tasks.TaskFactory
1.FromAsyncCoreLogic(IAsyncResult iar, Func2 endFunction, Action
1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
at Inedo.Data.SqlServerDatabaseContext.CreateConnectionAsync()
at Inedo.Data.DatabaseContext.ExecuteInternalAsync(String storedProcName, GenericDbParameter[] parameters)
at Inedo.Data.SqlServerDatabaseContext.ExecuteInternalAsync(String storedProcName, GenericDbParameter[] parameters)
at Inedo.Data.DatabaseContext.ExecuteTableAsync[TRow](String storedProcName, GenericDbParameter[] parameters)
at Inedo.ProGet.Feeds.NuGet.V2.LocalPackageSource.FindPackagesByIdAsync(NuGetQueryOptions options, String id) in C:\InedoAgent\BuildMasterTemp\192.168.44.60\Temp_E75101\Src\ProGetCoreEx\Feeds\NuGet\V2\LocalPackageSource.cs:line 161
at Inedo.ProGet.Feeds.NuGet.NuGetFeed.FindPackagesByIdAsync(NuGetQueryOptions options, String id) in C:\InedoAgent\BuildMasterTemp\192.168.44.60\Temp_E75101\Src\ProGetCoreEx\Feeds\NuGet\NuGetFeed.cs:line 157
at Inedo.ProGet.WebApplication.FeedEndpoints.NuGet.V3.PackageBaseAddressHandler.ProcessRequestAsync(NuGetFeed feed, HttpContext context, String relativeUrl) in C:\InedoAgent\BuildMasterTemp\192.168.44.60\Temp_E75101\Src\ProGet.WebApplication\FeedEndpoints\NuGet\V3\PackageBaseAddressHandler.cs:line 39
at Inedo.ProGet.WebApplication.FeedEndpoints.FeedEndpointHandler.FeedRequestHandler.ProcessRequestAsync(HttpContext context) in C:\InedoAgent\BuildMasterTemp\192.168.44.60\Temp_E75101\Src\ProGet.WebApplication\FeedEndpoints\FeedEndpointHandler.cs:line 192
ClientConnectionId:631e8659-835c-43aa-8815-79f9c5633cf5
Error Number:-2,State:0,Class:11
Some other errors in the container logs:
An error occurred processing a GET request to http://proget.xxx.io/nuget/xxx-nuget/v3/flatcontainer/system.memory/index.json: Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.
An error occurred processing a GET request to http://proget.xxx.io/nuget/xxx-nuget/v3/flatcontainer/microsoft.extensions.primitives/index.json: ExecuteNonQuery requires an open and available Connection. The connection's current state: Broken.
Can you please investigate what can we do about it? I would rather it takes longer to restore than all these timeout errors.
Note that this is with proget core. Also maybe there are more efficient ways to identify if a package does exist or not before querying the DB. Cache options?
PS: While this situation is occurring, the proget web becomes unresponsive and it stays loading forever until the requests are completed.
Thank you