Internal Server Error on nuget restore

We used ProGet in version 2.2.13 and had sometimes following error when restoring packages locally:
"The remote server returned an error: (500) Internal Server Error."
After this error there resides some packages as 0 byte temp file in the NuGet cache.

In the release notes I found following entry for version 3.1.0:
"PG-202 - FIX: Failed package downloads from connectors result in a 0-byte file"

Weeks ago we had updated our server to the new version 3.1.0. After the update I had more often errors (500).
On the ProGet server machine I recognized that the CPU usage was very high (dual core; 100%) in case of restoring packages locally.
The process sqlservr used the full CPU power. The error occures more often when I locally cleaned my NuGet cache. After all, I set
the property "RebuildPackageIndex" in AllSettings. After rebuild of the index on the SQL server, the CPU usage changed, but the "IIS Worker Process"
uses now the most CPU usage after a call "nuget restore mySolution.sln" (package folder and NuGet cache cleaned). The solution includes a lot of projects
and about 60 NuGet components as dependencies.

We use a default installation of ProGet with IIS hosting on Windows 7 Professional.

After all, I found out, that the problem is introduced with nuget restore command. The restore command allows parallel processing, so it runs faster.
But in this case the ProGet Server uses so much CPU power that sometimes the error 500 occures. If I use the parameter DisableParallelProcessing in
"nuget restore mySolution.sln -DisableParallelProcessing", it runs more stable.

How could we solve this problem on the ProGet server (with parallel processing)?
Have other the same issue?

Thanks

Product: ProGet
Version: 3.1.0

Where are you getting the 500 errors, in the web application or returned to the NuGet client? Are there any error messages associated with these 500 errors?

The SQL CPU usage is normal as it needs to index possibly thousands of packages.

Depending on what the parallel restore command does, it may or may not be supported yet by ProGet. If it just downloads a bunch of packages at the same time then it should already work.

Are you able to reproduce this with a small console program that we could reproduce in our test environment?

I have done following steps to reproduce:

Create a new visual studio project (eg. C Sharp class library)
Goto "Manage NuGet Packages..." on the new project.
Add a lot of (>50) packages from your ProGet feed.
Save the solution
Delete the packages folder (at the place from the solution)
Delete the NuGet Cache folder (C:\Users\par\AppData\Local\NuGet\Cache)
Goto the cmd.exe and run "nuget.exe restore mySolution.sln"

Sometimes I have to run the last three steps more than once until the error "The remote server returned an error: (500) Internal Server Error." or "Unable to find version '1.9.3.34' of package 'x.y.z'." occures. The second error is when the package in the NuGet cache has a size of 0 byte.

If you runs "nuget.exe restore mySoultion.sln -DisableParallelProcessing", it works much better.

Thanks

I've had no luck reproducing this behavior in ProGet v3.1.2 with NuGet.exe client v2.8.50320.36, attempting to both install all packages hosted by ProGet and from connected feeds with and without LDAP enabled. Could you try upgrading to ProGet v3.1.2 to see if you still have this issue? I'm not aware of any fixes that are directly related to this, but it's possible one of the bugfixes has indirectly fixed it.

For reference, here is the packages.config file contents I used to test:

<?xml version="1.0" encoding="utf-8"?>
<packages>
  <package id="angularjs" version="1.2.22" targetFramework="net45" />
  <package id="Antlr" version="3.5.0.2" targetFramework="net45" />
  <package id="Antlr" version="3.4.1.9004" targetFramework="net45" />
  <package id="AutoMapper" version="3.2.1" targetFramework="net45" />
  <package id="AWSSDK" version="2.2.4.0" targetFramework="net45" />
  <package id="bootstrap" version="3.2.0" targetFramework="net45" />
  <package id="Castle.Core" version="3.3.0" targetFramework="net45" />
  <package id="Castle.Windsor" version="3.3.0" targetFramework="net45" />
  <package id="Common.Logging" version="2.1.2" targetFramework="net45" />
  <package id="CommonServiceLocator" version="1.2" targetFramework="net45" />
  <package id="DotNetZip" version="1.9.3" targetFramework="net45" />
  <package id="EntityFramework" version="6.1.1" targetFramework="net45" />
  <package id="EntityFramework.SqlServerCompact" version="6.1.1" targetFramework="net45" />
  <package id="Iesi.Collections" version="4.0.1.4000" targetFramework="net45" />
  <package id="iTextSharp" version="5.5.2" targetFramework="net45" />
  <package id="jQuery" version="2.1.1" targetFramework="net45" />
  <package id="jQuery" version="1.9.0" targetFramework="net45" />
  <package id="log4net" version="2.0.3" targetFramework="net45" />
  <package id="MathNet.Numerics" version="3.2.1" targetFramework="net45" />
  <package id="Microsoft.AspNet.Mvc" version="5.2.0" targetFramework="net45" />
  <package id="Microsoft.AspNet.Razor" version="3.2.0" targetFramework="net45" />
  <package id="Microsoft.AspNet.SignalR" version="2.1.1" targetFramework="net45" />
  <package id="Microsoft.AspNet.SignalR.Core" version="2.1.1" targetFramework="net45" />
  <package id="Microsoft.AspNet.SignalR.JS" version="2.1.1" targetFramework="net45" />
  <package id="Microsoft.AspNet.SignalR.SystemWeb" version="2.1.1" targetFramework="net45" />
  <package id="Microsoft.AspNet.WebPages" version="3.2.0" targetFramework="net45" />
  <package id="Microsoft.Bcl" version="1.1.9" targetFramework="net45" />
  <package id="Microsoft.Bcl.Build" version="1.0.14" targetFramework="net45" />
  <package id="Microsoft.Net.Http" version="2.2.22" targetFramework="net45" />
  <package id="Microsoft.Owin" version="2.0.1" targetFramework="net45" />
  <package id="Microsoft.Owin.Host.SystemWeb" version="2.0.1" targetFramework="net45" />
  <package id="Microsoft.Owin.Security" version="2.0.1" targetFramework="net45" />
  <package id="Microsoft.SqlServer.Compact" version="4.0.8854.1" targetFramework="net45" />
  <package id="Microsoft.Web.Infrastructure" version="1.0.0.0" targetFramework="net45" />
  <package id="Modernizr" version="2.8.3" targetFramework="net45" />
  <package id="mongocsharpdriver" version="1.9.2" targetFramework="net45" />
  <package id="Moq" version="4.2.1408.0717" targetFramework="net45" />
  <package id="MvvmLight" version="4.2.30.0" targetFramework="net45" />
  <package id="MvvmLightLibs" version="4.3.31.1" targetFramework="net45" />
  <package id="Nancy" version="0.23.2" targetFramework="net45" />
  <package id="Newtonsoft.Json" version="6.0.4" targetFramework="net45" />
  <package id="Newtonsoft.Json" version="5.0.4" targetFramework="net45" />
  <package id="Ninject" version="3.2.2.0" targetFramework="net45" />
  <package id="NUnit" version="2.6.3" targetFramework="net45" />
  <package id="Owin" version="1.0" targetFramework="net45" />
  <package id="PagedList" version="1.17.0.0" targetFramework="net45" />
  <package id="PostSharp" version="3.1.49" targetFramework="net45" />
  <package id="protobuf-net" version="2.0.0.668" targetFramework="net45" />
  <package id="Quartz" version="2.2.4" targetFramework="net45" />
  <package id="RestSharp" version="104.4.0" targetFramework="net45" />
  <package id="SharpZipLib" version="0.86.0" targetFramework="net45" />
  <package id="sqlite-net" version="1.0.8" targetFramework="net45" />
  <package id="StackExchange.Redis" version="1.0.328" targetFramework="net45" />
  <package id="Unity" version="3.5.1404.0" targetFramework="net45" />
  <package id="WebGrease" version="1.6.0" targetFramework="net45" />
</packages>

At the moment, I'm not able to update our server. I do it later.

I will also test with your refernced packages in a test project.

Thanks a lot.