ProGet: Uploading Larger Files

Via the NuGet command line, I'm attempting to upload a ~350 MB file to ProGet and either get a 404 error or Out of Memory Exception. I narrowed the problem down to files around ~20 MB being OK, but failing at ~40 MB.

I thought this might be a problem with NuGet.exe, but I cannot upload greater than ~40 MB files directly in the ProGet Web interface either. I've tried pushing with NuGet 2.2.1, 2.2.5 alpha, and the ProGet command line tools to no avail.

Does ProGet support uploading large files? I'm trying to construct a build pipeline using NuGet as the repository and need the 350 MB file to accomplish this goal.

Thanks - below are the errors. I couldn't seem to locate any related logs on my ProGet server.

==============================================

(Occurs sometimes - I didn't nail down the pattern)

System.InvalidOperationException: Failed to process request. 'Not Found'.
The remote server returned an error: (404) Not Found.. ---> System.Net.WebException: The remote server returned an error: (404) Not Found.
at System.Net.HttpWebRequest.GetResponse()
...

(Occurs sometimes but seemingly less frequent than the 404 - I didn't nail down the pattern)

System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
at System.IO.MemoryStream.set_Capacity(Int32 value)
...

(From a Manual Upload)

404 - File or directory not found.

The resource you are looking for might have been removed, had its name changed, or is temporarily unavailable.

Found the answer after some more Googling...

Apparently the size limit is set in IIS, but the ProGet site can be updated to allow larger file sizes. I added the following to ProGet's web.config:

<system.webServer>
        <security>
            <requestFiltering>
                <requestLimits maxAllowedContentLength="524288000"/>
            </requestFiltering>
        </security>
</system.webServer>

which allows 500MB uploads. Upload works like a champ now.

The requestlimits value didn't exist in web.config. Perhaps as a feature request, would it be possible to add this and have it available as a setting in the Administration console?

Thank you for the detailed analysis and solution. We will look for a way to put that in the ProGet configuration, mainly because the web.config will be overwritten if you upgrade the software.

Curious, what sort of files are in the packages that increase its size to over 300 MB? Is this a build artifact of sorts? If so, have you seen our other tool, BuildMaster? It's designed to handle artifacts and deployments of that size much more gracefully :)

Hi Karl,

Yes I've made sure to document that workaround, because I expect to lose it on the next upgrade. :-)

Yeah, the big nupkg is because we're scooping up a whole product's set of binaries, etc. that we're now sharing with other dependent builds and potentially other groups in remote locations that won't be using our build server. I am considering the internal NuGet repo to be that sharing mechanism, because we'll have that anyway for the smaller, assembly references in Visual Studio. My file is big for a couple reasons :-), but I foresee that getting smaller over time.

I've seen your BuildMaster pages here, but we've got some investment in our current solution, so I didn't inspect things in too much detail. I'll check out how build artifact sharing works in that.

Thanks

Sounds good, by early next week we'll have some very specific documents that include exact deployment plans showing how we personally handle some cases with dependencies (specifically, how the BuildMaster installer deployment is automated, even though it depends on 4 separately-built BuildMaster extensions and BuildMaster itself).

In the meantime, the feature as we've categorized it is Dependency Management: http://inedo.com/buildmaster/features/dependency-management