This is a results of network connections and traffic, and the message is coming from IIS or the ASP.NET hosting components. The server hardware itself is fine, but there's not enough network bandwidth to handle the load.
Keep in mind that your clients (NuGet, npm) will make hundreds (sometimes thousands) of simultaneous requests to the ProGet server to just a single "package restore". These requests are "expensive" and time-consuming, because often they'll need to check with a remote connector (which opens a new connection), backend database (network connection), validate security, etc. In addition, the workstations are typically more powerful than the server, so they will have more resources to make these requests.
So it really only takes a handful of users doing simultaneous package restores, and you are basically at a "denial of service" level of traffic. So, in this case, load-balancing is the way to do it.