hi

npm feed + nodejs.org connector: huge memory footprint and npm install ECONNRESETs

we see a huge memory footprint on our proget installation when populating the npm feed + connector cache. once populated we see frequent ECONNRESET during 'npm ci' runs.

for resource footprint reason we switched to verdaccio to serve as our npm cache. though, we also see ECONNRESETs there.

some specs:
1 - during cache population we have a 4 cpu + 30 gb ecs service with 3 tasks hosting nginx, proget and mssql
2 - once the cache is populated we have a 2 cpu + 5 gb ecs service (same tasks)
3 - package.json resolve to 1500-2500 dependencies which hold about 1.5-2gb on node_modules
4 - 4-5 simultaneous 'npm ci' processes run against the ecs cluster (with local cache cleaned and removed node_modules directory)
5 - to compare - the same cache population works on a verdaccio ecs service (2 tasks hosting nginx, verdaccio) with 2gb ram (ignore the fact it does not need mssql - on proget, mssql plays a minimal role on resources footprint)

is the footprint expected ? how to overcome the ECONNRESET issue ?

any ideas ?

thank you

we experience mem usage on fresh filling of cache (20gb) with npm connector attached to a feed. once the cache is filled with all the dependencies usage is down to 2gb.

hello

i am facing the same issue

using Web.ConcurrentRequestLimit does not work as i dont find the setting

using nginx rate limiting which is in front of proget

http {
  limit_req_zone $server_name zone=perServer:10m rate=100r/s;
...

  upstream proget {
    server 127.0.0.1:8081;
  }
  server {
    location / {
      # npm needs some rate limiting
      limit_req zone=perServer burst=10; 
...
      proxy_pass http://proget;
...
    }
  }
}