Performance docs/case studies

Guest

Hi there,

We are currently evaluating ProGet to see how it can compliment our development lifecycle.
Are there any documents/case studies available which describe performance and capacity capabilities of ProGet server - i.e. number of concurrent requests/connections it can serve, number of packages it can host etc.

Thanks
Igor

Product: ProGet
Version: 5.0.8

Guest

Keep in mind that performance metrics like "number of concurrent requests/connections it can serve, number of packages it can host" are basically like asking "how many songs can an iPod hold"?

Hardware plays a big role (e.g. you can't store more packages than you have disk space, you can't have more requests than you have bandwidth, etc). But even if you could guess, there's no useful metric to even consider, because you don't what your future load will be like.

What's more important to consider is that, ProGet is highly optimized for performance thanks to years of being widely used in the field, and the software can (and has been) scaled to dozens of servers, on demand, in a single instance.

Guest

Respectfully disagree.
A 16G iPod can hold 4,000 songs (figure from the air), whilst 32G version - twice the number. MS SqlExpress used to support up to 10 concurrent connection etc...

Underlying infrastructure certainly plays a big part in performance metrics, but the software has its own limits, and those I am interested in.
E.g. can ProGet support hundreds of concurrent connections? Thousands?
Have you done any performance or stress tests?

In our environments we are pushing boundaries and limits and we have switched from some big brand name products to alternatives in CI space because they could not perform satisfactory under the generated load.

Guest

Ah, I see.

Well in the case of ProGet, the underlying infrastructure is definitely the most important aspect when it comes to performance, and there are no such limits in the ProGet software.

This is because ProGet is built on .NET to use IIS and SQL Server, so there's actually not much that ProGet even has control over with regards to things like concurrent connections. That's an application pool setting. Your disk space controls how many packages you can store, etc.

Of course we have tested with millions of packages and hundreds of feeds, but that's not a very useful test. It just makes sure our database indexes and the like are OK.

The second most important performance aspect is your configuration. If you "chain" a ton of connectors together (therefore making repository queries expensive), then it's going to require more hardware resources.

Anecdotally, we have heard customers say they switched to ProGet because it performed significantly better than the competitors. But I really don't know why, and to be totally honest, I don't think it would be a "fair" case study.

This is because we've seen very strange things causing performance problems. For example, a bad wireless access point used by a single dev group choked up the load balancer in front of ProGet.. I have no idea how they discovered it, but they said fixing that WAP fixed all performance problems.

gabe_7487

Bumping this because I have the same question and I feel it wasn't actually answered. 3 questions.

What metrics can I monitor to know that my proget installation is performing or not performing
What are benchmarks that Indeo considers to be "nominal" -- eg 'a properly configured proget instance, with 20k npm packages cached on a 2 core instance with 8gb of ram and a 400 dtu azure sql database can send 150 packages/sec'
Proget is noticably slower than plain old NPM. I'm sure part of that is just infrastructure related, by I would consider our instance to be fairly well provisioned. When my developers complain, what are basic troubleshooting steps I can take to locate the bottleneck?

atripp

Hello;

Your best metric is going to be your users; if they're complaining it's slow, then it's probably too slow. My guess is that it's intermittant (during peak times).

But in any case, it's not about the number of packages (a micro server can easily serve millions of packages nearly instantly), it's about the number of simultaneous connections (requests) to the server, and the features you've enabled (like license checking, etc).

It sounds like you've got a single "2 core instance with 8gb of ram" server; while you can try to increase the hardware, the bottleneck is most likely network related; a single server might not just be enough to keep up with the traffic from your developers. Keep in mind that "plain old NPM" (i.e. registry.npmjs.org) runs on a massive, dedicated data center and is heavily cached (basically read-only).

The ProGet Free to ProGet Enterprise article contains some general guidance that might help answer your questions:

High Performance: 1 Server per 50 Developers. The network never slows down, never crashes.
Average Performance: 1 Server per 100 Developers. The network slows down and drops productivity 10% on average during peak times daily.
Acceptable Performance: 1 Server per 200 Developers. The network slows down and drops productivity 10-30% during peak times daily.
Unacceptable Performance: 1 Server per 250+ Developers. The network is unstable and crashes.

You can try disabling connectors, disabling license checking, disabling vulnerability scanning -- but those are really important features, so your best bet will be to plan for a load-balanced scenario.

See How to Prevent Server Overload in ProGet to learn more.