Welcome to the Inedo Forums! Check out the Forums Guide for help getting started.
If you are experiencing any issues with the forum software, please visit the Contact Form on our website and let us know!
ProGet and MSSQL license
-
Hello,
I am interested in what situations occur when ProGet Basic (without Otter and Buildmaster) does not have enough MSSQL Experess and we need a different license (CPU/Mem/Disk requirement).
With how many Feeds and their packages did you have to abandon MSSQL Experess and choose a more complete database licensing model?
Please share your cases.
-
Hi @w-repinski_1472 ,
Even with heavy usage, SQL Server Express will be fine; performance issues from heavy usage will come at the network level, and that's solved by configuring ProGet as a High Availability & Load Balanced Cluster.
We've seen a few 10GB+ ProGet databases in the field, and 95% of the space is taken up with years of individual package download records stored in
PackageDownloadstable. Those are obviously okay to purge (or not enable at all).Otherwise, to move to a different SQL Server, just restore a database backup to the new server and update the connection string in ProGet.
Best,
Alana
-
Hello,
The answer is not completely satisfactory for me.
I assume that it is as you wrote on the forum and my concerns only result from the lack of information on how you communicate with MSSQL and what exactly you store in it.
I need to understand all this in more detail, I have many questions from my business.
We plan for ProGet to become a mirror for thousands of packages, we use RPM, DEB, Docker, Universal, npm, nuget. There will be definitely fewer of our own packages and we will work on them (about 300 packages, each with multiple release versions), mainly docker/universal and here there will be a lot of activities related to CI/CD pipelines (Jenkins). We build several versions of packages a day, mainly for testing. Periodically 1/month, there are releases (group of packages about 50). We need all statistical information for our packages. Preferably for a period of 2-3 years. In terms of mirrored packages, we are only interested in information about security vulnerabilities and where they are further used.
In your information you state that network connections are the bottleneck. I don't understand this completely in times when we have 100G cards, maybe I don't understand the scale on which ProGet is used in other companies.
I would rather point out the performance limitations of MSSQL Express due to the supported number of CPU cores and RAM. (I assume that the number of connections is not an issue due to the use of pool connections).
Can you explain this to me in more detail?
-
Hi @w-repinski_1472 ,
Based on your initial usage, I think SQL Server Express will suffice. The ProGet database is basically a package metadata index, and essentially stores things like package name, version number, and a variety of things including the manifest file (.nuspec, package.json, etc). It's maybe a few kb per package, and you'll need 100k's of packages to even reach 1GB of metadata storage.
In your information you state that network connections are the bottleneck. I don't understand this completely in times when we have 100G cards, maybe I don't understand the scale on which ProGet is used in other companies.
The issue is with the number of connections, and a single server struggling with 100's of expensive queries/requests per second. Running "nuget restore" or "npm restore" will hammer the repository with 1000's of simultaneous requests, and many of those need to go to nuget.org or npm.json to be resolved. When you have multiple users and multiple build servers running these kinds of restores, then you run into load issues.
At about 50 users, a load-balanced / high-availability cluster starts to make sense. After 250 users, sticking to just a single server doesn't make a lot of sense (cost of downtime is expensive). Once you need a server cluster, then upgrading SQL Server would probably make sense.
There's a big cost difference between a single server and a server cluster - in part the ProGet licensing fees, but also managing a server cluster is more complicated. Some organizations prefer to start with high-availability right away rather than worry about upgrading later.
hope that helps clarify!
Best,
Steve
