Posts made by rhessinger

Hi @nicholas-boltralik_3634,

What version of ProGet are you running?

Thanks,
Rich

Hi @dusbn,

I think you are looking to use the Drop Path. This would allow you have ProGet monitor a folder and automatically upload a package when it exists in that folder.

Alternatively you could use a PowerShell script to iterate through each nupkg and use nuget push to push each package.

Thanks,
Rich

Hi @scroak_6473,

You most likely have to setup a Resource Credential. On the Manage User Directories page, you should see a button for AD Credentials. Create a credential that has domain access. From there, in your domains to search, enter a value of {domain},{Secure Resource Name} ex: exampldomain.com,DomainAdmin. Can you give that a try and see if it connects?

Thanks,
Rich

Hi @mcascone ,

We have fixed the issue and the published date will now update when the package is overwritten. This will be released in ProGet 5.3.9 which is due out this Friday Augst 14, 2020.

Thanks,
Rich

Hi @markus4830,

I'm definitely sorry about this. The change was made to help to aide in improvements to other areas of the system related to NuGet. Unfortunately, it looks like it affected the NuGet API. Expect a more permanent solution in the near future.

Thanks,
Rich

Hi @thomas-ibel_2956,

I am not aware of any other changes that would have affected that specifically. There were a couple more changes to other areas of the system. It is always possible that there was an update to a Debian package that mono depends on that helped to improve connection handling in Docker. But I cannot confirm that was the case. The only thing I can confirm is that the top two layers in the image have a slight difference to them when comparing 5.3.8 and 5.3.8-ci.20 .

Thanks,
Rich

Hi @Jonathan-Engstrom ,

When you click your name in the upper right corner, go to Edit Profile. On the left you should see a button that says Change Email. Do you see a button there?

Thanks,
Rich

Hi @scroak_6473,

I created a branch and a blank page on GitHub for Docker Swarm. You should be able to add your notes and examples there. Please let me know if you have any issues!

Thanks,
Rich

Hi @thomas-ibel_2956,

Glad to hear it! Please let me know if this continues or if you start having issues again.

Thanks,
Rich

Hi All,

We have just released ProGet 5.3.8. Would you please upgrade and test with the Close Database Connections Early setting enabled?

Thanks,
Rich

Hi @jyip_5228 ,

We just released ProGet 5.3.8, which includes the Close Database Connections Early option in the advanced settings. Could you please upgrade and give this a try?

Thanks,
Rich

Hi @pluskal_4199,

Thanks for the information.

The FeedCleanup job is automatically scheduled and you can see the date and time by navigating to Administration -> Scheduled Jobs. Would you be able to tell me if the time that was automatically configured matches when you are seeing the timeout?

Our Docker image currently is running ProGet using mono which comes with its own subset of issues. I'm seeing the SNI_Error is a pretty common error with mono and we have seen mono has more trouble releasing connections than when running within the .NET Framework. The most annoying part of mono, is they don't always give us the most accurate and descriptive error messages. This could just be a similar exception to the max connection pool but based on DB transactions. We are currently in the process of migrating to .NET 5 and we will hopefully have an updated Docker image over the next couple of months. If you are interested, please keep an eye out for more information to come.

I would like to see some of these results on Windows-based installations also to help guide our solution.

Thanks,
Rich

Hi @pluskal_4199 ,

Are these builds running at the same time as the Feed Cleanup job for your NuGet feed by chance? Also, are you seeng the same error The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.?

Thanks,
Rich

Hi @jyip_5228,

Just an update! We will be shipping a potential fix in PG-1783, which adds a new checkbox in advanced settings (unchecked by default):

Close Database Connections Early
EXPERIMENTAL (for debugging/testing only) - As of ProGet 5.3, database connections are left open during the lifecycle of a NuGet API request as a means to reduce overhead; however, this may be causing ProGet to run out of available connections to SQL Server. Set this value true to open/close database connections as needed on NuGet feeds.

We'll update when this is shipped --- but if we can get some folks to verify that this works better (we can't repro, at all ), then we will likely make it the default. Hopefully this will do it. Seems better than raising connection pool limits

If so, then the savings in connection open/close overhead don't seem to make it particularly worthwhile. This "keep open" technique made a ton of differences elsewhere in our software, but since a NuGet API requests may yield a ton of other network requests (via connectors) and block, the pool may be getting drained too quickly...

just a theory, as I mentioned. Anyway hope this helps, stay tuned!!

Thanks!

Hi @pluskal_4199 ,

That is great to hear! How often were you seeing the error prior? Also what OS Version, SQL Server Edition, and SQL Server Version are you running?

Thanks,
Rich

Hi @christian_panten,

Are there any errors showing in the Diagnostics center?

Thanks,
Rich

Hi @christian_panten,

Can you please send over the error you are seeing when running nuget push? As long as your -Source is the NuGet API URL, then you shouldn't need to do anything special.

Thanks,
Rich

Hi @christian_panten_6482,

You would just push the snupkg the same way, but only push the snupkg. For example:

nuget push -source sources-feed MyPackage.1.2.0.snupkg

I also wanted to point out this from our documentation as well:

To summarize, push only the symbol package to your ProGet feed if you intend to use the symbol server. To prevent symbols from being downloaded with the NuGet package, see the Strip symbol files from packages downloaded from this feed option on the Manage Feed page under Symbol Server settings.

Thanks,
Rich

Hello @jyip_5228,

I'm having trouble recreating this locally. If my memory servers me right, you are currently running ProGet in Docker, is that correct? I will attempt to spin-up the Docker-based version of ProGet locally to attempt to recreate this issue. I do wonder if this is related to your database connection pool errors. Do you see any errors in the Diagnostics Center under Administration?

Thanks,
Rich

Hi @christian_panten_6482,

Since you are using SourceLink, you should be able to use the Symbol Server built into ProGet as long as you have ProGet running on Windows. For information on setting up a symbol server and pushing packages to it, please review our NuGet Symbol and Source Server documentation.

Hope this helps!

Thanks,
Rich

Hi @jyip_5228,

Thanks for the update. I'm currently implementing a potential fix to the connection stuff. I'll let you know when I have something ready.

Thanks,
Rich

Hi @jyip_5228,

Have you seen this issue with any other packages? Or is this just MethodTimer.Fody. Also, do you have multiple versions of MethodTimer.Fody installed on this machine?

Thanks,
Rich

Hi @jyip_5228,

Can you try adding the -NoCache parameter?

ex: C:\>nuget install MethodTimer.Fody -Version 3.1.2 -Source https://xxx/nuget/ks-nuget/v3/index.json -NoCache

Also, could you try clearing the local NuGet file caches by running nuget locals all -clear?

Can you please let me know if the install command works after either of those options.

Thanks,
Rich

Hi @jyip_5228,

Thanks for giving me the updated counts. Glad this workaround is working for you. We are still working on the issue internally. Do you happen to notice any other errors in the logs right before the timeout happens?

Thanks,
Rich

Hi @jyip_5228,

The default setting for SQL Server 2017 is unlimited. As long as you didnt set one on your SQL Server, then you should be good.

Thanks,
Rich

Hello @mcascone,

Thanks for sending this over to me. So there looks to be an issue with the universal feed not updating the published date when overwriting a package. I happened to test 3 other feed types and it worked fine. I have created an internal ticket to fix this and I will update you as soon as it is scheduled for release.

Thanks,
Rich

Hi @jyip_5228,

The last exception you are seeing is currently a known issue that we are currently working through. We have noticed this tends to affect the docker based ProGet installations the most. In most cases, increasing the max connection pool limit in the SQL connection string seems to fix the issue. Depending on your SQL Server setup, you may need to increase the allowed connections on SQL Server as well. Can you please try to increase the max connection pool in your SQL connection string? Also, can you please tell us what version and edition of SQL Server you are currently running?

Thanks,
Rich

Hi @Jonathan-Engstrom,

Have you reviewed the Backing up Otter documentation? We outline how to backup and restore a ProGet instance in there. If you are missing configurations, it shoulds like you may not have copied all of the files from your git repository or the configurations were not stored in the Git raft. Do you still have access to your old server? You could also navigate to Administration -> Rafts and download your Git raft as a zip file as well.

Thanks,
Rich

Hi @Jonathan-Engstrom,

You could transfer to a file-based raft. In that case, you can follow the same directions for migrating to a Git-based raft and just treat folder you configured for the file-based raft as your target. Instead of exporting your files from Otter, you could just check out the Git repository and copy the files to the folder you configured for the file-based raft (leave out the .git when you copy the files). Would that work for you?

Thanks,
Rich

Hi @philippe-camelio_3885,

I did a little more testing internally and it looks like Global GitLab Secure Resources seems to work for me in 1.7.6-RC.2. I only have the ability currently to test with hosted GitLab. I'm guessing the API key you are using does not have access to that project to list branches and/or projects.

When you are creating the secure resource, can you add the Secure Credential then click on the application, and does it show a list of your GitLab projects?

Thanks,
Rich

Hi @mcascone,

Can you please tell me what feed type you are using? Could you also include an example of the API request you are using? When I add the package manually from the UI, I noticed that the published date does change. If this is not happening with the API call, then I will need to create an internal ticket with steps to recreate it.

Thanks,
Rich

Hi @thomas-ibel_2956 & @blake-peno_2353,

Do you have ProGet using IIS or are you using the integrated web server?

Thanks,
Rich

Hi @thomas-ibel_2956,

Thanks for the updated information. I know this is not an ideal solution, but can you up the Max Pool Size to 1,000? We are still looking into what is preventing the connections from being released back to the connection pool, but this may work as a temporary workaround while we are in that process of resolving this.

Thanks,
Rich

Hi @philippe-camelio_3885,

Thanks for the information. Let me investigate this a little further on global resources.

Thanks,
Rich

Hi @mcascone_8142,

It is possible that this was fixed with another issue between 5.3.0 and 5.3.7. I would recommend updating 5.3.7 and testing it again. If it still happens, it would have to be a difference in tasks or how the user is configured that the API is impersonating. You could try to not impersonate a user for that API Key. The API would then run as an administrator.

Thanks,
Rich

Hi @scroak_6473,

This works as designed. Clair attaches a vulnerability at the Docker image layer, not at the image itself. This also means that that vulnerability can exist across multiple images because Docker will share layers between images.

Thanks,
Rich

Hi @philippe-camelio_3885,

I have identified the issue and created an internal ticket to fix this. I will reply to this thread as soon as we have scheduled this for release.

Thanks,
Rich

Hi @philippe-camelio_3885,

Does it work if you use a Secure Resource that exists at the application level instead of the global level?

Thanks,
Rich

Hi @thomas-ibel_2956,

We are currently investigating this issue further. As a workaround, could you try increasing the Max Pool Size to 400 in your SQL connection string? Please let us know if that changes anything for you.

Thanks,
Rich

Hi @scroak_6473,

Glad to hear it!

Thanks,
Rich

Hi @gravufo,

Awesome! Just reply back here if it does start up again.

Thanks,
Rich

Hi @mcascone_8142,

What version of ProGet are you running? When I tested it in 5.3.6, I noticed the Published Date on the metadata tab changed to today.

Thanks,
Rich

Hi @gravufo,

No problem. It does seem weird that >50 builds would cause that. Unless if it is the SQL server that is getting overwhelmed at that time.

Thanks,
Rich

Hi @mcascone_8142,

Out of the box, the administrator rights have access to overwrite a package. The reason you are not seeing an error is most likely because the package was successfully overwritten. You have two options, you can either remove the overwrite privilege from the Administer task or you could have the API key impersonate a user with a lower set of rights. Here is some documentation on how to create and customize tasks in ProGet.

Thanks,
Rich

Hi @gravufo,

Can you please check your scheduled jobs in the Administration page of ProGet. It could be possible all of the jobs are running at 4:00 am.

Thanks,
Rich

Hi @mcascone_8142.,

Is the user that is pushing the packages an administrator?

Thanks,
Rich

Hi @zion-b_8257,

Currently, there is no simple way built into ProGet to bulk upload Maven artifacts to ProGet. The only thing I can suggest currently is to write a script that uses the Maven CLI to publish the artifacts to ProGet.

If these are from another Maven feed, you could create a connector in ProGet and use the Package Promotion API to pull and promote those packages to a feed. The Package Promotion API does require ProGet Basic or Enterprise though. It could also be possible to pull these packages to ProGet using the Native API, but I do not have an example of how to do that.

Thanks,
Rich

Hi @gravufo,

There is nothing that changed in the NuGet API that would cause SQL connection issues. We did add the ability to enable the NuGet v3 API in addition to the V2 API, but you would have to enable V3 manually for your existing feeds. Still, that does not affect open a connection to SQL. What is weird to me is that this happens every day at the same time. This screams like there is something scheduled. Can you check your Application Pool in IIS and verify you don't have an application pool restart happening at that time?

Thanks,
Rich

Hi @gravufo,

No problem, please let me know either way. Thanks!

Thanks,
Rich

Hi @gravufo,

Could you try increasing the Max Pool Size to 300 in your SQL connection string? I would like to see if this is an issue with the number of SQL connections.

Thanks,
Rich