Posts made by jyip_5228

Found this.

An error occurred processing a PUT request to http://xxxx/nuget/ks-nuget/: Synchronous operations are disallowed. Call ReadAsync or set AllowSynchronousIO to true instead.
 Details: System.InvalidOperationException: Synchronous operations are disallowed. Call ReadAsync or set AllowSynchronousIO to true instead.
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpRequestStream.Read(Byte[] buffer, Int32 offset, Int32 count)
at Microsoft.AspNetCore.WebUtilities.FileBufferingReadStream.Read(Byte[] buffer, Int32 offset, Int32 count)
at System.IO.Stream.ReadByte()
at Inedo.IO.StreamSearcher.FindNext(Stream stream)
at Inedo.IO.StreamSearcher.FindAll(Stream stream)+MoveNext()
at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
at Inedo.Web.HttpRequestExtensions.GetPostedFilesInternal(HttpContext context)
at Inedo.Web.HttpRequestExtensions.GetPostedFiles(HttpContext context)
at Inedo.ProGet.WebApplication.FeedEndpoints.NuGet.NuGetApi.PutHandler.ProcessPutRequestAsync(HttpContext context, RequestData urlData, NuGetFeed feed) in C:\InedoAgent\BuildMasterTemp\192.168.44.60\Temp\_E55466\Src\ProGet.WebApplication\FeedEndpoints\NuGet\NuGetApi.PutHandler.cs:line 42
at Inedo.ProGet.WebApplication.FeedEndpoints.NuGet.NuGetApi.ProcessRequestAsync(HttpContext context, NuGetFeed feed, String relativeUrl) in C:\InedoAgent\BuildMasterTemp\192.168.44.60\Temp\_E55466\Src\ProGet.WebApplication\FeedEndpoints\NuGet\NuGetApi.cs:line 37
at Inedo.ProGet.WebApplication.FeedEndpoints.FeedEndpointHandler.FeedRequestHandler.ProcessRequestAsync(HttpContext context) in C:\InedoAgent\BuildMasterTemp\192.168.44.60\Temp\_E55466\Src\ProGet.WebApplication\FeedEndpoints\FeedEndpointHandler.cs:line 192

Hi,
Switched to progetcore due to a possible fix of DB timeouts. however found the following problem when trying to push packages via nuget.

[Exec] "C:\Txx\tools\NuGet.CommandLine.4.7.1\tools\Nuget.exe" push "C:\xxx.nupkg" -Source "https://xxxxxx/nuget/ks-nuget/" -ApiKey "xxxxxxxx"
[14:58:16][Exec] Pushing xxx.nupkg to 'https://xxxxxx/nuget/ks-nuget/'...
[14:58:16][Exec]   PUT https://xxxxxx/nuget/ks-nuget/
[14:58:16][Exec]   InternalServerError https://xxxxxx/nuget/ks-nuget/ 282ms
[14:58:16][Exec]   PUT https://xxxxxx/nuget/ks-nuget/
[14:58:17][Exec]   InternalServerError https://axxxxxx/nuget/ks-nuget/ 171ms
[14:58:17][Exec]   PUT https://xxxxxx/nuget/ks-nuget/
[14:58:17][Exec]   InternalServerError https://xxxxxx/nuget/ks-nuget/ 168ms
[14:58:17][Exec] Response status code does not indicate success: 500 (Internal Server Error).

Reverting back to the regular proget:5.3.10 works perfectly.

I havent tracked it, but i restarted container on Monday, by Tuesday morning when people returned to office/ started running builds, I started seeing Timeout errors. I will switch to the core version and see how that goes.

Hi I'm running 5.3.10 and getting timeouts again / max pool reached, is the database issue occurring again?

An error occurred in the web application: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
 Details: URL: http://xxx/nuget/ks-nuget/v3/flatcontainer/xxx/xxx/xxxxx.nupkg
Referrer: (not set)
User: (unknown)
User Agent: NuGet Desktop MSBuild Task/5.5.0 (Microsoft Windows NT 10.0.18363.0)
Stack trace: at System.Data.ProviderBase.DbConnectionFactory.TryGetConnection (System.Data.Common.DbConnection owningConnection, System.Threading.Tasks.TaskCompletionSource`1[TResult] retry, System.Data.Common.DbConnectionOptions userOptions, System.Data.ProviderBase.DbConnectionInternal oldConnection, System.Data.ProviderBase.DbConnectionInternal& connection) [0x0024d] in <0864334e7e474248b37afca9b637daa9>:0
at System.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal (System.Data.Common.DbConnection outerConnection, System.Data.ProviderBase.DbConnectionFactory connectionFactory, System.Threading.Tasks.TaskCompletionSource`1[TResult] retry, System.Data.Common.DbConnectionOptions userOptions) [0x00036] in <0864334e7e474248b37afca9b637daa9>:0
at System.Data.ProviderBase.DbConnectionClosed.TryOpenConnection (System.Data.Common.DbConnection outerConnection, System.Data.ProviderBase.DbConnectionFactory connectionFactory, System.Threading.Tasks.TaskCompletionSource`1[TResult] retry, System.Data.Common.DbConnectionOptions userOptions) [0x00000] in <0864334e7e474248b37afca9b637daa9>:0
at System.Data.SqlClient.SqlConnection.TryOpen (System.Threading.Tasks.TaskCompletionSource`1[TResult] retry) [0x0005d] in <0864334e7e474248b37afca9b637daa9>:0
at System.Data.SqlClient.SqlConnection.Open () [0x0003b] in <0864334e7e474248b37afca9b637daa9>:0
at Inedo.Data.SqlServerDatabaseContext.CreateConnection () [0x0006c] in <8a0468ee94a94ef1b2bba97871f29990>:0
at Inedo.Data.DatabaseContext.ExecuteInternal (System.String storedProcName, Inedo.Data.GenericDbParameter[] parameters) [0x00064] in <8a0468ee94a94ef1b2bba97871f29990>:0
at Inedo.Data.SqlServerDatabaseContext.ExecuteInternal (System.String storedProcName, Inedo.Data.GenericDbParameter[] parameters) [0x00005] in <8a0468ee94a94ef1b2bba97871f29990>:0
at Inedo.Data.DatabaseContext+<>c__DisplayClass29_0`1[TRow].<EnumerateTable>b__0 () [0x00000] in <8a0468ee94a94ef1b2bba97871f29990>:0
at Inedo.Data.StrongDataReader+<Read>d__11`1[TRow].MoveNext () [0x0003e] in <8a0468ee94a94ef1b2bba97871f29990>:0
at System.Linq.Enumerable.TryGetFirst[TSource] (System.Collections.Generic.IEnumerable`1[T] source, System.Boolean& found) [0x00045] in <22384ee444974b39bb55b725de39c721>:0
at System.Linq.Enumerable.FirstOrDefault[TSource] (System.Collections.Generic.IEnumerable`1[T] source) [0x00000] in <22384ee444974b39bb55b725de39c721>:0
at Inedo.ProGet.Data.DB+Context.Feeds_GetFeed (System.Nullable`1[T] Feed_Id, System.String Feed_Name) [0x0003d] in <888866fba54f44d68ecb60a2adf95719>:0
at Inedo.ProGet.Feeds.Feed.GetFeed (System.String feedName, Inedo.ProGet.Data.DB+Context externalDbContext) [0x0000b] in <888866fba54f44d68ecb60a2adf95719>:0
at Inedo.ProGet.WebApplication.FeedEndpoints.FeedEndpointHandler.GetHttpHandler (Inedo.Web.Handlers.GetHandlerParams args) [0x0017e] in <d560b10d729641519d596b0266d076a4>:0
at Inedo.Web.Handlers.DynamicHttpHandling.GetHandler (System.Web.HttpContext context, System.String requestType, System.String url, System.String pathTranslated) [0x0003b] in <8a0468ee94a94ef1b2bba97871f29990>:0
at Inedo.Web.InedoHttpModule.MapHandlerAndBeginRequestAsync (System.Web.HttpApplication app) [0x00029] in <8a0468ee94a94ef1b2bba97871f29990>:0
at Inedo.Web.InedoHttpModule.HandleBeginRequestInternalAsync (System.Web.HttpApplication app) [0x0000f] in <8a0468ee94a94ef1b2bba97871f29990>:0
at Inedo.Web.InedoHttpModule.ProcessBegin (System.Object sender, System.EventArgs e, System.AsyncCallback cb, System.Object extraData) [0x0000d] in <8a0468ee94a94ef1b2bba97871f29990>:0
at System.Web.HttpApplication+<RunHooks>d__217.MoveNext () [0x000a4] in <b4f0b153c02f4f0588d3f7549d75281b>:0
at System.Web.HttpApplication+<Pipeline>d__225.MoveNext () [0x0012c] in <b4f0b153c02f4f0588d3f7549d75281b>:0
at System.Web.HttpApplication.Tick () [0x00000] in <b4f0b153c02f4f0588d3f7549d75281b>:0

still looking forward to this in 5.3.11, but wanted to let you know release 5.3.10 had a fixed a bug that was caching unintentionally. Since these packages I do not intend to cache, the case sensitivity issue is not affecting me for now.

this is excellent news, i am looking forward to this fix.

thank you.

nuget version 4.7.1.5393

are there any other ideas to fix this?
new packages are going into the wrong folder and i have to copy it to the .Wpf one from .WPF
new package:

C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\Common7\IDE\CommonExtensions\Microsoft\NuGet\NuGet.targets(114, 5): The feed 'https://xxx/nuget/ks-nuget/v3/index.json [https://xxx/nuget/ks-nuget/v3/index.json]' lists package 'Syncfusion.Grid.WPF.18.2.0.54' but multiple attempts to download the nupkg have failed. The feed is either invalid or required packages were removed while the current operation was in progress. Verify the package exists on the feed and try again.
  Unable to find package 'Syncfusion.Grid.WPF.18.2.0.54'.

Running this command every time it breaks something cause a new package is not feasible.
Technically the package shouldnt be caching, so will PG-1804 fix this?

@atripp
So there is no trace from Proget side where it receives the request and see how it responds?

Also for the stacktrace @rhessinger mentioned should it be v2, when I'm using v3 ?

i completely agree that it shouldnt be using v2 api.. and i dont know why cause it is pointed to the v3 url.

I have only seen issues with remote packages. The clear command didn't do anything. Is there a way to find logs on why it failed and today it worked? Need this to work reliably.

Also seems like the setting for no cache does not work.

@rhessinger

This issue appears to still exist occasionally, is there a way to increase logging to try and troubleshoot this?

error NU1101: Unable to find package Microsoft.Office.Interop.Word. No packages exist with this id in source(s): https://xxx/nuget/ks-nuget/v3/index.json

Clearly searching via UI

I sometimes when it feels like it would work after running the build job a few times, this happened to the package 'automapper' eventually it worked.

Please assist.

hi, it looks good so far, will see if it helps other issues as well.

Did a quick look in our packages directory, looks like Syncfusion Inc has done this to other packages as well with the case rename, got 10+ packages that have different cases :(

I want to add that after the manual change, when I did a reindex it seems like Proget detected it as an invalid package and removed it (i checked the delete option)

What are the chances of storing everything in lowercase getting added? Can the same fix be applied to the cloud storage bit apply to the Linux setup?

Has anyone reported that disabling cache connector still caches the packages locally for nuget? If this was working I guess I wouldnt be affecting me cause I have cache off.
Despite cache off, the cache size has been increasing, such as the Syncfusion.Grid.WPF being downloaded locally..

So it looks like Proget is not even searching Syncfusion.Grid.WPF directory despite most of the new packages going into that.

I deleted Syncfusion.Grid.Wpf directory and it couldn't find the package despite being a copy inside Syncfusion.Grid.WPF directory.

C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\Common7\IDE\CommonExtensions\Microsoft\NuGet\NuGet.targets(114, 5): The feed 'https://xxx/nuget/ks-nuget/v3/index.json [https://xxx/nuget/ks-nuget/v3/index.json]' lists package 'Syncfusion.Grid.WPF.17.4.0.47' but multiple attempts to download the nupkg have failed. The feed is either invalid or required packages were removed while the current operation was in progress. Verify the package exists on the feed and try again.
  Unable to find package 'Syncfusion.Grid.WPF.17.4.0.47'.

What made this work was copy/move and rename 'Syncfusion.Grid.WPF\Syncfusion.Grid.WPF.17.4.0.47.nupkg' to 'Syncfusion.Grid.Wpf\Syncfusion.Grid.Wpf.17.4.0.47.nupkg'

If no cache is set for the connector, none of this should technically matter, correct?

Great! hoping this will fix the issue.

Took awhile clicking through each diagnostics message to find the timeout error below. Is there an easier way to search through these messages?

Prior to the timeout message there were error messages regarding database out of max pool size reached. So they may be correlated.

error NU1101: Unable to find package System.IO.FileSystem.Watcher. No packages exist with this id in source(s): https://xxx/nuget/ks-nuget/v3/index.json

Message: Connector error: The operation has timed out.

Request URL: https://www.nuget.org/api/v2/FindPackagesById()?id=%27system.io.filesystem.watcher%27&semVerLevel=2.0.0

System.Net.WebException: The operation has timed out.
at System.Net.HttpWebRequest.RunWithTimeoutWorker[T] (System.Threading.Tasks.Task`1[TResult] workerTask, System.Int32 timeout, System.Action abort, System.Func`1[TResult] aborted, System.Threading.CancellationTokenSource cts) [0x000f8] in <4d8f95b7df544b3fa1a27dda924b4424>:0
at System.Net.HttpWebRequest.EndGetResponse (System.IAsyncResult asyncResult) [0x00020] in <4d8f95b7df544b3fa1a27dda924b4424>:0
at System.Threading.Tasks.TaskFactory`1[TResult].FromAsyncCoreLogic (System.IAsyncResult iar, System.Func`2[T,TResult] endFunction, System.Action`1[T] endAction, System.Threading.Tasks.Task`1[TResult] promise, System.Boolean requiresSynchronization) [0x0000f] in <c453bf849bb84e719dbec0475e99db95>:0
--- End of stack trace from previous location where exception was thrown ---

at Inedo.ProGet.Feeds.Connector.GetDirectResponseAsync (System.String url) [0x000eb] in <5675ed1ff2ff49938725a260ae4389ad>:0
at Inedo.ProGet.Feeds.Connector.GetCachedResponseAsync (System.String url) [0x000a8] in <5675ed1ff2ff49938725a260ae4389ad>:0
at Inedo.ProGet.Feeds.Connector.GetCachedTextReaderAsync (System.String url) [0x00075] in <5675ed1ff2ff49938725a260ae4389ad>:0
at Inedo.ProGet.Feeds.NuGet.V2.NuGetFeedClient.PerformRequestWithAbsoluteUrlAsync (System.String fullUrl) [0x000d9] in <5675ed1ff2ff49938725a260ae4389ad>:0

Correct, this is on docker.
This may be related to the database error; I can further test this when the database error is resolved. This particular issue was not something I can produce consistently.

we are actually using both versions, some of our legacy applications appear to be pointing to the 12.x version. I copied the package into Wpf folder and had to modify the names of the actual package for it to work. The casing of the package had to match as well for it to work.

Please work on a fix for this. I think this type of mucking around in the packages directory may pollute the contents.

It is multiple packages, not just MethodTimer.Fody. Our code/dependencies does use multiple versions of MethodTimer.Fody, and we have multiple versions cached locally on Proget.

both of those commands run fine, however issue is unless the package is cached on the Proget server, it cant be used by MSBuild. And i actually have connector caching turned off, but it still caches..

this is definitely breaking builds:

C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\Common7\IDE\CommonExtensions\Microsoft\NuGet\NuGet.targets(114, 5): The feed 'https://xxx/nuget/ks-nuget/v3/index.json [https://xxx/nuget/ks-nuget/v3/index.json]' lists package 'Syncfusion.Grid.WPF.17.4.0.47' but multiple attempts to download the nupkg have failed. The feed is either invalid or required packages were removed while the current operation was in progress. Verify the package exists on the feed and try again.
  Unable to find package 'Syncfusion.Grid.WPF.17.4.0.47'.

Package is clearly there at least on the server, probably searching in the wrong directory

Actually seems like 1000 is not enough, i just hit the error again upon compiling a bigger project. These are the two errors that are grouped together usually.

 An error occurred processing a GET request to http://xxxxx.nupkg: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.

An error occurred in the web application: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.

had to up it to 1000 before i stopped seeing the issue

During build i occasionally get this issue where the nuget package is not found, i can search it in the UI and it can see it on nuget.org via connector but unable to install.. however if i manually do a nuget install, it caches locally and then Msbuild is able to get the package from proget and compile.

NU1101: Unable to find package MethodTimer.Fody. No packages exist with this id in source(s): https://xxx/nuget/ks-nuget/v3/index.json

C:\>nuget install MethodTimer.Fody -Version 3.1.2 -Source https://xxx/nuget/ks-nuget/v3/index.json

Doing the 'pull to proget' in the UI does not seem to make it work.

Anyone having a similar issue?

i set it to 300 in the connection string still seeing max pool, gonna increase it to 500 and see how that goes.

i am running MSSQL 2017 in docker, will that require increasing the max allowed connections on the SQL server as well? I will increase the max pool size to 300 and see how that goes.

actually I would like to add that Proget is also reaching out to nuget.org to search for internal packages as well and gets a timeout. Is it suppose to reach out for something that it should have found locally?
I wonder if these additional calls to nuget.org contributes to to this exception that I have also been getting.

Details: System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.

I am hitting an issue when trying to install nuget from proget, it reports the package cannot be found despite reporting having it. If i recall correctly the message may be similar to this

multiple attempts to download the nupkg have failed. The feed is either invalid or required packages were removed while the current operation was in progress

I had 2 nuget connectors originally, v2 and v3, but I deleted the v2 just to see if this error would stop. But it still happened. Works for me if its nothing to be concerned about.

thank you.

I am using nuget v3 api but for some reason getting connection timeout on v2.

Request URL: https://www.nuget.org/api/v2/FindPackagesById()?id=%27microsoft.extensions.dependencyinjection.abstractions%27&semVerLevel=2.0.0

System.Net.WebException: The operation has timed out.
at System.Net.HttpWebRequest.RunWithTimeoutWorker[T] (System.Threading.Tasks.Task`1[TResult] workerTask, System.Int32 timeout, System.Action abort, System.Func`1[TResult] aborted, System.Threading.CancellationTokenSource cts) [0x000f8] in <4d8f95b7df544b3fa1a27dda924b4424>:0
at System.Net.HttpWebRequest.EndGetResponse (System.IAsyncResult asyncResult) [0x00020] in <4d8f95b7df544b3fa1a27dda924b4424>:0
at System.Threading.Tasks.TaskFactory`1[TResult].FromAsyncCoreLogic (System.IAsyncResult iar, System.Func`2[T,TResult] endFunction, System.Action`1[T] endAction, System.Threading.Tasks.Task`1[TResult] promise, System.Boolean requiresSynchronization) [0x0000f] in <c453bf849bb84e719dbec0475e99db95>:0
--- End of stack trace from previous location where exception was thrown ---

at Inedo.ProGet.Feeds.Connector.GetDirectResponseAsync (System.String url) [0x000eb] in <5675ed1ff2ff49938725a260ae4389ad>:0
at Inedo.ProGet.Feeds.Connector.GetCachedResponseAsync (System.String url) [0x000a8] in <5675ed1ff2ff49938725a260ae4389ad>:0
at Inedo.ProGet.Feeds.Connector.GetCachedTextReaderAsync (System.String url) [0x00075] in <5675ed1ff2ff49938725a260ae4389ad>:0
at Inedo.ProGet.Feeds.NuGet.V2.NuGetFeedClient.PerformRequestWithAbsoluteUrlAsync (System.String fullUrl) [0x000d9] in <5675ed1ff2ff49938725a260ae4389ad>:0

Is there a chance that Proget is caching files and then searching for files in the wrong directory due to case sensitivity?
Check out the screenshots below where the package name does not reflect properly in storage path.
ALSO i have set the connector to NOT cache. But it still caches..
Using proget 5.3.7 - docker
For example, i reindex the nuget feed, its looking a package in Syncfusion.Grid.Wpf (however it appears to have stored in Syncfusion.Grid.WPF directory

Let me know if I can provide more logs or something.

I tested the following: changing the nuget connector to utilize v2. The effect is the same where I cannot pull from nuget.org. This definitely worked prior to v5.3

Right now i have proget running a v3 api for the feed with a v2 connector and it is working fine so far.

I believe its a bug when the feed is running v2.

I might have found a temporary fix by utilizing enabling and utilizing v3 api, running this allows it to install perfectly fine it seems.

nuget install CommonServiceLocator -Version 1.3.0 -Source https://localnugetserver/nuget/ks-nuget/v3/index.json

can someone look into this issue/ verify that it is not just me having the issue?

I have noticed that since upgrading to v5.3.5 Proget, the connector/nuget feed is not working properly. For example I am trying to download CommonServiceLocator.1.3.0.0.nupkg, it is an external package in the public nuget server. This is what i get when running in CLI

C:\Dev\Nuget>nuget install CommonServiceLocator -Version 1.3.0 -Source https://localprogetserver/nuget/ks-nuget/
Feeds used:
  C:\Dev\NuGet\packages
  https://localprogetserver/nuget/ks-nuget/

Attempting to gather dependency information for package 'CommonServiceLocator.1.3.0' with respect to project 'C:\Dev\Nuget', targeting 'Any,Version=v0.0'
Gathering dependency information took 342 ms
Attempting to resolve dependencies for package 'CommonServiceLocator.1.3.0' with DependencyBehavior 'Lowest'
Resolving dependency information took 0 ms
Resolving actions to install package 'CommonServiceLocator.1.3.0'
Resolved actions to install package 'CommonServiceLocator.1.3.0'
Retrieving package 'CommonServiceLocator 1.3.0' from 'https://localprogetserver/nuget/ks-nuget/'.
  GET https://localprogetserver/nuget/ks-nuget/package/CommonServiceLocator/1.3
  NotFound https://localprogetserver/nuget/ks-nuget/package/CommonServiceLocator/1.3 152ms
WARNING: Install failed. Rolling back...
Executing nuget actions took 199 ms
Package 'CommonServiceLocator.1.3.0' is not found on source 'https://localprogetserver/nuget/ks-nuget/'.

In the pictures above, it shows that its clearly available online through the external connector.
Prior/ expected result when I try to install a nuget package not on our local proget server is that I get proxied out to the external nuget.org and get the installation.

Please assist.

you should update the link to download the docker image because it is pointing to dockerhub...

https://inedo.com/proget/download

hi,

i just checked the diagnostics center, I see other errors regarding feeds etc, but i do anything regarding docker pull.

Its weird i only get this issue when i'm not logged in as a user; unless this was by design.

I have setup a nginx proxy with SSL certs setup. I am able to do docker pull and docker push, IF i do a docker login first with Admin for example. The anonymous permissions allows for the tasks of 'view and download packages' however I am unable to do a docker pull without any login.

I get the following error:
Error response from daemon: received unexpected HTTP status: 500 Internal Server Error

I am running the proget docker image with mssql and nginx.

Can we confirm if its possible to download docker images without login?

Thank you