Inedo Community Forums Forums
    • Recent
    • Tags
    • Popular
    • Login
    1. Home
    2. scott.wright_8356
    3. Posts

    Welcome to the Inedo Forums! Check out the Forums Guide for help getting started.

    If you are experiencing any issues with the forum software, please visit the Contact Form on our website and let us know!

    S Offline
    • Profile
    • Following 0
    • Followers 0
    • Topics 17
    • Posts 44
    • Groups 0

    Posts

    Recent Best Controversial
    • ProGet Connector Timeout

      Hi Everyone,

      We have an issue with our ProGet server running 24.0.6. We have added some new caching feeds and getting the following error in the logs consistently a few times a day.

      Level:Warning
      Category:Connector
      Message:Unable to fetch connector data; using cached copy.
      Details:System.Net.WebException: The operation has timed out.
      at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
      at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
      --- End of stack trace from previous location ---
      at System.Net.WebRequest.GetResponseAsync()
      at Inedo.ProGet.Feeds.Connector.GetDirectResponseAsync(String url)
      at Inedo.ProGet.Feeds.Connector.GetCachedResponseAsync(String url)
      

      We are unable to tell from the logs which connector is timing out. We want to check the network and firewall and ensure no errors for the public URL. Is there a way to determine this?

      Regards Scott

      posted in Support
      S
      scott.wright_8356
    • RE: ProGet Lock Issues

      I set our metadata caching to 10000 queries over 24 hours. I will reduce to the default first and monitor.

      Thanks Dean

      posted in Support
      S
      scott.wright_8356
    • RE: ProGet Lock Issues

      Hi Dean,

      We have narrowed the problem down to a particular query causing SQL CPU Blocking. Connectors_GetCachedResponse. This is causing significant timeouts when our server is under load.

      ALTER PROCEDURE [dbo].[Connectors_GetCachedResponse]
      (
        @Connector_Id INT,
        @Request_Hash BINARY(32)
      )
      AS BEGIN

      SET NOCOUNT ON

      BEGIN TRY
        BEGIN TRANSACTION

        UPDATE [ConnectorResponses]
         SET [LastUsed_Date] = GETUTCDATE(),
         [Request_Count] = [Request_Count] + 1
         WHERE [Connector_Id] = @Connector_Id
         AND [Request_Hash] = @Request_Hash

        SELECT *
         FROM [ConnectorResponses]
         WHERE [Connector_Id] = @Connector_Id
         AND [Request_Hash] = @Request_Hash

        COMMIT
      END TRY BEGIN CATCH
        IF XACT_STATE()<>0 ROLLBACK
        EXEC [HandleError]
      END CATCH

      END

      3517e4f8-864b-472f-ba1b-3ebf69a23c67-image.png

      posted in Support
      S
      scott.wright_8356
    • ProGet Lock Issues

      Hi ProGet Support,

      We have an issue where our High-Availability ProGet infrastructure uses over 1000 SQL connections and reaches the max pool size. Some investigation on the SQL server using the activity monitor has shown the following queries running.

      CREATE PROCEDURE [PgvdVulnerabilities_GetVulnerabilitiesForPackage]
      (
      	@PackageName_Id INT
      )
      AS
      BEGIN
       
          SELECT * INTO #PgvdPackageNames FROM [PgvdPackageNames_Extended] WHERE [PackageName_Id] = @PackageName_Id
          SELECT * FROM [PgvdVulnerabilities_Extended] 
           WHERE [Pgvd_Id] IN (SELECT [Pgvd_Id] FROM #PgvdPackageNames)
       
          SELECT * FROM #PgvdPackageNames
       
          SELECT * FROM [PgvdAssessments_Extended]
           WHERE [Pgvd_Id] IN (SELECT [Pgvd_Id] FROM #PgvdPackageNames)
       
      END
      
      CREATE PROCEDURE [Feeds_GetExtendedConfiguration]
      (
      	@Feed_Id INT
      )
      AS
      BEGIN
       
      	SET NOCOUNT ON
       
      	SELECT *
      	  FROM [FeedPackageAccessRules]
      	 WHERE [Feed_Id] = @Feed_Id
      	 ORDER BY [Sequence_Number]
       
      	SELECT C.*
      	  FROM [FeedConnectors] FC
      	       INNER JOIN [Connectors] C
      		           ON C.[Connector_Id] = FC.[Connector_Id]
      	 WHERE FC.[Feed_Id] = @Feed_Id
      	 ORDER BY FC.[Sequence_Number]
       
      	SELECT *
      	  FROM [FeedPackageFilters]
      	 WHERE [Feed_Id] = @Feed_Id
      	 ORDER BY [Sequence_Number]
       
      END
      

      This is causing an object lock

      0a7b85d3-e641-4094-bc4e-4be74f4c0756-image.png

      Our CD/CI process keeps requesting packages, and within 5 minutes, all the have 1000's of connections, and we get the following errors in ProGet when the lock releases.

      An error occurred processing a GET request to http://proget.XXXXX/npm/Registry/cache-base: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
      Details:System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
      at Microsoft.Data.Common.ADP.ExceptionWithStackTrace(Exception e)
      --- End of stack trace from previous location ---
      at Inedo.Data.SqlServerDatabaseContext.CreateConnectionAsync()
      at Inedo.Data.DatabaseContext.ExecuteInternalAsync(String storedProcName, GenericDbParameter[] parameters)
      at Inedo.Data.SqlServerDatabaseContext.ExecuteInternalAsync(String storedProcName, GenericDbParameter[] parameters)
      at Inedo.Data.DatabaseContext.ExecuteTableAsync[TRow](String storedProcName, GenericDbParameter[] parameters)
      at Inedo.ProGet.WebApplication.FeedEndpoints.Npm.NpmPackageMetadataHandler.TryProcessRequestAsync(AhHttpContext context, WebApiContext apiContext, NpmFeed feed, String relativeUrl)
      at Inedo.ProGet.WebApplication.FeedEndpoints.Npm.NpmHandler.ProcessRequestAsync(AhHttpContext context, WebApiContext apiContext, NpmFeed feed, String relativeUrl)
      at Inedo.ProGet.WebApplication.FeedEndpoints.FeedEndpointHandler.FeedRequestHandler.ProcessRequestAsync(AhHttpContext context)
      

      We are running version 23.0.31.

      Regards Scott

      posted in Support
      S
      scott.wright_8356
    • RE: Conda Feed to SMB Share

      @rhessinger Wonderful, thank you very much and credit to Pawel Ostrowski who also did a lot of investigation on this

      posted in Support
      S
      scott.wright_8356
    • RE: Conda Feed to SMB Share

      Hi Alana,

      What we did find to get our small applications working was the following post. This mentioned that when opening the file the path should start with four backslashes. "\\"

      https://stackoverflow.com/questions/41330660/sqlite-database-file-cant-be-opened-when-placed-in-network-folder

      https://stackoverflow.com/questions/17303076/sqlite-cannot-open-network-file-programmatically-even-though-worked-before

      This worked for our small applications and allowed the SQL lite package to work over the SMB share.

      Regards Scott

      posted in Support
      S
      scott.wright_8356
    • RE: Conda Feed to SMB Share

      Hi Alana,

      We have written many simple programs using the same SQL Lite package as above. We cannot find a scenario where SQL Lite works over an SMB share, and we have been able to find content that says not to use it over the network. We were able to confirm that the create database option works fine. It is the opening that fails every time. What happens with the OpenOrCreateDatabase is it rolls back the file created when the open fails, so you end up with nothing.

      https://www.sqlite.org/useovernet.html

      https://stackoverflow.com/questions/788517/sqlite-over-a-network-share

      We are using the IIS setup. Has this been tested with IIS and SMB share for Conda? I assume the index information for the package repository is what is going into the SQL Lite DB? Would it not be faster to store this information locally on each node? Does the SQL Lite DB need to be shared?

      Is there an option to set the Conda/Debian package storage on the network share and the SQL Lite DB locally for each node?

      Regards Scott

      posted in Support
      S
      scott.wright_8356
    • RE: Conda Feed to SMB Share

      Hi Alana,

      We wrote a program to use the SQL Lite library and ran it from the ProGet Node.

      e46f533d-817b-44d4-a112-8ff1c59c8a17-image.png

      0dcf3131-557c-4515-9b4b-c87a9012e8e9-image.png

      3b9357dd-30db-4bc3-a6a9-d6adfcf24974-image.png

      This is from the ProGet node server to our ProGet SMB share, and there are no issues. It is just the ProGet application that is having a problem.

      posted in Support
      S
      scott.wright_8356
    • Conda Feed to SMB Share

      Hi Support,

      We have an issue creating a Conda feed to an SMB share. The SQL Lite file cannot be created there; we are running a High Availability structure with no issues with NuGet, Npm, Python, and Maven feeds. With the Conda feed, we get
      20aa33ef-3605-434e-b200-ff8ac39766df-image.png

      Is there a particular port that ProGet requires when communicating with SMB share for SQL Lite? We can get the feed working with local storage on one of the nodes but not with the shared storage. When we put the SQL lite file there manually and refresh the feed, it is deleted. It is like clearing the file and then attempting to write it there using a different process than the other feeds.

      Regards Scott

      posted in Support
      S
      scott.wright_8356
    • RE: ProGet New Vulnerability scanning removes ability to block a package

      Thank you very much for this, Alana. Very much appreciated.

      posted in Support
      S
      scott.wright_8356
    • RE: ProGet New Vulnerability scanning removes ability to block a package

      Hi Alana,

      How do we set the package status to blocked? I cannot find this option against these packages, which is what I have been looking for. I remember it being there, but it appears to be gone in version 23.0.31. Also, regarding the above response, do you intend to allow users to modify the vulnerability database or not? The style of the answer indicates no.

      Regards Scott

      posted in Support
      S
      scott.wright_8356
    • ProGet New Vulnerability scanning removes ability to block a package

      Hi Support,

      We have an issue with the Moq library in NuGet. The Vulnerability assessment is wrong and has a limited list of versions to block.

      PGV-2358804: Moq v4.20.0-rc to 4.20.1 share hashed user data

      We have confirmed that 40.20.2 and 4.20.69 still have the vulnerability, but we cannot clock these in the new versions of ProGet now. How do we extend the vulnerabilities to block packages that are a problem? I can understand why these versions were missed, as you need to go through the commits that make up the versions and check them for yourself. This is why the ability to block manually on the existing version was so important.

      Regards Scott

      posted in Support
      S
      scott.wright_8356
    • Package Latest Version not showing in All Versions

      Hi Support,

      We often have an issue showing up for a npm feed. The feed is a public cache feed of https://registry.npmjs.org with metadata caching enabled. What is happening is that on the feed when searching the package, it is showing the latest version.

      8fd21fa9-b3de-4610-b727-05f184256152-image.png

      When selecting that version, we get the following error.

      2be5ac82-e5b5-4c0c-844c-011c6d07349c-image.png

      Also, we cannot see the latest package when looking at All Versions.

      9f856f59-990a-4aec-be95-08d64cc9b986-image.png

      This issue fixes itself over time. However, for example, this package is updated hourly, consumed as a dependency, and causes intermittent build issues in our CD/CI.

      Regards Scott

      posted in Support
      S
      scott.wright_8356
    • ProGet Enteprise License Update

      Hi Support,

      We just updated our High Availably structure with a new ProGet Enterprise license key and received the following error message.

      Logged:2/04/2024 9:32:36 AM
      Level:Error
      Category:Web
      Message:An error occurred in the web application: No primary service node has been registered.
      Details:URL: http://proget.wtg.zone/administration/licensing/change-license-key
      Referrer: https://proget.wtg.zone/administration/licensing/change-license-key
      User: SW
      User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36 Edg/122.0.0.0
      Stack trace: at Inedo.ProGet.WebApplication.AggregateServiceMessengerClient.GetPrimaryNodeId()
      at Inedo.ProGet.WebApplication.AggregateServiceMessengerClient.SendMessage(ServiceMessage message)
      at Inedo.ProGet.WebApplication.Pages.Administration.Licensing.ChangeLicenseKeyPage.<>c__DisplayClass1_0.<<CreateChildControls>b__0>d.MoveNext()
      --- End of stack trace from previous location ---
      at Inedo.Web.PageFree.SimplePageBase.ExecutePageLifeCycleAsync()
      at Inedo.Web.PageFree.SimplePageBase.ProcessRequestAsync(AhHttpContext context)
      at Inedo.Web.AhWebMiddleware.InvokeAsync(HttpContext context)
      

      I went back to the page and found the key had successfully updated. I manually ran the activation process. It appears to be working, as the expiry date has updated on the license.

      Regards Scott

      posted in Support
      S
      scott.wright_8356
    • RE: Maven Andriod Endpoint URL

      @atripp We determined what the issue was. Our firewall had application identification enabled and hence blocked it. Can we find out what the Connector Health check is doing? In our case, some connectors were not healthy. ProGet was reporting healthy in its checks.

      Regards Scott

      posted in Support
      S
      scott.wright_8356
    • RE: Maven Andriod Endpoint URL

      Thank you Alana,

      I will investigate a lot more and see if I can work something out over the weekend. Will post back here the results if still stuck.

      posted in Support
      S
      scott.wright_8356
    • RE: Maven Andriod Endpoint URL

      @atripp said in Maven Andriod Endpoint URL:

      https://dl.google.com/android/maven2

      We have a problem with this on our old server. We have the packages on the feed. We have imported them to the new ProGet Server. The connector is setup correctly.

      53d256d5-6ff4-44c3-aa0c-75dbcad069c1-image.png

      It is healthy

      e3a8eb7c-6dae-4066-ac8c-5ba2e3aa0a18-image.png

      It is connected to the feed.

      b5a54564-3e6c-46d7-9c5b-97ed07e9e1d3-image.png

      When we request the package, we are after we get the following.

      9dcc7759-2fe9-4786-b1af-05042e7fcb16-image.png

      We have attempted to import the old servers' packages directory as a temporary fix, but we get the following error for all the imports from the other package directory.

      Added Maven Package com.android.tools.build/gradle 1.5.0.
      Could not add file "\proget-smb.wtg.zone\Proget\F12\android.arch.core\common\1.1.0\common-1.1.0-sources.jar". There was not matching pom file.
      Could not add file "\proget-smb.wtg.zone\Proget\F12\android.arch.core\common\1.1.0\common-1.1.0.jar". There was not matching pom file.

      When we look at packages that were successfully imported, we get the following error:

      3f9ac923-6ebe-417f-a358-619e11ebd1bf-image.png

      With a corresponding record in the diagnostic log.

      Level:Error
      Category:Web
      Message:An error occurred in the web application: Object reference not set to an instance of an object.
      Details:URL: http://proget.wtg.zone/feeds/Maven-Android/com.android.tools.build/com.android.tools.build.gradle/1.5.0
      Referrer: https://proget.wtg.zone/packages
      User: SW
      User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36 Edg/122.0.0.0
      Stack trace: at Inedo.ProGet.Feeds.Maven.MavenFeed.Inedo.ProGet.Feeds.IBrowsablePackageFeed.ListPackageVersionsAsync(BrowsablePackageId id)
      at Inedo.ProGet.WebApplication.Pages.Packages.PackagePageBase.CreateChildControlsAsync()
      at Inedo.ProGet.WebApplication.Pages.ProGetSimplePage.InitializeAsync()
      at Inedo.Web.PageFree.SimplePageBase.ExecutePageLifeCycleAsync()
      at Inedo.Web.PageFree.SimplePageBase.ProcessRequestAsync(AhHttpContext context)
      at Inedo.Web.AhWebMiddleware.InvokeAsync(HttpContext context)
      
      posted in Support
      S
      scott.wright_8356
    • Maven Andriod Endpoint URL

      Hi Everyone,

      We are trying to work out the endpoint for the Maven Android feeds at Google. We have tried the following with no success.

      https://dl.google.com/dl/android/maven2
      https://dl.google.com
      https://dl.google.com/android/maven2
      https://maven.google.com/web/index.html

      Any help would be great.

      Cheers Scott

      posted in Support
      S
      scott.wright_8356
    • RE: Maven feed for cache

      @atripp We downgraded, but still no luck. It is incredibly slow and still getting the Object Reference errors. The 500 error in accessing Maven packages also occurs in both versions. What we did find, looking at our build logs, was the download from the feed was incredibly slow. All features were turned off the feed, and still, the download was slow. We went through the connectors and removed them one by one. We found that our connector for Google Andriod was incredibly slow, it was reporting healthy but was killing the feed.

      6eef55b3-599c-4b78-8266-cc81f0da5ede-image.png

      Is there a way to tell the speed of a public connector response so we can see the really slow ones and see if they are outdated? I think this might be a deprecated feed URL maybe.

      posted in Support
      S
      scott.wright_8356
    • RE: Maven feed for cache

      We are running ProGet 2023.31 and the problem with the Maven feeds is occurring on our test system as well.

      posted in Support
      S
      scott.wright_8356
    • 1
    • 2
    • 3
    • 1 / 3