Navigation

    Inedo Community Forums

    Forums

    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    1. Home
    2. scott.wright_8356
    S
    • Profile
    • Following
    • Followers
    • Topics
    • Posts
    • Best
    • Groups

    scott.wright_8356

    @scott.wright_8356

    2
    Reputation
    44
    Posts
    4
    Profile views
    0
    Followers
    0
    Following
    Joined Last Online
    Location Sydney, Australia

    scott.wright_8356 Follow

    Best posts made by scott.wright_8356

    • RE: SQL Error after upgrade version

      @atripp We had the same error when upgrading each node on a high availability install. Does the upgrade routine delete and recreate the stored procedures for each node upgrade? It felt like there was no check of a version for each node upgrade.

      posted in Support
      S
      scott.wright_8356
    • RE: Maven Andriod Endpoint URL

      @atripp said in Maven Andriod Endpoint URL:

      https://dl.google.com/android/maven2

      We have a problem with this on our old server. We have the packages on the feed. We have imported them to the new ProGet Server. The connector is setup correctly.

      53d256d5-6ff4-44c3-aa0c-75dbcad069c1-image.png

      It is healthy

      e3a8eb7c-6dae-4066-ac8c-5ba2e3aa0a18-image.png

      It is connected to the feed.

      b5a54564-3e6c-46d7-9c5b-97ed07e9e1d3-image.png

      When we request the package, we are after we get the following.

      9dcc7759-2fe9-4786-b1af-05042e7fcb16-image.png

      We have attempted to import the old servers' packages directory as a temporary fix, but we get the following error for all the imports from the other package directory.

      Added Maven Package com.android.tools.build/gradle 1.5.0.
      Could not add file "\proget-smb.wtg.zone\Proget\F12\android.arch.core\common\1.1.0\common-1.1.0-sources.jar". There was not matching pom file.
      Could not add file "\proget-smb.wtg.zone\Proget\F12\android.arch.core\common\1.1.0\common-1.1.0.jar". There was not matching pom file.

      When we look at packages that were successfully imported, we get the following error:

      3f9ac923-6ebe-417f-a358-619e11ebd1bf-image.png

      With a corresponding record in the diagnostic log.

      Level:Error
      Category:Web
      Message:An error occurred in the web application: Object reference not set to an instance of an object.
      Details:URL: http://proget.wtg.zone/feeds/Maven-Android/com.android.tools.build/com.android.tools.build.gradle/1.5.0
      Referrer: https://proget.wtg.zone/packages
      User: SW
      User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36 Edg/122.0.0.0
      Stack trace: at Inedo.ProGet.Feeds.Maven.MavenFeed.Inedo.ProGet.Feeds.IBrowsablePackageFeed.ListPackageVersionsAsync(BrowsablePackageId id)
      at Inedo.ProGet.WebApplication.Pages.Packages.PackagePageBase.CreateChildControlsAsync()
      at Inedo.ProGet.WebApplication.Pages.ProGetSimplePage.InitializeAsync()
      at Inedo.Web.PageFree.SimplePageBase.ExecutePageLifeCycleAsync()
      at Inedo.Web.PageFree.SimplePageBase.ProcessRequestAsync(AhHttpContext context)
      at Inedo.Web.AhWebMiddleware.InvokeAsync(HttpContext context)
      
      posted in Support
      S
      scott.wright_8356

    Latest posts made by scott.wright_8356

    • ProGet Connector Timeout

      Hi Everyone,

      We have an issue with our ProGet server running 24.0.6. We have added some new caching feeds and getting the following error in the logs consistently a few times a day.

      Level:Warning
      Category:Connector
      Message:Unable to fetch connector data; using cached copy.
      Details:System.Net.WebException: The operation has timed out.
      at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
      at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
      --- End of stack trace from previous location ---
      at System.Net.WebRequest.GetResponseAsync()
      at Inedo.ProGet.Feeds.Connector.GetDirectResponseAsync(String url)
      at Inedo.ProGet.Feeds.Connector.GetCachedResponseAsync(String url)
      

      We are unable to tell from the logs which connector is timing out. We want to check the network and firewall and ensure no errors for the public URL. Is there a way to determine this?

      Regards Scott

      posted in Support
      S
      scott.wright_8356
    • RE: ProGet Lock Issues

      I set our metadata caching to 10000 queries over 24 hours. I will reduce to the default first and monitor.

      Thanks Dean

      posted in Support
      S
      scott.wright_8356
    • RE: ProGet Lock Issues

      Hi Dean,

      We have narrowed the problem down to a particular query causing SQL CPU Blocking. Connectors_GetCachedResponse. This is causing significant timeouts when our server is under load.

      ALTER PROCEDURE [dbo].[Connectors_GetCachedResponse]
      (
        @Connector_Id INT,
        @Request_Hash BINARY(32)
      )
      AS BEGIN

      SET NOCOUNT ON

      BEGIN TRY
        BEGIN TRANSACTION

      UPDATE [ConnectorResponses]
         SET [LastUsed_Date] = GETUTCDATE(),
         [Request_Count] = [Request_Count] + 1
         WHERE [Connector_Id] = @Connector_Id
         AND [Request_Hash] = @Request_Hash

      SELECT *
         FROM [ConnectorResponses]
         WHERE [Connector_Id] = @Connector_Id
         AND [Request_Hash] = @Request_Hash

      COMMIT
      END TRY BEGIN CATCH
        IF XACT_STATE()<>0 ROLLBACK
        EXEC [HandleError]
      END CATCH

      END

      3517e4f8-864b-472f-ba1b-3ebf69a23c67-image.png

      posted in Support
      S
      scott.wright_8356
    • ProGet Lock Issues

      Hi ProGet Support,

      We have an issue where our High-Availability ProGet infrastructure uses over 1000 SQL connections and reaches the max pool size. Some investigation on the SQL server using the activity monitor has shown the following queries running.

      CREATE PROCEDURE [PgvdVulnerabilities_GetVulnerabilitiesForPackage]
      (
      	@PackageName_Id INT
      )
      AS
      BEGIN
       
          SELECT * INTO #PgvdPackageNames FROM [PgvdPackageNames_Extended] WHERE [PackageName_Id] = @PackageName_Id
          SELECT * FROM [PgvdVulnerabilities_Extended] 
           WHERE [Pgvd_Id] IN (SELECT [Pgvd_Id] FROM #PgvdPackageNames)
       
          SELECT * FROM #PgvdPackageNames
       
          SELECT * FROM [PgvdAssessments_Extended]
           WHERE [Pgvd_Id] IN (SELECT [Pgvd_Id] FROM #PgvdPackageNames)
       
      END
      
      CREATE PROCEDURE [Feeds_GetExtendedConfiguration]
      (
      	@Feed_Id INT
      )
      AS
      BEGIN
       
      	SET NOCOUNT ON
       
      	SELECT *
      	  FROM [FeedPackageAccessRules]
      	 WHERE [Feed_Id] = @Feed_Id
      	 ORDER BY [Sequence_Number]
       
      	SELECT C.*
      	  FROM [FeedConnectors] FC
      	       INNER JOIN [Connectors] C
      		           ON C.[Connector_Id] = FC.[Connector_Id]
      	 WHERE FC.[Feed_Id] = @Feed_Id
      	 ORDER BY FC.[Sequence_Number]
       
      	SELECT *
      	  FROM [FeedPackageFilters]
      	 WHERE [Feed_Id] = @Feed_Id
      	 ORDER BY [Sequence_Number]
       
      END
      

      This is causing an object lock

      0a7b85d3-e641-4094-bc4e-4be74f4c0756-image.png

      Our CD/CI process keeps requesting packages, and within 5 minutes, all the have 1000's of connections, and we get the following errors in ProGet when the lock releases.

      An error occurred processing a GET request to http://proget.XXXXX/npm/Registry/cache-base: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
      Details:System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
      at Microsoft.Data.Common.ADP.ExceptionWithStackTrace(Exception e)
      --- End of stack trace from previous location ---
      at Inedo.Data.SqlServerDatabaseContext.CreateConnectionAsync()
      at Inedo.Data.DatabaseContext.ExecuteInternalAsync(String storedProcName, GenericDbParameter[] parameters)
      at Inedo.Data.SqlServerDatabaseContext.ExecuteInternalAsync(String storedProcName, GenericDbParameter[] parameters)
      at Inedo.Data.DatabaseContext.ExecuteTableAsync[TRow](String storedProcName, GenericDbParameter[] parameters)
      at Inedo.ProGet.WebApplication.FeedEndpoints.Npm.NpmPackageMetadataHandler.TryProcessRequestAsync(AhHttpContext context, WebApiContext apiContext, NpmFeed feed, String relativeUrl)
      at Inedo.ProGet.WebApplication.FeedEndpoints.Npm.NpmHandler.ProcessRequestAsync(AhHttpContext context, WebApiContext apiContext, NpmFeed feed, String relativeUrl)
      at Inedo.ProGet.WebApplication.FeedEndpoints.FeedEndpointHandler.FeedRequestHandler.ProcessRequestAsync(AhHttpContext context)
      

      We are running version 23.0.31.

      Regards Scott

      posted in Support
      S
      scott.wright_8356
    • RE: Conda Feed to SMB Share

      @rhessinger Wonderful, thank you very much and credit to Pawel Ostrowski who also did a lot of investigation on this

      posted in Support
      S
      scott.wright_8356
    • RE: Conda Feed to SMB Share

      Hi Alana,

      What we did find to get our small applications working was the following post. This mentioned that when opening the file the path should start with four backslashes. "\\"

      https://stackoverflow.com/questions/41330660/sqlite-database-file-cant-be-opened-when-placed-in-network-folder

      https://stackoverflow.com/questions/17303076/sqlite-cannot-open-network-file-programmatically-even-though-worked-before

      This worked for our small applications and allowed the SQL lite package to work over the SMB share.

      Regards Scott

      posted in Support
      S
      scott.wright_8356
    • RE: Conda Feed to SMB Share

      Hi Alana,

      We have written many simple programs using the same SQL Lite package as above. We cannot find a scenario where SQL Lite works over an SMB share, and we have been able to find content that says not to use it over the network. We were able to confirm that the create database option works fine. It is the opening that fails every time. What happens with the OpenOrCreateDatabase is it rolls back the file created when the open fails, so you end up with nothing.

      https://www.sqlite.org/useovernet.html

      https://stackoverflow.com/questions/788517/sqlite-over-a-network-share

      We are using the IIS setup. Has this been tested with IIS and SMB share for Conda? I assume the index information for the package repository is what is going into the SQL Lite DB? Would it not be faster to store this information locally on each node? Does the SQL Lite DB need to be shared?

      Is there an option to set the Conda/Debian package storage on the network share and the SQL Lite DB locally for each node?

      Regards Scott

      posted in Support
      S
      scott.wright_8356
    • RE: Conda Feed to SMB Share

      Hi Alana,

      We wrote a program to use the SQL Lite library and ran it from the ProGet Node.

      e46f533d-817b-44d4-a112-8ff1c59c8a17-image.png

      0dcf3131-557c-4515-9b4b-c87a9012e8e9-image.png

      3b9357dd-30db-4bc3-a6a9-d6adfcf24974-image.png

      This is from the ProGet node server to our ProGet SMB share, and there are no issues. It is just the ProGet application that is having a problem.

      posted in Support
      S
      scott.wright_8356
    • Conda Feed to SMB Share

      Hi Support,

      We have an issue creating a Conda feed to an SMB share. The SQL Lite file cannot be created there; we are running a High Availability structure with no issues with NuGet, Npm, Python, and Maven feeds. With the Conda feed, we get
      20aa33ef-3605-434e-b200-ff8ac39766df-image.png

      Is there a particular port that ProGet requires when communicating with SMB share for SQL Lite? We can get the feed working with local storage on one of the nodes but not with the shared storage. When we put the SQL lite file there manually and refresh the feed, it is deleted. It is like clearing the file and then attempting to write it there using a different process than the other feeds.

      Regards Scott

      posted in Support
      S
      scott.wright_8356
    • RE: ProGet New Vulnerability scanning removes ability to block a package

      Thank you very much for this, Alana. Very much appreciated.

      posted in Support
      S
      scott.wright_8356