Inedo Community Forums Forums
    • Recent
    • Tags
    • Popular
    • Login

    Welcome to the Inedo Forums! Check out the Forums Guide for help getting started.

    If you are experiencing any issues with the forum software, please visit the Contact Form on our website and let us know!

    Proget: Debian2 connector to https://packages.microsoft.com/ubuntu/22.04/prod/ results in unique constraint failed error

    Scheduled Pinned Locked Moved Support
    6 Posts 3 Posters 31 Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • I Offline
      it4it_9320
      last edited by it4it_9320

      When creating a new debian connector to https://packages.microsoft.com/ubuntu/22.04/prod/ , it results in the following error:

      An unexpected error occurred while listing packages: constraint failed UNIQUE constraint failed: Packages.PackageIndex_Id, Packages.Package_Name, Packages.Package_Version. Additional information has been logged in the diagnostics center.

      How can we get Proget to correctly show the packages of the repository?

      We are using version 2024.15

      The error message in the diagnostics center is this:
      constraint failed UNIQUE constraint failed: Packages.PackageIndex_Id, Packages.Package_Name, Packages.Package_Version

      code = Constraint (19), message = System.Data.SQLite.SQLiteException (0x87AF202F): constraint failed
UNIQUE constraint failed: Packages.PackageIndex_Id, Packages.Package_Name, Packages.Package_Version
at System.Data.SQLite.SQLite3.Step(SQLiteStatement stmt)
at System.Data.SQLite.SQLiteDataReader.NextResult()
at System.Data.SQLite.SQLiteDataReader..ctor(SQLiteCommand cmd, CommandBehavior behave)
at System.Data.SQLite.SQLiteCommand.ExecuteNonQuery(CommandBehavior behavior)
at Inedo.ProGet.Feeds.Debian2.Debian2ConnectorIndex.UpdateIndexAsync(String component, String architecture, Byte[] hash, Func`1 stanzas)
at Inedo.ProGet.Feeds.Debian2.Debian2ConnectorIndex.UpdateIndexAsync(String component, String architecture, Byte[] hash, Func`1 stanzas)
at Inedo.ProGet.Feeds.Debian2.Debian2Connector.UpdateIndexAsync(Debian2ConnectorIndex localIndex)
at Inedo.ProGet.Feeds.Debian2.Debian2Connector.GetAndUpdateIndexAsync()
at Inedo.ProGet.Feeds.Debian2.Debian2Connector.ListPackagesAsync(Nullable`1 maxCount)+MoveNext()
at Inedo.ProGet.Feeds.Debian2.Debian2Connector.ListPackagesAsync(Nullable`1 maxCount)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()
at Inedo.ProGet.Feeds.Debian2.Debian2Feed.GetConnectorPackagesAsync(Func`2 getPackages)+MoveNext()
at Inedo.ProGet.Feeds.Debian2.Debian2Feed.GetConnectorPackagesAsync(Func`2 getPackages)+MoveNext()
at Inedo.ProGet.Feeds.Debian2.Debian2Feed.GetConnectorPackagesAsync(Func`2 getPackages)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()
at System.Linq.AsyncEnumerable.UnionAsyncIterator`1.MoveNextCore() in /_/Ix.NET/Source/System.Linq.Async/System/Linq/Operators/Union.cs:line 131
at System.Linq.AsyncIteratorBase`1.MoveNextAsync() in /_/Ix.NET/Source/System.Linq.Async/System/Linq/AsyncIterator.cs:line 70
at System.Linq.AsyncIteratorBase`1.MoveNextAsync() in /_/Ix.NET/Source/System.Linq.Async/System/Linq/AsyncIterator.cs:line 75
at System.Linq.Internal.Lookup`2.CreateAsync(IAsyncEnumerable`1 source, Func`2 keySelector, IEqualityComparer`1 comparer, CancellationToken cancellationToken) in /_/Ix.NET/Source/System.Linq.Async/System/Linq/Operators/Lookup.cs:line 105
at System.Linq.Internal.Lookup`2.CreateAsync(IAsyncEnumerable`1 source, Func`2 keySelector, IEqualityComparer`1 comparer, CancellationToken cancellationToken) in /_/Ix.NET/Source/System.Linq.Async/System/Linq/Operators/Lookup.cs:line 105
at System.Linq.AsyncEnumerable.GroupedAsyncEnumerable`2.MoveNextCore()
at System.Linq.AsyncIteratorBase`1.MoveNextAsync() in /_/Ix.NET/Source/System.Linq.Async/System/Linq/AsyncIterator.cs:line 70
at System.Linq.AsyncIteratorBase`1.MoveNextAsync() in /_/Ix.NET/Source/System.Linq.Async/System/Linq/AsyncIterator.cs:line 75
at Inedo.ProGet.Feeds.Debian2.Debian2Feed.GetLatestVersionOfEach(IAsyncEnumerable`1 packages)+MoveNext()
at Inedo.ProGet.Feeds.Debian2.Debian2Feed.GetLatestVersionOfEach(IAsyncEnumerable`1 packages)+MoveNext()
at Inedo.ProGet.Feeds.Debian2.Debian2Feed.GetLatestVersionOfEach(IAsyncEnumerable`1 packages)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()
at System.Linq.AsyncEnumerable.SelectEnumerableAsyncIterator`2.MoveNextCore() in /_/Ix.NET/Source/System.Linq.Async/System/Linq/Operators/Select.cs:line 221
at System.Linq.AsyncIteratorBase`1.MoveNextAsync() in /_/Ix.NET/Source/System.Linq.Async/System/Linq/AsyncIterator.cs:line 70
at System.Linq.AsyncIteratorBase`1.MoveNextAsync() in /_/Ix.NET/Source/System.Linq.Async/System/Linq/AsyncIterator.cs:line 75
at System.Linq.AsyncEnumerable.<ToListAsync>g__Core|424_0[TSource](IAsyncEnumerable`1 source, CancellationToken cancellationToken) in /_/Ix.NET/Source/System.Linq.Async/System/Linq/Operators/ToList.cs:line 36
at System.Linq.AsyncEnumerable.<ToListAsync>g__Core|424_0[TSource](IAsyncEnumerable`1 source, CancellationToken cancellationToken) in /_/Ix.NET/Source/System.Linq.Async/System/Linq/Operators/ToList.cs:line 36
at Inedo.ProGet.Feeds.Debian2.Debian2Feed.SearchPackagesAsync(String query, Int32 maxCount, Boolean includePrerelease)
at Inedo.ProGet.WebApplication.Pages.Packages.ListPackagesPage.PackageList.InitializeAsyncInternal()
at Inedo.ProGet.WebApplication.Pages.Packages.ListPackagesPage.PackageList.InitializeAsync()
      atrippA 1 Reply Last reply Reply Quote 0
      • atrippA Offline
        atripp inedo-engineer @it4it_9320
        last edited by

        Hi @it4it_9320 ,

        I was able to reproduce this; it looks like the jammy/main/binary-amd64/Packages index is invalid and duplicates several packages, including this:

        Package: aadsshlogin
Version: 1.0.023850001
Architecture: amd64
Section: utils
Priority: optional
Maintainer: Yancho Yanev <yyanev@microsoft.com>
Description: AAD NSS, PAM and certhandler extensions
 This package installs NSS, PAM and certhandler extensions to allow SSH login for AAD users.
Conflicts: aadlogin
Depends: libc6 (>= 2.34), libcurl4 (>= 7.16.2), libpam0g (>= 0.99.7.1), libselinux1 (>= 3.1~), libsemanage2 (>= 2.0.32), libssl3 (>= 3.0.0~~alpha1), libuuid1 (>= 2.16), passwd, openssh-server (>=6.9)
Pre-Depends: grep, sed
SHA256: efad79eb58c10155710ef59171fbe73d67e765a49ce4cc4f4e3622163f4c2f84
Size: 332574
Filename: pool/main/a/aadsshlogin/aadsshlogin_1.0.023850001_amd64.deb

Package: aadsshlogin-selinux
Version: 1.0.023850001
Architecture: amd64
Section: utils
Priority: optional
Maintainer: Yancho Yanev <yyanev@microsoft.com>
Description: Selinux configuration for AAD NSS and PAM extensions.
Conflicts: aadlogin-selinux
Depends: policycoreutils (>=3.3-1), selinux-utils, selinux-policy-default
SHA256: 6a0c3277754585d81d7c1216a23fa034bca6cacef7f162aba0af301ea734fc49
Size: 2214
Filename: pool/main/a/aadsshlogin-selinux/aadsshlogin-selinux_1.0.023850001_amd64.deb

        So, as a result, the error occurrs. We will add some checking code for this bad index file, and plan to fix this in the upcoming maintenance release via PG-2834

        Thanks,
        Alana

        1 Reply Last reply Reply Quote 0
        • I Offline
          it4it_9320
          last edited by

          Hi Alana,

          Thank you for the response! We'll be waiting for the maintenance release.

          Another thing we noticed about the sqlite files is that they keep growing. After a couple of days one of the sqlite file grew to 12GB. I thought they only indexed the files from the connector endpoint, but apparently they are storing some more data. Is there any way to stop these files from growing? At the moment we're deleting them daily.

          1 Reply Last reply Reply Quote 0
          • gdivisG Offline
            gdivis inedo-engineer
            last edited by

            Hi @it4it_9320,

            I've confirmed that the constraint violation issue will be fixed in this Friday's release of ProGet 2024.21.

            Regarding the other issue about growing indexes, which remote repository are you seeing this with? Is it the same one that exhibited the constraint error? In all of my testing, the sqlite index is usually in the range of 2-10MB - it's nothing more than the contents of the Packages index file.

            -Greg

            1 Reply Last reply Reply Quote 0
            • I Offline
              it4it_9320
              last edited by

              Hi Greg,

              It is the following connector:
              947b6368-749a-445d-b4a2-a80da178f72e-image.png
              At the moment it is 105MB

              Kind regards,
              Tom

              1 Reply Last reply Reply Quote 0
              • gdivisG Offline
                gdivis inedo-engineer
                last edited by

                Hi @it4it_9320,

                That's about the size I'd expect for a connector to that index. So far, I can't get it to grow much beyond that - I'm wondering if this was caused by the transaction rollback after the constraint error happening over and over. In any case, we'll add an explicit VACCUUM after major updates that ought to prevent the index from expanding again.

                -Greg

                1 Reply Last reply Reply Quote 0

                Hello! It looks like you're interested in this conversation, but you don't have an account yet.

                Getting fed up of having to scroll through the same posts each visit? When you register for an account, you'll always come back to exactly where you were before, and choose to be notified of new replies (either via email, or push notification). You'll also be able to save bookmarks and upvote posts to show your appreciation to other community members.

                With your input, this post could be even better 💗

                Register Login
                • 1 / 1
                • First post
                  Last post
                Inedo Website Home • Support Home • Code of Conduct • Forums Guide • Documentation