1. Home
  2. Categories
  3. Support
  4. Proget: Debian2 connector to https://packages.microsoft.com/ubuntu/22.04/prod/ results in unique constraint failed error

Welcome to the Inedo Forums! Check out the Forums Guide for help getting started.

If you are experiencing any issues with the forum software, please visit the Contact Form on our website and let us know!

Proget: Debian2 connector to https://packages.microsoft.com/ubuntu/22.04/prod/ results in unique constraint failed error

  • I

    When creating a new debian connector to https://packages.microsoft.com/ubuntu/22.04/prod/ , it results in the following error:

    An unexpected error occurred while listing packages: constraint failed UNIQUE constraint failed: Packages.PackageIndex_Id, Packages.Package_Name, Packages.Package_Version. Additional information has been logged in the diagnostics center.

    How can we get Proget to correctly show the packages of the repository?

    We are using version 2024.15

    The error message in the diagnostics center is this:
    constraint failed UNIQUE constraint failed: Packages.PackageIndex_Id, Packages.Package_Name, Packages.Package_Version

    code = Constraint (19), message = System.Data.SQLite.SQLiteException (0x87AF202F): constraint failed
UNIQUE constraint failed: Packages.PackageIndex_Id, Packages.Package_Name, Packages.Package_Version
at System.Data.SQLite.SQLite3.Step(SQLiteStatement stmt)
at System.Data.SQLite.SQLiteDataReader.NextResult()
at System.Data.SQLite.SQLiteDataReader..ctor(SQLiteCommand cmd, CommandBehavior behave)
at System.Data.SQLite.SQLiteCommand.ExecuteNonQuery(CommandBehavior behavior)
at Inedo.ProGet.Feeds.Debian2.Debian2ConnectorIndex.UpdateIndexAsync(String component, String architecture, Byte[] hash, Func`1 stanzas)
at Inedo.ProGet.Feeds.Debian2.Debian2ConnectorIndex.UpdateIndexAsync(String component, String architecture, Byte[] hash, Func`1 stanzas)
at Inedo.ProGet.Feeds.Debian2.Debian2Connector.UpdateIndexAsync(Debian2ConnectorIndex localIndex)
at Inedo.ProGet.Feeds.Debian2.Debian2Connector.GetAndUpdateIndexAsync()
at Inedo.ProGet.Feeds.Debian2.Debian2Connector.ListPackagesAsync(Nullable`1 maxCount)+MoveNext()
at Inedo.ProGet.Feeds.Debian2.Debian2Connector.ListPackagesAsync(Nullable`1 maxCount)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()
at Inedo.ProGet.Feeds.Debian2.Debian2Feed.GetConnectorPackagesAsync(Func`2 getPackages)+MoveNext()
at Inedo.ProGet.Feeds.Debian2.Debian2Feed.GetConnectorPackagesAsync(Func`2 getPackages)+MoveNext()
at Inedo.ProGet.Feeds.Debian2.Debian2Feed.GetConnectorPackagesAsync(Func`2 getPackages)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()
at System.Linq.AsyncEnumerable.UnionAsyncIterator`1.MoveNextCore() in /_/Ix.NET/Source/System.Linq.Async/System/Linq/Operators/Union.cs:line 131
at System.Linq.AsyncIteratorBase`1.MoveNextAsync() in /_/Ix.NET/Source/System.Linq.Async/System/Linq/AsyncIterator.cs:line 70
at System.Linq.AsyncIteratorBase`1.MoveNextAsync() in /_/Ix.NET/Source/System.Linq.Async/System/Linq/AsyncIterator.cs:line 75
at System.Linq.Internal.Lookup`2.CreateAsync(IAsyncEnumerable`1 source, Func`2 keySelector, IEqualityComparer`1 comparer, CancellationToken cancellationToken) in /_/Ix.NET/Source/System.Linq.Async/System/Linq/Operators/Lookup.cs:line 105
at System.Linq.Internal.Lookup`2.CreateAsync(IAsyncEnumerable`1 source, Func`2 keySelector, IEqualityComparer`1 comparer, CancellationToken cancellationToken) in /_/Ix.NET/Source/System.Linq.Async/System/Linq/Operators/Lookup.cs:line 105
at System.Linq.AsyncEnumerable.GroupedAsyncEnumerable`2.MoveNextCore()
at System.Linq.AsyncIteratorBase`1.MoveNextAsync() in /_/Ix.NET/Source/System.Linq.Async/System/Linq/AsyncIterator.cs:line 70
at System.Linq.AsyncIteratorBase`1.MoveNextAsync() in /_/Ix.NET/Source/System.Linq.Async/System/Linq/AsyncIterator.cs:line 75
at Inedo.ProGet.Feeds.Debian2.Debian2Feed.GetLatestVersionOfEach(IAsyncEnumerable`1 packages)+MoveNext()
at Inedo.ProGet.Feeds.Debian2.Debian2Feed.GetLatestVersionOfEach(IAsyncEnumerable`1 packages)+MoveNext()
at Inedo.ProGet.Feeds.Debian2.Debian2Feed.GetLatestVersionOfEach(IAsyncEnumerable`1 packages)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()
at System.Linq.AsyncEnumerable.SelectEnumerableAsyncIterator`2.MoveNextCore() in /_/Ix.NET/Source/System.Linq.Async/System/Linq/Operators/Select.cs:line 221
at System.Linq.AsyncIteratorBase`1.MoveNextAsync() in /_/Ix.NET/Source/System.Linq.Async/System/Linq/AsyncIterator.cs:line 70
at System.Linq.AsyncIteratorBase`1.MoveNextAsync() in /_/Ix.NET/Source/System.Linq.Async/System/Linq/AsyncIterator.cs:line 75
at System.Linq.AsyncEnumerable.<ToListAsync>g__Core|424_0[TSource](IAsyncEnumerable`1 source, CancellationToken cancellationToken) in /_/Ix.NET/Source/System.Linq.Async/System/Linq/Operators/ToList.cs:line 36
at System.Linq.AsyncEnumerable.<ToListAsync>g__Core|424_0[TSource](IAsyncEnumerable`1 source, CancellationToken cancellationToken) in /_/Ix.NET/Source/System.Linq.Async/System/Linq/Operators/ToList.cs:line 36
at Inedo.ProGet.Feeds.Debian2.Debian2Feed.SearchPackagesAsync(String query, Int32 maxCount, Boolean includePrerelease)
at Inedo.ProGet.WebApplication.Pages.Packages.ListPackagesPage.PackageList.InitializeAsyncInternal()
at Inedo.ProGet.WebApplication.Pages.Packages.ListPackagesPage.PackageList.InitializeAsync()
    0 atripp 1 Reply
  • atripp
    inedo-engineer

    Hi @it4it_9320 ,

    I was able to reproduce this; it looks like the jammy/main/binary-amd64/Packages index is invalid and duplicates several packages, including this:

    Package: aadsshlogin
Version: 1.0.023850001
Architecture: amd64
Section: utils
Priority: optional
Maintainer: Yancho Yanev <yyanev@microsoft.com>
Description: AAD NSS, PAM and certhandler extensions
 This package installs NSS, PAM and certhandler extensions to allow SSH login for AAD users.
Conflicts: aadlogin
Depends: libc6 (>= 2.34), libcurl4 (>= 7.16.2), libpam0g (>= 0.99.7.1), libselinux1 (>= 3.1~), libsemanage2 (>= 2.0.32), libssl3 (>= 3.0.0~~alpha1), libuuid1 (>= 2.16), passwd, openssh-server (>=6.9)
Pre-Depends: grep, sed
SHA256: efad79eb58c10155710ef59171fbe73d67e765a49ce4cc4f4e3622163f4c2f84
Size: 332574
Filename: pool/main/a/aadsshlogin/aadsshlogin_1.0.023850001_amd64.deb

Package: aadsshlogin-selinux
Version: 1.0.023850001
Architecture: amd64
Section: utils
Priority: optional
Maintainer: Yancho Yanev <yyanev@microsoft.com>
Description: Selinux configuration for AAD NSS and PAM extensions.
Conflicts: aadlogin-selinux
Depends: policycoreutils (>=3.3-1), selinux-utils, selinux-policy-default
SHA256: 6a0c3277754585d81d7c1216a23fa034bca6cacef7f162aba0af301ea734fc49
Size: 2214
Filename: pool/main/a/aadsshlogin-selinux/aadsshlogin-selinux_1.0.023850001_amd64.deb

    So, as a result, the error occurrs. We will add some checking code for this bad index file, and plan to fix this in the upcoming maintenance release via PG-2834

    Thanks,
    Alana

    0
  • I

    Hi Alana,

    Thank you for the response! We'll be waiting for the maintenance release.

    Another thing we noticed about the sqlite files is that they keep growing. After a couple of days one of the sqlite file grew to 12GB. I thought they only indexed the files from the connector endpoint, but apparently they are storing some more data. Is there any way to stop these files from growing? At the moment we're deleting them daily.

    0
  • gdivis
    inedo-engineer

    Hi @it4it_9320,

    I've confirmed that the constraint violation issue will be fixed in this Friday's release of ProGet 2024.21.

    Regarding the other issue about growing indexes, which remote repository are you seeing this with? Is it the same one that exhibited the constraint error? In all of my testing, the sqlite index is usually in the range of 2-10MB - it's nothing more than the contents of the Packages index file.

    -Greg

    0
  • I

    Hi Greg,

    It is the following connector:
    947b6368-749a-445d-b4a2-a80da178f72e-image.png
    At the moment it is 105MB

    Kind regards,
    Tom

    0
  • gdivis
    inedo-engineer

    Hi @it4it_9320,

    That's about the size I'd expect for a connector to that index. So far, I can't get it to grow much beyond that - I'm wondering if this was caused by the transaction rollback after the constraint error happening over and over. In any case, we'll add an explicit VACCUUM after major updates that ought to prevent the index from expanding again.

    -Greg

    0
Log in to reply
 
Inedo Website Home • Support Home • Code of Conduct • Forums Guide • Documentation