Hi,

it somehow healed "itself"? This morning it presented me with Issues in the feed and I started a reindex. Now I can browse the feed again.

The data was originally uploaded from a build in jenkins through curl:

curl --parallel --user usr:psw --config config.txt

with the config.txt file like this
config.txt-Schema:

url = "https://<proget_url>/rpm/archive_name"
upload-file="rpm_1"
.
.
.
url = "https://<proget_url>/rpm/archive_name"
upload-file="rpm_13861"

We are hosting an RPM feed, which can't be used in the UI anymore, because if only shows this error:

This is logged in the Diagnostics Center.

The archive entry was compressed using an unsupported compression method.

System.IO.InvalidDataException: The archive entry was compressed using an unsupported compression method.
   at System.IO.Compression.Inflater.Inflate(FlushCode flushCode)
   at System.IO.Compression.Inflater.ReadInflateOutput(Byte* bufPtr, Int32 length, FlushCode flushCode, Int32& bytesRead)
   at System.IO.Compression.Inflater.ReadOutput(Byte* bufPtr, Int32 length, Int32& bytesRead)
   at System.IO.Compression.Inflater.InflateVerified(Byte* bufPtr, Int32 length)
   at System.IO.Compression.DeflateStream.CopyToStream.Write(Byte[] buffer, Int32 offset, Int32 count)
   at System.IO.Compression.DeflateStream.CopyToStream.CopyFromSourceToDestination()
   at Inedo.ProGet.Feeds.Rpm.Metadata.RpmHeader.Inedo.ProGet.Feeds.IPackageManifest<Inedo.ProGet.Feeds.Rpm.Metadata.RpmHeader>.ReadMetadataFromDataRow(IPackageVersionTable row) in C:\Users\builds\AppData\Local\Temp\InedoAgent\BuildMaster\192.168.44.60\Temp\_E593291\Src\src\ProGet\Feeds\Rpm\Metadata\RpmHeader.cs:line 205
   at Inedo.ProGet.Feeds.PackageFeed`5.LocalPackageMetadata..ctor(TPackageVersionRow row) in C:\Users\builds\AppData\Local\Temp\InedoAgent\BuildMaster\192.168.44.60\Temp\_E593291\Src\src\ProGet\Feeds\PackageFeed.PackageGet.cs:line 372
   at Inedo.ProGet.Feeds.Rpm.RpmFeed.GetLocalPackage(FeedPackageVersions_Extended row) in C:\Users\builds\AppData\Local\Temp\InedoAgent\BuildMaster\192.168.44.60\Temp\_E593291\Src\src\ProGet\Feeds\Rpm\RpmFeed.cs:line 293
   at System.Linq.Enumerable.SelectEnumerableIterator`2.MoveNext()
   at System.Collections.Generic.HashSet`1.UnionWith(IEnumerable`1 other)
   at System.Collections.Generic.HashSet`1..ctor(IEnumerable`1 collection, IEqualityComparer`1 comparer)
   at System.Linq.Enumerable.ToHashSet[TSource](IEnumerable`1 source, IEqualityComparer`1 comparer)
   at Inedo.ProGet.Feeds.PackageFeed`5.SearchLatestPackagesAsync(String query, Nullable`1 maxCount, Boolean includePrerelease, CancellationToken cancellationToken)
   at Inedo.ProGet.Feeds.PackageFeed`5.Inedo.ProGet.Feeds.IPackageFeed.SearchLatestPackagesAsync(String query, Nullable`1 maxCount, Boolean includePrerelease, CancellationToken cancellationToken) in C:\Users\builds\AppData\Local\Temp\InedoAgent\BuildMaster\192.168.44.60\Temp\_E593291\Src\src\ProGet\Feeds\PackageFeed.PackageGet.cs:line 61
   at Inedo.ProGet.WebApplication.Pages.Packages.ListPackagesPage.PackageList.InitializeAsyncInternal() in C:\Users\builds\AppData\Local\Temp\InedoAgent\BuildMaster\192.168.44.60\Temp\_E593291\Src\src\ProGet\WebApplication\Pages\Packages\ListPackagesPage.PackageList.cs:line 61
   at Inedo.ProGet.WebApplication.Pages.Packages.ListPackagesPage.PackageList.InitializeAsync() in C:\Users\builds\AppData\Local\Temp\InedoAgent\BuildMaster\192.168.44.60\Temp\_E593291\Src\src\ProGet\WebApplication\Pages\Packages\ListPackagesPage.PackageList.cs:line 43

::Web Error on 09/17/2025 11:58:42::

They are actually disrupting usage.
Feeds take a long time to load, or don't load at all.

Reproduction is complicated. It seems to work fine after restart. Then after a day or so, probably after a bunch of data has been deployed to ProGet, it starts breaking down.

Hi,

some really weird behaviour, but quite worrying:

In a few random download cases, instead of the requested data, we are getting data that looks like a piece of the process memory.

In one observed case, we received this bit of data instead of the actual requested package (librados2-perl debian package)

Any idea why that is happening? This doesn't look comforting in a security sense.

Sadly, I don't have any concrete hints on how to reproduce this.

We are currently migrating towards ProGet.
Since we are using it, we observe a growing number of exceptions in the user interface:

An error occurred in the web application: Cannot transcode invalid UTF-8 JSON text to UTF-16 string.

URL: http://<hostname>/containers/registry?feedId=1
Referrer: https://<hostname>/containers/repositories/<container-feed>/sub-python-3.12/overview
User: (unknown)
User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/140.0.0.0 Safari/537.36 Edg/140.0.0.0
IP Address: <Client IP>
Stack trace:    at Inedo.Web.PageFree.SimplePageBase.ProcessRequestAsync(AhHttpContext context)
   at Inedo.Web.AhWebMiddleware.InvokeAsync(HttpContext context)

::Web Error on 09/15/2025 13:28:11::

Unhandled exception: Internal Npgsql bug: unexpected value 0 of enum BackendMessageCode. Please file a bug.

System.InvalidOperationException: Internal Npgsql bug: unexpected value 0 of enum BackendMessageCode. Please file a bug.
   at Npgsql.ThrowHelper.ThrowInvalidOperationException(String message)
   at Npgsql.Internal.NpgsqlConnector.ParseServerMessage(NpgsqlReadBuffer buf, BackendMessageCode code, Int32 len, Boolean isPrependedMessage, Boolean handleCallbacks)
   at Npgsql.Internal.NpgsqlConnector.ReadMessageLong(Boolean async, DataRowLoadingMode dataRowLoadingMode, Boolean readingNotifications, Boolean isReadingPrependedMessage)
   at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder`1.StateMachineBox`1.System.Threading.Tasks.Sources.IValueTaskSource<TResult>.GetResult(Int16 token)
   at Npgsql.Internal.NpgsqlConnector.ReadMessageLong(Boolean async, DataRowLoadingMode dataRowLoadingMode, Boolean readingNotifications, Boolean isReadingPrependedMessage)
   at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder`1.StateMachineBox`1.System.Threading.Tasks.Sources.IValueTaskSource<TResult>.GetResult(Int16 token)
   at Npgsql.NpgsqlDataReader.NextResult(Boolean async, Boolean isConsuming, CancellationToken cancellationToken)
   at Npgsql.NpgsqlDataReader.NextResult(Boolean async, Boolean isConsuming, CancellationToken cancellationToken)
   at Npgsql.NpgsqlDataReader.NextResult()
   at Npgsql.NpgsqlCommand.ExecuteReader(Boolean async, CommandBehavior behavior, CancellationToken cancellationToken)
   at Npgsql.NpgsqlCommand.ExecuteReader(Boolean async, CommandBehavior behavior, CancellationToken cancellationToken)
   at Npgsql.NpgsqlCommand.ExecuteNonQuery(Boolean async, CancellationToken cancellationToken)
   at Inedo.ProGet.Data.PostgresDatabaseContext.CreateConnection() in C:\Users\builds\AppData\Local\Temp\InedoAgent\BuildMaster\192.168.44.60\Temp\_E593291\Src\src\ProGet\Data\PostgresDatabaseContext.cs:line 70
   at Inedo.Data.DatabaseContext.ExecuteInternal(String storedProcName, GenericDbParameter[] parameters, DatabaseCommandReturnType returnType)
   at Inedo.Data.DatabaseContext.ExecuteNonQuery(String storedProcName, GenericDbParameter[] parameters)
   at Inedo.ProGet.Executions.ManualExecutionLogScope.System.IDisposable.Dispose() in C:\Users\builds\AppData\Local\Temp\InedoAgent\BuildMaster\192.168.44.60\Temp\_E593291\Src\src\ProGet\Executions\ManualExecutionLogScope.cs:line 62
   at Inedo.ProGet.Service.Executions.ActiveScheduledTaskExecution.ExecuteAsync() in C:\Users\builds\AppData\Local\Temp\InedoAgent\BuildMaster\192.168.44.60\Temp\_E593291\Src\src\ProGet\Service\Executions\ActiveScheduledTaskExecution.cs:line 49
   at Inedo.ProGet.Service.Executions.ActiveScheduledTaskExecution.ExecuteAsync() in C:\Users\builds\AppData\Local\Temp\InedoAgent\BuildMaster\192.168.44.60\Temp\_E593291\Src\src\ProGet\Service\Executions\ActiveScheduledTaskExecution.cs:line 53
   at Inedo.ProGet.Service.ExecutionDispatcherRunner.ExecuteAsync(Int32 id, ActiveExecution exec, TaskRunnerSubtask`1 context) in C:\Users\builds\AppData\Local\Temp\InedoAgent\BuildMaster\192.168.44.60\Temp\_E593291\Src\src\ProGet\Service\TaskRunners\ExecutionDispatcherRunner.cs:line 189

::ExecutionDispatcherRunner Error on 09/15/2025 23:58:46::

The output char buffer is too small to contain the decoded characters, encoding codepage '65001' and fallback 'System.Text.DecoderReplacementFallback'. (Parameter 'chars')

System.ArgumentException: The output char buffer is too small to contain the decoded characters, encoding codepage '65001' and fallback 'System.Text.DecoderReplacementFallback'. (Parameter 'chars')
   at System.Text.Encoding.GetCharsWithFallback(ReadOnlySpan`1 bytes, Int32 originalBytesLength, Span`1 chars, Int32 originalCharsLength, DecoderNLS decoder, Boolean throwForDestinationOverflow)
   at System.Text.UTF8Encoding.GetCharsWithFallback(ReadOnlySpan`1 bytes, Int32 originalBytesLength, Span`1 chars, Int32 originalCharsLength, DecoderNLS decoder, Boolean throwForDestinationOverflow)
   at System.Text.Encoding.GetCharsWithFallback(Byte* pOriginalBytes, Int32 originalByteCount, Char* pOriginalChars, Int32 originalCharCount, Int32 bytesConsumedSoFar, Int32 charsWrittenSoFar, Boolean throwForDestinationOverflow)
   at System.Text.UTF8Encoding.GetChars(Byte* bytes, Int32 byteCount, Char* chars, Int32 charCount)
   at System.String.CreateStringFromEncoding(Byte* bytes, Int32 byteLength, Encoding encoding)
   at System.Text.Encoding.GetString(ReadOnlySpan`1 bytes)
   at System.Text.Json.Nodes.JsonNode.ToJsonString(JsonSerializerOptions options)
   at Inedo.Web.Controls.AhTable.SerializeData(IEnumerable`1 rows)
   at Inedo.Web.Controls.AhTable.Inedo.Web.Controls.ISimpleCallbackHandler.ProcessRequestAsync(AhHttpContext context, String eventName, String eventArgument)

:: Error on 09/15/2025 13:15:15::

This makes the possible adoption of ProGet a bit hard ... What can we do?
Our setup is ProGet Community Edition running on Ubuntu 24.04 VM, using podman as Container Engine.

My setup:

• Host is Ubuntu 24.04 LTS
• Running proget in a quadlet
• Using embedded Database
• With a caddy as a reverse proxy in front of it
• Caddy also running in a quadlet
• proget and caddy in a common podman network
• Caddyfile as simple as possible:

<hostname {
    tls <certpath> <keypath>
    reverse_proxy http://proget:80
}

Well ...

info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
      Request starting HTTP/1.1 HEAD http://vl-dev-repo.ki.lan/v2/sst-devops-oci-prod/confluence-cmdb/blobs/sha256:341aba9cd1b201183186977186c3e3c8e8a9f79967b31f6fc2bff2a8da510c5d - - -
A 500 error occurred in sst-devops-oci-prod: Nullable object must have a value.
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
      Request finished HTTP/1.1 PUT http://vl-dev-repo.ki.lan/v2/sst-devops-oci-prod/confluence-cmdb/blobs/uploads/88cd2aa3-f537-4816-bcd8-23b0ed5eefa4?digest=sha256%3A31e705e6bb0246691b44e7e35991da5066444c0e861d9ef40703695b037552b8 - 500 90 application/json 32.8098ms
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
      Request finished HTTP/1.1 HEAD http://vl-dev-repo.ki.lan/v2/sst-devops-oci-prod/confluence-cmdb/blobs/sha256:fdab6c583bb88b36e350d7c2011d1a6490bc8a501cc503d9b4a0198ca1f1cfb0 - 404 158 application/json 13.9966ms
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
      Request starting HTTP/1.1 HEAD http://vl-dev-repo.ki.lan/v2/sst-devops-oci-prod/confluence-cmdb/blobs/sha256:3626f18144516cf17d3c15f60a55c075d34130f1cc3a00d43fe68edf5568d994 - - -
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
      Request finished HTTP/1.1 PATCH http://vl-dev-repo.ki.lan/v2/sst-devops-oci-prod/confluence-cmdb/blobs/uploads/1cf960f6-1b21-4a89-9eb7-1064b2f9a9ad - 202 0 - 11.7030ms
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
      Request finished HTTP/1.1 HEAD http://vl-dev-repo.ki.lan/v2/sst-devops-oci-prod/confluence-cmdb/blobs/sha256:123542a56fc98af9ea8a10cb74548ace0c5e05db21e2f82e4ef989870dc72adf - 200 246 application/octet-stream 17.2649ms
A 500 error occurred in sst-devops-oci-prod: Nullable object must have a value.
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
      Request finished HTTP/1.1 PUT http://vl-dev-repo.ki.lan/v2/sst-devops-oci-prod/confluence-cmdb/blobs/uploads/ba1a754f-3c96-4b3a-8d04-8dd9c90e3db3?digest=sha256%3Aa2c2378190e0b91ba3e274e1d2c236873b1960d15602027fb9110f787feee901 - 500 90 application/json 25.0325ms
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
      Request finished HTTP/1.1 HEAD http://vl-dev-repo.ki.lan/v2/sst-devops-oci-prod/confluence-cmdb/blobs/sha256:3626f18144516cf17d3c15f60a55c075d34130f1cc3a00d43fe68edf5568d994 - 200 2338 application/octet-stream 6.4659ms
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
      Request finished HTTP/1.1 HEAD http://vl-dev-repo.ki.lan/v2/sst-devops-oci-prod/confluence-cmdb/blobs/sha256:341aba9cd1b201183186977186c3e3c8e8a9f79967b31f6fc2bff2a8da510c5d - 404 158 application/json 16.9192ms
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
      Request starting HTTP/1.1 PUT http://vl-dev-repo.ki.lan/v2/sst-devops-oci-prod/confluence-cmdb/blobs/uploads/1cf960f6-1b21-4a89-9eb7-1064b2f9a9ad?digest=sha256%3A7af4036786251654395076ed8392df109d652ae2e1c6de74aac94db89a9f9235 - application/octet-stream 0
A 500 error occurred in sst-devops-oci-prod: Nullable object must have a value.
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
      Request finished HTTP/1.1 PUT http://vl-dev-repo.ki.lan/v2/sst-devops-oci-prod/confluence-cmdb/blobs/uploads/1cf960f6-1b21-4a89-9eb7-1064b2f9a9ad?digest=sha256%3A7af4036786251654395076ed8392df109d652ae2e1c6de74aac94db89a9f9235 - 500 90 application/json 16.1379ms
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
      Request starting HTTP/1.1 HEAD http://vl-dev-repo.ki.lan/v2/sst-devops-oci-prod/confluence-cmdb/blobs/sha256:972115bef0bf0586114d827ecd50b5591446d6e260d2f734e2861280eb620422 - - -
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
      Request finished HTTP/1.1 HEAD http://vl-dev-repo.ki.lan/v2/sst-devops-oci-prod/confluence-cmdb/blobs/sha256:972115bef0bf0586114d827ecd50b5591446d6e260d2f734e2861280eb620422 - 200 14142880 application/octet-stream 3.5634ms
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
      Request starting HTTP/1.1 GET http://vl-dev-repo.ki.lan/0x44/proget/Inedo.ProGet.WebApplication.Controls.Layout.NotificationBar/GetNotifications - - -
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
      Request finished HTTP/1.1 GET http://vl-dev-repo.ki.lan/0x44/proget/Inedo.ProGet.WebApplication.Controls.Layout.NotificationBar/GetNotifications - 200 30 - 6.2359ms

doesn't look like the logging is extended ... @atripp

Something I found: I switched from podman to docker as a test and that is able to push images that podman can't

Chiming in here:
I have the same issue and it is behaving very erratic.

• I have some images that are pushed fine without any parameter.
• I have some images that are pushed fine with zstd
• I have some images that are pushed fine with zstd:chunked

This is getting very weird ...

Hi,

I am not sure, that I could help you with an instance here (sadly).

Regarding the username/password issue:

• It works fine up to the point of choosing the repository to mirror from. When I then click Start Import the UI returns a 500 and the following exception in the log

An error occurred in the web application: Response status code does not indicate success: 403.

URL: http://vl-dev-repo.ki.lan/feed/import/packages/importer?feedId=10&step=ConfirmImport&tempKey=PackageImport_V2_5b4cb993-1550-4ba4-be36-c557d48bb2c8
Referrer: https://vl-dev-repo.ki.lan/feed/import/packages/importer?feedId=10&step=SelectImportFeedOptions&tempKey=PackageImport_V2_5b4cb993-1550-4ba4-be36-c557d48bb2c8
User: Admin
User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:142.0) Gecko/20100101 Firefox/142.0
Stack trace:    at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
   at Inedo.ProGet.FeedImporters.Artifactory.ArtifactoryImportService.GetFilesAsync(String repositoryName, CancellationToken cancellationToken)+MoveNext() in C:\Users\builds\AppData\Local\Temp\InedoAgent\BuildMaster\192.168.44.60\Temp\_E584854\Src\src\ProGet\FeedImporters\Artifactory\ArtifactoryImportService.cs:line 80
   at Inedo.ProGet.FeedImporters.Artifactory.ArtifactoryImportService.GetFilesAsync(String repositoryName, CancellationToken cancellationToken)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()
   at System.Linq.AsyncEnumerable.<CountAsync>g__Core|71_0[TSource](IAsyncEnumerable`1 source, CancellationToken cancellationToken) in /_/Ix.NET/Source/System.Linq.Async/System/Linq/Operators/Count.cs:line 41
   at System.Linq.AsyncEnumerable.<CountAsync>g__Core|71_0[TSource](IAsyncEnumerable`1 source, CancellationToken cancellationToken) in /_/Ix.NET/Source/System.Linq.Async/System/Linq/Operators/Count.cs:line 41
   at Inedo.ProGet.FeedImporters.FeedImportService.OptionsSummary.CreateInternalAsync(ArtifactoryImportService service, ArtifactoryImportOptions importOptions, FeedType feedType, CancellationToken cancellationToken) in C:\Users\builds\AppData\Local\Temp\InedoAgent\BuildMaster\192.168.44.60\Temp\_E584854\Src\src\ProGet\FeedImporters\FeedImportService.OptionsSummary.cs:line 33
   at Inedo.ProGet.WebApplication.Pages.Feeds.ImportWithImporterPage.<>c__DisplayClass19_0.<<CreateChildControlsAsync>g__confirmImport|5>d.MoveNext() in C:\Users\builds\AppData\Local\Temp\InedoAgent\BuildMaster\192.168.44.60\Temp\_E584854\Src\src\ProGet\WebApplication\Pages\Feeds\ImportPackagesOfTheNewPage.cs:line 291
--- End of stack trace from previous location ---
   at Inedo.ProGet.WebApplication.Pages.Feeds.ImportWithImporterPage.CreateChildControlsAsync() in C:\Users\builds\AppData\Local\Temp\InedoAgent\BuildMaster\192.168.44.60\Temp\_E584854\Src\src\ProGet\WebApplication\Pages\Feeds\ImportPackagesOfTheNewPage.cs:line 79
   at Inedo.ProGet.WebApplication.Pages.ProGetSimplePage.InitializeAsync() in C:\Users\builds\AppData\Local\Temp\InedoAgent\BuildMaster\192.168.44.60\Temp\_E584854\Src\src\ProGet\WebApplication\Pages\ProGetSimplePage.cs:line 69
   at Inedo.Web.PageFree.SimplePageBase.ExecutePageLifeCycleAsync()
   at Inedo.Web.PageFree.SimplePageBase.ProcessRequestAsync(AhHttpContext context)
   at Inedo.Web.AhWebMiddleware.InvokeAsync(HttpContext context)

::Web Error on 08/27/2025 11:31:07::

I am especially confused about the fact that it prints Windows paths, even though it is running as Linux Docker container.

Hi @atripp,

just to chime in here: I am getting the same issues right now.
Sadly, this is on an internal Artifactory that is not available to the public.

I also found a small other issue with the importer (when using User/Password instead of Token). Where should I report that? (I got past that with the token login)