@artur-wisniowski_4029 thanks for the troubleshooting! We'll investigate/fix this via PG-2671; we're going to target this Friday's maintenance release
Posts made by stevedennis
-
RE: [ProGet] Alpine Feed Connector - Package Caching Broken
-
RE: ProGet 2023 - IIS App pool stopping
The issue sounds like it's related to LDAP configuration (i.e. slow queries to your LDAP/AD server), but it's hard to say. This wouldn't behave any differently in IIS for IWS.
The first thing I would try is disabling LDAP / AD. And Windows Integrated Authentication (if you have it enabled). If the server is still slow, then I would try http:// instead of https://
Once you've identified where the slowness is coming from, we can address it. The most common issue with LDAP is recursive/nested group searches - especially when there are like thousands of groups and everyone's a member of something.
I would "play" with your LDAP settings and try to isolate why it's so slow.
Thanks,
Steve -
RE: Two tier nuget Feeds and controlling build server access with minimum changes to yaml build/pipelines
Ultimately this is going to involve training for your developers. Just like instituting a code review process will be new and uncomfortable at first, a package review process will be the same. Developers will not like it and they will complain.
However, 99% of the time, developers will be fine using the
approved
feed. 1% of the time (when they want to test a new package or upgrade), then will use theunapproved
feed. They'll just need to learn how to switch package sources (it's a drop-down in Visual Studio) and then learn not commit commit these package references.My advise is to make it incumbent upon developers to not commit code/configuration that depends on unapproved packages. If they do, then it will "break the build" because the packages aren't available. This is an expected behavior - it would be like if a developer decided to upgrade to .NET9-beta.
"Don't break the build" is a common mantra on development teams, and it means developers can't commit code changes that will do that. Just extend that to using unapproved packages.
-
RE: ProGet SCA 2024 Preview Feedback - Package detection still hit or miss
Hi @jw ,
We added a
compliance
property via PG-2658 in the next maintenance release.It basically shows what's in the database (which is also what the page in the UI does):
writer.WritePropertyName("compliance"); writer.WriteStartObject(); writer.WriteString("result", Domains.PackageAnalysisResults.GetName(package.Result_Code)); if (package.Detail_Text is not null) writer.WriteString("detail", package.Detail_Text); if (package.Analysis_Date.HasValue) writer.WriteString("date", package.Analysis_Date.Value); writer.WriteEndObject();
I think you can rely on
result=Inconclusive
meaning the package isn't in ProGet. That's all we use the status for now, but in the future it might be used for something else. Aresult=Error
means that our code crashed and you shouldn't ever see that.We'll definitely considering doing something other than a single result string down the line, but for now this was the easiest :)
Thanks,
Steve -
RE: Two tier nuget Feeds and controlling build server access with minimum changes to yaml build/pipelines
Hi @pbinnell_2355 ,
The normal workflow for a two-feed package approval is to generally have developers use the
approved
feed but allow them to use theunapproved
feed when they need to use a new package or version.However, this shouldn't be their default development style. If they want to use packages that aren't approved, they can request approval for the package(s) they want to use.
This obviously slows down development, but so does code review. And it's a tradeoff in general.
You can use bulk promotion if you'd like. Go to the PAckages page, then select "Cached" from the type of package, then select the packages you wish to promote.
Hope that helps,
Steve -
RE: Package Vulnerabilities - API
@rick-edwards_9161 that is correct, these will only be developed for ProGet 2024
-
RE: Two tier nuget Feeds and controlling build server access with minimum changes to yaml build/pipelines
Hi @pbinnell_2355 ,
It sounds like you've built a kind of package approval process?
https://blog.inedo.com/nuget/package-approval-workflow/
If that's the case, you'll need to promote the packages developers need to feed "B" or ask the developers to not use unapproved packages.
Thanks,
Steve -
RE: Package Vulnerabilities - API
Hi @rick-edwards_9161 ,
There is a corresponding API, but we haven't documented it yet.
For now, you have to "reverse engineer" the code (ProGetClient.cs):
public async IAsyncEnumerable<VulnerabilityInfo> AuditPackagesForVulnerabilitiesAsync(IReadOnlyList<PackageVersionIdentifier> packages, [EnumeratorCancellation] CancellationToken cancellationToken = default) { ArgumentNullException.ThrowIfNull(packages); using var response = await this.http.PostAsJsonAsync("api/sca/audit-package-vulns", packages, ProGetApiJsonContext.Default.IReadOnlyListPackageVersionIdentifier, cancellationToken).ConfigureAwait(false); await CheckResponseAsync(response, cancellationToken).ConfigureAwait(false); using var stream = await response.Content.ReadAsStreamAsync(cancellationToken).ConfigureAwait(false); await foreach (var v in JsonSerializer.DeserializeAsyncEnumerable(stream, ProGetApiJsonContext.Default.VulnerabilityInfo, cancellationToken).ConfigureAwait(false)) yield return v!; }
We do plan to document all this in the coming weeks/months.
Thanks,
Steve -
RE: NPM Package name case sensitivity
Hi @dan-brown_0128 ,
Sorry that we your reply. It must have been closed by mistake on our dashboard or something.
You're correct - this does require a database cleanup. As it so happens, we do have a "duplicates clean-up" script available, but it's intended for ProGet 2024. It's extraordinarily complicated, as you can see:
https://gist.github.com/apxltd/351d328023c1c32852c30c335952fabb
If you're able to send us a copy of your database (we can share a secure link for you in EDO-10419, just let us know), then we can review and think of the best plan to fix it. That will be either running the script in ProGet 2023 or upgrading then running the script.
Let us know your thoughts
Thank you,
Steve -
RE: ProGet: Exception when NuGet README links to image that is not contained in the package
@jw we'll definitely keep this in mind, it doesn't look trivial based on our usage of that marked library
Personally, I always try to keep the Diagnostic Center clean and empty, so when new issues show up I can easily spot and address them. Sifting through messages that are basically spam, without being able to filter or ignore them costs me more of my time that I would like to invest for monitoring.
We do not recommend using the Diagnostic Center for proactive monitoring. It's only intended as a tool for troubleshooting things like connector or 500 errors that you / end-users encounter.
There are a lot of non-problem errors and warnings logged that aren't worth time even looking at.
-
RE: ProGet: Exception when NuGet README links to image that is not contained in the package
Thanks for clarifying @jw !
I understand how this can be annoying, but I don't think we want to change the
404
not logging error for this issue in particular. Open to ideas if you have them.As an FYI..
- I think we use the marked library to turn text into Markdown, like this:
el.innerHTML = marked(el.textContent);
- some users definitely use absolute urls within readmes to navigate and show content/images within ProGet
- some users may use relative urls for links, but I can't imagine that ever working for images
If there's a way to supress relative urls in the marked library somehow, we could probably add it if you know how!
- I think we use the marked library to turn text into Markdown, like this:
-
RE: ProGet SCA: Add license type issue
@jw thanks for the bug report! This will be fixed via PG-2650 in the next maintenance release. It should say "License Id" instead - and it will then redirect to the Edit page after, which is almost as convenient as using the same dialog but won't require rewriting the page ;)
-
RE: ProGet: Exception when NuGet README links to image that is not contained in the package
Hi @jw,
This is expected behavior, as 404 errors are logged for non-API requests, and that's a relative URL. Is
README.assets
a kind of documented standard? It might just intended for GitHub?Thanks,
Steve -
RE: Package Vulnerabilities - API
Hey @rick-edwards_9161 ,
Yes - this would be easiest to do with the
pgutil vulns audit
command, which we're still working on documenting.Description: List vulnerabilities associated with a package or project file Usage: pgutil vulns audit [options] Options: --input=<input> Project to audit for vulnerable packages --package=<package> Name of package to audit for vulnerabilities --type=<type> Type of package to audit for vulnerabilities Valid values: apk, deb, maven, nuget, conda, cran, helm, npm, pypi, rpm, gem --version=<version> Version of package to audit for vulnerabilities -?, --help Show help and usage information
See Getting started with pgutil to learn more.
-
RE: ProGet 2023 - IIS App pool stopping
Hi @rick-kramer_9238 ,
I wouldn't recommend upgrading to ProGet 2024 solely for that reason, as it's a major release and if there were issues, it'd be easier to isolate Integrated Web Server vs Regression.
Here are the upgrade notes:
https://docs.inedo.com/docs/proget-upgrade-2024That said, if you need to rollback to ProGet 2023, you can do so without restoring the database by simply using the Inedo Hub. While there are database schema changes, they are all backwards-compatible with ProGet 2023, which means you can safely rollback your ProGet installation if there's a showstopper bug, and then upgrade later.
However, you should backup your database as an extra precaution anyway.
Thank you,
Steve -
RE: Custom endpoint URL
Hi @rmusick_7875,
I'm afraid API endpoints are not customizable and we do not support doing "reverse proxy" or otherwise rewriting the URLs. Hopefully this will be a good chance to make the endpoint-url more easy to configure/change - this will be important, as you may wish to move to multiple feeds, etc.
Good luck,
Steve -
RE: ProGet: Chocolately unable to connect
Hi @greg-swiderski_0221 ,
It looks like there is an "Object reference not set to an instance of an object" error that's occurring while trying to connect:
2024-04-30 09:52:41,341 8256 [WARN ] - Unable to connect to source 'http://localhost:8624/nuget/approved-choco/': Object reference not set to an instance of an object.
That error is presumably occurring from the Chocolatey client (
choco
), and unfortunately there's no way to know what it means. Most likely, it's an "error reporting an error" message, but it's hard to say.You could use some HTTP monitoring software (Fiddler Classic, Wireshark), and see if ProGet is returning an error of some kind.... but even if so, choco should report that error.
I would check with the chocolatey team on this one.
Thanks,
Steve -
RE: [BM / OT] Renaming "user/password" or "private key" credentials breaks Linux config
@philippe-camelio_3885 I just cloned the issue for Otter (OT-509), so should be an easy fix. Maybe this will hlep w/ data sync issues as well!
-
RE: NuGet no longer works after upgrading to 2024
Hi @jw ,
Thanks - we modified the script and tested it against your database backup:
https://gist.github.com/apxltd/351d328023c1c32852c30c335952fabbThat said, your duplicate data should not cause any problems. The only feed packages this seems to modify in your database are
Microsoft.NetCore.App.Runtime.win-x86-8.0.0
andMicrosoft.NetCore.App.Runtime.win-x64-8.0.0
. Both are cached.That said, given that it had a bug, we probably won't put this script in 2024.2; we'd much rather "share the script on a case-by-case" basis to fix problems, until we're confident they will be resolved. Then decide how to have users repair the data.
Thank you,
Steve -
RE: [OT] Modifying a server configuration triggers a full "Server Check"
@philippe-camelio_3885 this is related to SQL Server timeouts/performance issues
Unfortunately we don't have any other information available from the information provided, and about the only suggest I would have is to attempt to increase hardware/CPU on the SQL Server, perform database maintenance (see advanced maintenance) , and finally use SQL Server performance monitoring tools to find where there are problems
There may be some queries that can be optimized or missing indexes. If you can find any of those, please let us know and we can explore to add those optimizations in the product
-
RE: NuGet no longer works after upgrading to 2024
@jw the script should also work in ProGet 2023
That error is implying that data in the
FeedPackageVersion
didn't get cleaned up, as it should have been done on this portion:
https://gist.github.com/apxltd/351d328023c1c32852c30c335952fabb#file-gistfile1-txt-L449Unfortunately this difficult to debug on its own, and it will be impossible to debug w/o the database itself, but if you're comfortable with SQL feel free to modify or tweak. It's not an easy script :(
We haven't yet run this against all customer databases yet, but it's our list this week. If you haven't sent us your database already, please do :)
-
RE: Errors in PackageAnalyzer logs
Hi @v-makkenze_6348 ,
Just an FYI, we do have a duplicates clean-up script available, and ran it against your database with no issues.
https://gist.github.com/apxltd/351d328023c1c32852c30c335952fabb
We haven't yet decided how to roll it out, but if you do run into more package analyzer issues like this (particularly on the SBOM side), it might help.
Thank you,
Steve -
RE: [BM / OT] Renaming "user/password" or "private key" credentials breaks Linux config
Looks like this is a bug when severs have list-type configuration variables; we'll get this fixed in an upcoming maintenance release via BM-3944
Otherwise I can't think of a work-around that would be any simpler then editing the impacted servers.
Thank you,
Steve -
RE: [OT] Modifying a server configuration triggers a full "Server Check"
A Full Server Check will occur once per hour by default - does that also crash? You can also trigger by going to Admin > Service.
Also, can you give more detail about what you mean by crashing? There is no "app pool" on Linux/Docker (doesn't use IIS), and the web application isn't responsible for running the server check (that's the Service application). So unclear what the crash would be.
Thank you,
Steve -
RE: NuGet no longer works after upgrading to 2024
Hi @jw ,
So far as we can tell, these sort of errors are only occurring when there is a 4-part NuGet version ending in 0 that was added in a previous version. For example, if you added
4.2.1.0
in a previous version, it could yield an analysis error when trying to view the package as4.2.1
(i.e. how NuGet requests it).The duplicate package names were a problem in ProGet 2023 as well, but we haven't seen any crashes do to that yet. Just weird results in the SBOM analysis. But these would have also been a problem in ProGet 2023.
So long story short, I don't think you need to run the script.
However, if you upgrade and run into these issues, then you can run it:
https://gist.github.com/apxltd/351d328023c1c32852c30c335952fabbIf that still doesn't work, then rollback to ProGet 2023 and restore the database. And hopefully send it to us again so we can investigate the issue :)
Thank you,
Steve -
RE: NuGet no longer works after upgrading to 2024
Hi @jw ,
We have a data-cleanup script but did not ship it with ProGet 2024.1; we are considering it in ProGet 2023.2.
The above error occurred because of a 4-part version that ended in
.0
, but if you just delete that from the feed and add it back again, its should work. Can you try that?We can provide you the data clean-up script if you want to try it. We ran it against several customer databases and it seems to work okay, and clean-up a lot of "bad" versions and casing.
Thank you,
Steve -
RE: NPM Package name case sensitivity
Hi @dan-brown_0128 ,
Oh that's really weird --- I must have mis understood from the get-go. I'm not sure how that's scenario is possible. At least in ProGet 2023, 2024, I wasn't able to reproduce this under any circumstances
Perhaps it was a bug in an import from a long-while ago. But re-reading your first post, it sounds like you've worked around it?
Otherwise I think it would take a kind of data-cleanup script to handle.
Thanks,
Steve -
RE: NPM Package name case sensitivity
hi @dan-brown_0128,
ProGet simply does not support case-sensitive npm package names. This is a 10+ year old design decision (back from ProGet v3), which we made because npm package were always supposed to be lower-cased: https://docs.npmjs.com/creating-a-package-json-file#required-name-and-version-fields
JSONPath
is an invalid package that entered the public registry due to an npmjs.org bug. It's an old package -- latest version version 0.1.2, is 8+ years old -- and according to the package page says that it's "moved tojsonpath-plus
to avoid npm problems in dealing with upper-case packages."ProGet permits these packages, but there are just going to be quirks if you have them in your feed. For example, if you have both
jsonpath-1.1.1
andJSONPath-0.1.2
in your feed, then they will generally show up asjsonpath
, but in some API queries you may seeJSONPath
.We understand this is different than how the npmjs.org registry behaves, but as I mentioned, this was a "day 1" choice that we can't change today. No one's mentioned this as an issue until now.
If npm audit is crashing because sometimes
JSONPath
is returned in results, that sounds like a bug in npm audit to me? We obviously can't add support for case-sensitive npm packages nor change how ProGet returns data in npm API calls just to work-around an npm audit bug like this.The easiest solution is to stop using
JSONPath
and remove from your feed, which it sounds like you already have. It's a really old package, and there are only a handful like this on the public registry.Best,
Steve -
RE: NPM Vulnerability with Exception
@dan-brown_0128 I understand
So you'd either have to upgrade (where we fixed the code) or reassess it to Low so npm audit won't crash. I suppose you could also patch npm audit so it doesn't crash.
-
RE: NPM Vulnerability with Exception
Hi @dan-brown_0128 ,
Thanks for the detailed analysis; we'll get this fixed via PG-2636 in 2024.1 (due later today).
You should be able to assess it as "Low" to work-around it in the meantime. FYI, here is the code that is used to map a vulnerability to an npm vulnerability:
//assessed if (vuln.Severity_Code != null) { return vuln.Severity_Code switch { Domains.AssessmentSeverityCodes.Error => npmAuditSeverity.critical, Domains.AssessmentSeverityCodes.Warning => npmAuditSeverity.high, Domains.AssessmentSeverityCodes.NotApplicable => npmAuditSeverity.info, Domains.AssessmentSeverityCodes.Custom => npmAuditSeverity.info, _ => npmAuditSeverity.info }; } // unassessed and has a Severity Score else if (!string.IsNullOrWhiteSpace(vuln.ScoreSeverity_Text)) { return vuln.ScoreSeverity_Text switch { nameof(CVSSRange.Low) => npmAuditSeverity.low, nameof(CVSSRange.Medium) => npmAuditSeverity.moderate, nameof(CVSSRange.High) => npmAuditSeverity.high, nameof(CVSSRange.Critical) => npmAuditSeverity.critical, _ => npmAuditSeverity.info, }; } return npmAuditSeverity.none;
Thank you,
Steve -
RE: NPM Package name case sensitivity
Hi @dan-brown_0128,
I understand that npm is currently case insensitive, but there was a brief period that it wasn't. This led to several duplicate packages being published:
I don't believe this is possible anymore? Either way, ProGet is case insensitive for npm packages, which means that
JSONPath
andjsonpath
are considered the same package.IfIn either case, the package name should be returned in results asJSONPath
came first, then that's what the package name is I guess?jsonpath
It doesn't seem to cause any issues, except with
npm audit
?This is a rare case, so I would just not worry about it. We don't recommend
npm audit
and you should usepgutil audit
anyway. More comprehensive, covers licenses, all that.. -
RE: Support for CocoaPods
Thanks for the inquiry; I've updated our Other feed types docs with a link to this thread.
On a first glance, it looks like a CocoaPod itself is just a basic text file that acts as a "pointer" to a GitHub repository. The pod client uses the git client to "download" files and tags for versioning. In other words, there is no package file.
A CocoaPod repository is a Git repository, and the pod client seems to just use git client to commit/push files to the repo. In other words, there is no API for ProGet to implement.
Here is what a CocoPod repo looks like (note how it's just a bunch of files in a Git repo):
https://github.com/CocoaPods/Specs/tree/master/SpecsTo make a "private repo", you basically just create a Git repository:
https://guides.cocoapods.org/making/private-cocoapodsThis means that ultimately to implement a CocoPods repository, your only option is to create a private Git repository? That's my inital assessment at least.
And of course, we have no plans to add Git source code hosting to ProGet :)
From here, I recommend to ask the developers to research this a bit more, and maybe contribute their thoughts? It just doesn't seem like the iOS devs uses packages - it's all very open-source and just GitHub repositories.
Steve
-
RE: GPG error updating Debian repositories
@daniel-scati we'll also get this fixed via PG-2635 in an upcoming maintenance release (hopefully 2024.2), which is targeted for next Friday.
-
RE: Date in Debian feed release file malformed
Hi @daniel-scati , thanks for the analysis!
We'll get this fixed via PG-2635 in an upcoming maintenance release (hopefully 2024.2), which is targeted for next Friday.
-
RE: Proget as registry proxy
ProGet works as a "Private Docker Registry" which seems to be different than a "Docker Hub Mirror".
Last time we researched Docker Hub Mirrors, they seemed to be primarily intended to provide image to certain geographic regions (like China) where Docker Hub content would otherwise be restricted. They could also be used to set up a "local mirror" of Docker Hub, but in all cases, it seemed to basically just redirect traffic from the default docker.io URL (or whatever) - so it wasn't intended to be used as "Private Docker Registries".
In any case, Mirrors don't seem to be a good fit for ProGet; instead, if you wish to use
nginx
, we would advise to "privatizing" and "lock" images using sematic tag, so that you can be assured thatcorp.local/images/nginx
is a tested/safe image with tags you control.Best,
Steve -
RE: What is the general design philosophy regarding permissions and visibility in ProGet?
@jw excellent, thanks for the finds! I also added this to our "final touches" for ProGet 2024 to address all these-- all pretty easy fixes I think
-
RE: ProGet feature request: Dedicated permission for marking packages as deprecated
Hi @jw ,
I'm afraid this is a bit too granular for us now, but it's something we can consider re-evaluating down the line, especially as we will likely want to add specialize permissions for projects, policies, etc. We expect that will happen later in the year, after ProGet 2024's new features get more adoption. We'll see if anyone else requests package-level permissions, etc.
As for the Advanced, I put a note in ProGet 2024's final touches to address that. Honestly I thought those were only on Debug builds only, but clearly not... thanks for reporting!
Cheers,
Steve -
RE: ProGet SAML group mapping
Hi @proget-markus-koban_7308 ,
It's highly unlikely we would consider implementing anything Keycloak-specific, but if it's something that SAML supports - and something done by the major providers like Azure, Ping ID, Okta, etc -- we definitely consider it. We just don't know much about it.
We haven't done any further research since that post and we likely won't do any further research on our own, since only one user asked in a few years (and they ended up not needing it anyway).
If this is something you'd be interested in exploring, it'd be best to collaborate and help us bridge the gap between SAML and ProGet.
Here's some relevant questions/discussion from that topic:
I'm not so familiar with SAML behind the scenes... do you know how "SAML group claims" work? For example...
- Is it something that comes back in the XML response, or does it require a separate request?
- What do the "group claims" look like? Like a list of human-readable group names?
And them most importantly... what should ProGet do with such claims upon receipt? Treat the user as if they're in the group (kind of like LDAP groups), and allow permissions to be assigned against that group (like LDAp, but without searching)?
The hardest part is going to be figuring out how to set this up in a SAML provider, document it, etc.
Thanks,
Steve -
RE: [BM] /!\ Proget Integration broken - given key was not present
No idea I'm afraid; there's clearly some issue with unexpected data coming from your ProGet server that's not being validated.
Can you share the results of
/health
(first API call) then/api/management/feeds/list
(using the API token you specified)?With that we can hopefully spot something.
Thanks,
Steve -
RE: Build execution failure for .Net projects
Hi @bbalavikram ,
The
framework
option may not do what you think; first and foremost,4.5
is not a valid option for framework. You need to use a "framework monifier" that's defined here:
https://learn.microsoft.com/en-us/dotnet/standard/frameworksBut keep in mind that framework monifier must also be in your project file. The
framework
argument fordotnet
simply selects which of the frameworks in your project file to build. It's really only useful for multi-targeted builds, which you probably don't have.It's possible that
dotnet
simply will not work with your project. This is unfortunately the case with many old projects. You can continue to try to "play" with your csproj files to try get it to work (note: you can run the samedotnet
commands on your workstation).If you can't get it to work, then you'll need to use
MSBuild:Build-Project
orDevEnv::Build
. We do not have script templates from these, but you can convert your script template to OtterScript and then try modifying the script that way.Here is some information on build scripts:
https://docs.inedo.com/docs/buildmaster-platforms-dotnet#creating-build-scriptsBest,
Steve -
RE: ProGet internal webserver HTTP->HTTPS redirection
Hi @jw ,
We'll add this as a "if time" on our ProGet 2024 roadmap... and hopefully it's as simple as just setting that flag. We'll update as we get closer to the final release.
Cheers,
Steve -
RE: Build execution failure for .Net projects
Hi @bbalavikram ,
This error message is coming from the
dotnet
command-line tool, and I think it has something to with an old/legacy project file format. If you were to run the same command on your workstation, you would get the same rror.From here, I would compare/contrast the .csproj files in the broken projects, and see if you can figure out what's the difference.
Note that if you search "root element is visualstudioproject expected project", you'll see a lot of people have a similar error, but their solution is also to do similar things - i.e. edit the file and fix the format.
Once you fix the project file, if you check the code back into Git, it shoudl work the next time.
Best,
Steve -
RE: Debian Feed - Package can only be downloaded by apt once
Hi @dan-brown_0128 , @gdivis ,
We received a support ticket that seems to be a similar topic...
I am using a ProGet Debian feed to mirror http://deb.debian.org/debian. I am using it on Debian containers running on my desktop. I have noticed that packages that I had installed already once are no longer available for a second installation. I have attached a screenshot that uses the command "apt update && apt install -y git" as example. According to the error message, the package "git" is unknown.
However when looking it up in the ProGet feed, it is listed as a cached package as it was installed in an earlier execution of the container. I have then set a retention rule on the feed to delete absolutely all packages and run the job manually to clean all packages cached on the feed. When re-executing the exact same command, the package git and its dependencies are found and installed.It appears that cached packages are no longer returned to the Linux instance when executing "apt update". I can force a complete cleanup of the ProGet feed prior to an installation as a workaround but this is a bit tedious.
So I wonder if this has something to do with how we're generating indexes, or something to that effect?
I also asked the other user for input - but we're currently stuck trying to reproduce this, so any other insight would be helpful.
Steve
-
RE: Debian Feed (New) connector errors
Hi @dan-brown_0128 ,
The error that you discovered is about all that would be logged on ProGet; you'll need to use a tool like ProcMon to monitor file system activity for the processes.
Best,
Steve -
RE: Debian Feed (New) connector errors
Hi @dan-brown_0128 ,
I'm afraid that's not really possible; the sqllite databases are stored in the package store, which needs to be shared/common across servers.
Cheers,
Steve -
RE: [BM] Getting 500 Error when accessing Global Pipelines
Hi @andy222 ,
This was fixed in 2023.11 via BM-3933; as a work-around, you can see the global pipelines by clicking "Shared (Global)" under any application > settings > pipelines.
Cheers,
Steve -
RE: Debian Feed (New) connector errors
Hi @dan-brown_0128,
Hmm, that's strange, and I can't imagine anything in ProGet that would yield the behavior that you're describing. We've been following this pattern for a few years now and haven't had any issues like this. Behind the scenes, here is what's happening:
- If the
index.sqllite3
file exists, then ProGet attempts to opens it - If the file cannot be opened (corrupt, old schema version, etc), then it is deleted
- ProGet then instructs SqlLite library to open a database with the file;
- If the file does not exist, SQL Lite will create it
There's really no difference between writing to a disk path or a UNC path - that's all handled at the operating system level. Few things to note...
- A health check for a Debian2 feed simply downloads and updates the index file - i.e. it's the same thing that happens when you browse the feed the UI.
- Connector Health Check is performed by the ProGet Service, which is a separate program than the ProGet Web application
Cheers,
Steve - If the
-
RE: Debian Feed (New) connector errors
Hi @dan-brown_0128,
ProGet uses SqlLite databases as a cache for a variety of operations. I would first try to delete and the recreate the connector, to see if it makes any difference. That will create a new disk path.
If ProGet does indeed have full control over that directory, you'd need to use some sort of file system monitoring tool like ProcMon to see what activity is happening when attempting to create/open the database file.
It wouldn't surprise me if there is some kind of "malware" or "rootkit" installed on your server that is interfering with ProGet's operation. We see these cause all sorts of strange problems, especially when they start "quarantining" package files that users upload because they contain "dangerous executable code" like
.dll
filesBest,
Steve -
RE: Debian Feed (New) connector errors
Hi @dan-brown_0128 ,
Thanks for the stack trace and additional information; that definitely helps pinpoint the issue.
Debian require that the entire remote repository is downloaded (which can sometimes take more than 10 seconds on slow connections), and this repository information is cached in a disk-based SQL Lite database. And that error message is coming from SqlLite, which is implying some kind of file access issue.
The database is stored in
«Storage.DebianPackagesLibrary»\C«connector-id»\index.sqlite3"
, and you can find the values under Admin > Advanced Settings.Do you see that
index.sqllite3
file on disk? Any reason that ProGet wouldn't be able to write to that directory?Thanks,
Steve