Posts made by atripp

Hi @m-lee_3921 ,

Thanks for the heads-up; we'll get this fixed via PG-3059 in the next maintenance release (scheduled for later today).

And you're right -- its basically the same issue. SQL Server silently truncates string values whereas PostgreSQL does not, hence these errors.

We decided to not auto-truncate, since it's better to know about these cases than not. In this case, it's a IPv6 address that was longer than the 20-character limit. We plan to refactor API key logging at some point, so we'll just continue to truncate v6 IPs.

Thanks,
Alana

Hi @carsten_1879 ,

Can you try out buildmaster:25.0.0-ci.23 container image?

I would do a new/fresh installation - new applications created in it should use the new Git library. It seems to work, but would like to see if it works for you as well.

Thanks,
Alana

@it_6176 thanks for sharing!

You're right, we did not consider this usecase when designing Debian connectors. Our "default mode" in ProGet is "dynamic indexes", but the quirkiness of the Debian repository "API" and obscene size of operating-system libraries make that impractical when there is a lot of feed activity (i.e. packages being added to the feed via caching).

We are going to redesign this at some point, and allow for some kind of "periodic indexing" for some connectors. That would be run as a job, as opposed to triggered by a user's request.

As for the database.... we've had one other user report that those SQLLite databases "blew up" as well, and our conclusion is that "something" crashed at the wrong time. Probably, due to the resource overusage? It has something to do with the WAL Files, but we're not totally sure.

I think we just need some kind of regular clean-up for these "huge" connectors so the database doesn't get so huge.

Hi @caterina ,

I agree and there doesn't seem to be any good usecases or official guidance from npm tags command, but we do not recommend using them: https://blog.inedo.com/npm/smarter-npm-versioning-with-semver/

From the api/technical standpoint, npm tags are a little weird -- a package themselves doesn't contain tags. Instead, the registry has a key/value tag store. By convention, the store has keys like latest and values like 4.8.1, but it seems to allow anything? At least from data we've seen on npmjs.org

In any case, the npm client seems to set a latest tag, so we'll just support that use case then.

FYI here is the logic/code I added:

        var tags = await this.GetPackageTagsAsync(purl.Name, false, cancellationToken: cancellationToken);
        if (tags.TryGetValue("latest", out var latestVersion) && latestVersion == PackageVersion<SemanticVersion2>.Parse(purl.Version))
            await DB.Npm_SetTagVersionAsync(toFeed.FeedId, purl.Name.Id, "latest", purl.Version, DateTime.UtcNow);

It sets the latest tag on the other feed if the current feed's latest tag points to the package you are promoting.

Thanks,
Alana

Hi @caterina ,

This behavior is expected as only the package itself is promoted, not repository metadata.

I'm not an npm developer, but one concern I have is that the latest tag doesn't always refer to the most recent version of a package? It's a little confusing to be honest, but I read that developers will sometimes use latest to "pin" what they feel is the most stable version?

I don't understand why this is a practice in npm, but I think this is probably an exception. So, we will change the behavior via PG-3057 such that the "latest" tag is set on the new feed when promoting latest-tagged npm package.

This will be in the next maintenance release of ProGet 2025.

Cheers,
Alana

Hi @phopkins_6694

Looks like this is a regression in ProGet 2025; we'll get it fixed in the next maintenance releaese, scheduled for this Friday via PG-3056

Cheers,
Alana

Hi @kc_2466 ,

Can you share how / why you're create Universal Packages in the UI? And perhaps how you're using upacks in general?

We added them as a way to introduce the format to new users, but didn't really anticipate it being used beyond that.

Thanks,
Alana

Hi @kc_2466 ,

We'll Add support for LC_TIME environment variable & user setting via PG-3054 in an upcoming maintenance release -- p[robably not this week's, but in the next or following.

Thanks,
Alana

Hi @carsten_1879 ,

We should have a build of BuildMaster 2025 later this week that you can try out that includes the new library. We'll let you know!

Thanks,
Alana

Hi @imm0rtalsupp0rt ,

Thanks - I was able to reproduce this issue. It's some long-standing behavior / bug, and not a trivial fix.

We will see about fixing it in the upcoming maintenance release (PG-3048), but it's lower priority and will likely get pushed back a few releases.

Cheers,

Alana

Nice idea! That was pretty trivial to add in there, and will be in the next maintenance release via PG-3038:

Hi @misael-esperanzate_5668 ,

I'm not really sure, but I'll explain how things work so it might help troubleshoot.

First, the Maven API does not provide a file listing. While you (as a user) can often "see" via an HTML page (like this listing at jboss.org), it's simply not available for listing in the API. The only required file in an artifact is the .pom file, so when you "pull" an artifact to ProGet, that's all you'll get.

Next, the "remote" icons next to the files indicate that they were cached (i.e. added to the feed) via a connector. That means the files were successfully written to disk, recorded in the database... but now they are gone. Hence, why you keep getting the "file not found" message.

The most likely culprit for this is something deleting the files from a packagestore. We often see security tools doing that for "safety" reasons, since they are .jar files that may be dangerous I guess.

Overall, the maven API is a very simple series of GETs. So perhaps, you can just experiment with this? First, start with a brand new feed and a connector.

Then, run the command:;

curl http://my.server.local:8624/maven2/my-new-feed/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar --output hamcrest-core-1.3.jar

You should see the file download via CURL. Afterwards, you should see the artifact as a cached package (Top Navigation > Packages > Select Cached). You should also see that .jar file on disk, written to the package store location (Manage Feed > Storage).

If you don't see a .jar file downloaded to that location on disk, then it means something is "blocking" the file being written. If it's there, then it means something is deleting it after.

Thanks,
Alana

Hi @rel_0477 ,

Sounds like this is a pretty specific edge case. Can you provide a reproduction case so we can take a look?

Thanks,
Alana

Hi Dan,

In general, a ProGet feed will be slower than a "real" Debian repository. The reason is that Debian repositories are just static file system like this:
http://ftp.us.debian.org/debian/dists/bookworm/

There is obviously a lot more overhead with each ProGet request, since index files are dynamically generated, involve connectors, need to be permission-checked, etc.

In addition, ProGet indexes the Debian indexes on-demand, which means downloading all of the Contents-*.gz files and indexing those. Like in here: http://ftp.us.debian.org/debian/dists/bookworm/main/

While these files are cached, they do need to be updated when the remote repository updates.

And note that each web node maintains its own local index cache, so you will see these long-running requests multiple times. Some organizations will periodically "warm up" the indexes periodically by just hitting the InRelease endpoint.

Hope that helps,
Alana

Hi @sgardj_2482 ,

ProGet's Integrated Web Server already supports Windows Integrated Authentication (WIA). Actually it supports WIA better than better than IIS in that only the Web UI and supported feeds (or ones you configure) will do the WIA challenge.

Behind the scenes, the Integrated Web Server (IWS) uses Kestrel unless you're doing so-called "port sharing", which would be having two web applications sharing the same port. That's becoming less and less common, and isn't something Microsoft recommends anymore.

If you bind to a host name in IWS (i.e. port sharing), then the operating-system level HTTP.SYS component is used. That is much less flexible with WIA and every request must be authenticated. Which means it'll never work with Docker, npm, etc. So it's not recommended.

Thanks,
Alana

Hi @michal-roszak_0767 ,

401/403 are not logged, so you won't see a server side event.

403 means authentication was successful but the permissions are not OK

My guess is that the wrong feed or credentials are being specified. Like maybe using an API key?

Thanks,
Alana

Hi @michal-roszak_0767,

I'm not sure, but maybe it's something simple like a typo. I don't see it though.

That 401 message will occur when credentials weren't sent.

Behind the scenes, pgutil uses that API and authenticates by adding the same header:
https://github.com/Inedo/pgutil/blob/thousand/Inedo.ProGet/ProGetClient.cs#L37

Perhaps you can use a proxy like Fiddler or ProxyMan to see the difference in HTTP Traffic?

Thanks,
Alana

Hi @bill-hertzing_2810,

It sounds like you want to enable API v3, which uses the `v3/index.json' URL suffix?

In that case, make sure to set the useApiV3 property is set. Also, I don't think you can set all those properties on create... you may have to create, and then update.

Also note that you cannot set the endpointUrl property, it's just readonly. That is generated based on the incoming reques, so if you're viewing it on localhost you'll see that. If you view it on myserver.corp you'll see that, etc.

Hope that helps,
Alana

Hi @parthu-reddy ,

This looks more related to Server Overload / Database than anything to me. Note how all the requests are coming in at exact same second. Can you throttle your load balancer a bit, so they don't all hit at the same time? Even a slight delay will help.

FYI - we are tracking a recent regression to SQL Server analysis engine (???) that is causing one particular query (NuGet_GetPackage) to go incredibly slow under extreme traffic. For some reason, it's using the wrong plan suddly. It's been happening to a few users after a recent upgrade/patch to SQL Server. We have a work-around but would like to test it in the field against a user.

Thanks,
Alana

Hi @marc-ledent_9164 ,

This is available in InedoCore-3.0.4, so if you go to Admin > Extensions, you should be bale to update.

thanks,
Alana

Hi @mhelp_5176 ,

I haven't investigated BitBucket Cloud any further, but it sounds like there's an issue with the integration -- and it's definitely something we can look at later. My guess is that it's some kind of change to the API/authentication. But we're all pretty focused on getting ProGet 2025 out the door, so it'll have to be after that.

That said, the main difference between connecting to a "Git host" like GitHub, GitLab, Gitea, BitBucket, etc. vs a "generic Git repository" is that there will be some intelligent drop downs to help you select a repository. There are a few other differences as well, but mostly it's UI.

So for example, on a GitHub connection, you'll see a list of organizations and repositories, and then would select the one to connect to. Compare this to the "Generic Git repository", where you simply paste in the repository clone url.

But in either case, you need to configure each repository connection individually; typically each application will have one repository, which is why this is part of the application creation process.

It's definitely not a common practice to be able to "pull in all the repositories in a workspace" at once - that's not really how Git works, and it would involve some kind of script that iterates a list of repositories and clones/updates each one indivdiaully based on configured remotes in subfolder.,

Cheers,
Alana

Hi Fritz,

Thanks for reporting this; the underlying with the Packages index file isn't so much the migration, but the code that generates the index file on the new feed.

Basically, the control files seem to be have an extra new line that isn't being trimmed. The old feed code must have removed them.

Anyways we will address this via PG-2985 in the next maintenance release or you can try the proget.inedo.com/productimages/inedo/proget:24.0.37-ci.2 container now.

Cheers,
Alana

Hi @michal-roszak_0767 ,

Looking over the code, the API Key Access Logs contain the api-key sent by the user, which is being displayed there. In retrospect, this isn't a great design...

It would be technically possible to link that to the "Edit API Key Page" itself, which would show you the description etc.... just a matter of whether that's worth us doing.

Thanks,
Alana

Hi @mhelp_5176,

Based on the error, it sounds like there is some kind of configuration error / invalid response coming from the BitBucket Cloud API. It's possible the API/authentication policies changed -- we honestly don't really test against BitBucket Cloud, just BitBucket Server.

How about just using a Generic Git Repository? That uses a different API (the Git API) , so that might just work simpler/easier for you.

Thanks,
Alana

Hi @marc-ledent_9164 ,

Yes, it's a somewhat common configuration; I would go to Admin > Security, and you can add a new directory/domain connection. Note you'll need information like a domain host, name/password, etc.

Thanks,
Alana

Hi @steviecoaster ,

Under Admin > Service, you should be able to see some logs being printed out. The DropPathMonitor will run periodically (or you can manually run it), and then you'll see some kind of log messages pertaining to what's happening.

It could be a lack of permissions, package already exists, etc.

Cheers,
Alana

Hi @misael-esperanzate_5668 ,

The "File not found on disk" message means that the file was indexed in the database but it's not on disk. This can happen when files are directly deleted from the packagestore, removed by a quarantine tool, disk paths changed (common on Docker, if you mount wrong directory), etc.

Hopefully that makes sense and will help you troubleshoot from here. You can try to clear the cached files, run a feed-reindex, etc. But if all of your files are missing then that's probably not what you want to do.

Thanks,
Alana

@stefan-hakansson_8938 thanks for the heads up! We'll update this via PG-2982 in the next maintenance release to align with our docs page, which doesn't recommend apt-key

Hi @parthu-reddy ,

These are stored in the database and if you click "status" you can see the cached queries.

Does this metadata caching feature reduce the queries to fetch versions that you mentioned?

Yes, instead of going to NuGet.org, the database will be queried.

HOWEVER, this may actually reduce performance; NuGet.org is a static file system running on Azure Blob storage, and it might be quicker to download the query from Azure than to query your SQL Database.

You'd really need to expirement and see which is faster / more load.

Thanks,
Alana

@michal-roszak_0767 it wouldn't hurt to start a new topic about it, but it's design decision so you'd have to make a good use case for it :)

I can't find the discussion (maybe it was internal), but the idea is that "generic website connectors" for Asset Directories are a poor solution for "offline" environments. A "generic proxy" like ngnix will always be better in every way.

Hi @michal-roszak_0767 ,

Ah, that's what I thought you were asking in the first place :)

No, that's not a supported use case, I understand it's a design decision

Thanks,
Alana

Hi @michal-roszak_0767 ,

Thanks for clarifying; I thought you were looking to "proxy" a website like that...

ProGet has Asset Directories, which basically acts like a web-based file system.

Thanks,
Alana

Hi @michal-roszak_0767,

By RAW repository, I think you mean like a website / web-based file system?

In any case, that's not something ProGet is designed to do -- I guess that'd be more like a "generic HTTP proxy server" type of tool?

Cheers,
Alana

Hi @michal-roszak_0767 ,

Just as an update, we've added this to our roadmap planning for ProGet 2026.

We will begin that process later this year, after ProGet 2025 has been released (see road to ProGet 2025).

Cheers,

Alana

Hi @carsten_1879 ,

Can you try using buildmaster:24.0.5 instead of latest? There was a change in the Git library, and I'm wondering if this is a kind of regression on some platforms.

Thanks ,
Alana

Hi @pmsensi,

I'm sorry I'm not sure. The error HTTP_1_1_REQUIRED indicates that an intermediary (e.g., proxy or load balancer) is rejecting HTTP/2 requests and requiring HTTP/1.1.

So, I would check with the proxies or load balancers between Kaniko (running in your GitLab pipeline) and ProGet. I suspect it's interfering with HTTP/2 traffic, causing the server to fall back to HTTP/1.1.

Alternatively, you can try forcing Kaniko to use HTTP/1.1 instead of HTTP/2. I don't know how to do that, but there may be an --insecure flag that may also work.

Thanks,
Alana

Hi @guilherme-silva_2532 ,

This error means there's some kind of problem/issue with the format of the blob file. Tar is a very finicky format, so we'd need to be able to recreate this in a lab/debug setting.

If you can create a simple repro case then we'd be happy to investigate further.

As for vulnerabilities, each version of ProGet ships with an offline database. So you should be fine as long as you upgrade semi-regularly.

Cheers,
Alana

@michal-roszak_0767 currently maven2 is a workaround, so it will continue to warm. In ProGet 2025, Maven will create the expected feed type.

Hi @michal-roszak_0767,

This is what I meant by the Security API -- a combination of HTTP Endpoints and pgutil commands. We will consider them for our ProGet 2026 roadmap.

So until then you'll need to use the Native API; At this time the Native API is the option.

You can also check out @steviecoaster 's PowerShell module, which has some modules that can help https://github.com/steviecoaster/InedoOps

Thanks,
Alana

Hi @itadmin_9894

It looks like you're trying to do a GET request, which is why it's not working.

You need to POST an array of package identifiers like this:

POST /api/sca/audit-package-vulns
[ 
    {
        "name": "NuGet.CommandLine"
        "version": "6.8.0"
        "type": "nuget"
    }
]

Cheers,
Alana

Hi @michal-roszak_0767,

Just an FYI, we discussed creating/documenting/etc. a new security API (users, groups, permissions) for our ProGet 2025 roadmap, but decided against fitting it in. We'll reevaluate when it comes time to planning out ProGet 2026!

Thanks,
Alana

Hi @michal-roszak_0767,

Ah, that must have been an oversight; trivial change but best we don't change it until ProGet 2025.

I think you can use maven2 as the type for now, it's the internal identifier. Later on, we will make it maven and mavenlegacy

Thanks,
Alana

Hi @itadmin_9894 ,

It doesn't seem to be documented, but we do have a pgutil packages audit command:

$> pgutil packages audit --help

Description:
  Analyzes a package for compliance issues

Usage:
  pgutil packages audit [options]

Options:
  --package=<package> (REQUIRED)  Name (and group where applicable) of package
  --version=<version> (REQUIRED)  Version of package

Common Options (packages):
  --api-key=<api-key>             ProGet API key used to authorize access
  --feed=<feed>                   Name of feed in ProGet
  --no-connectors                 Only include local (non-connector) package data in results
  --password=<password>           ProGet user password used to authorize access
  --source=<source>               Named source or URL of ProGet
  --username=<username>           ProGet user name used to authorize access
  -?, --help                      Show help and usage information

Examples:
  $> pgutil packages audit --feed=approved-nuget --package=Newtonsoft.Json --version=12.0.3
  $> pgutil packages audit --feed=public-npm --package=@babel/runtime --version=7.25.0
  $> pgutil packages audit --feed=private-pypi --package=Django --version=5.0.6 --filename=Django-5.0.6.tar.gz

Cheers,
Alana

Hi @michal-roszak_0767 ,

This is a third-party / community plugin and we have no plans to maintain it ourselves. I hear it still works, but is a bit outdated.

HOWEVER, the former owner very recently "handed it over" to another community member. So this may mean we'll see some "new life" or changes -- I think they're still figuring out how to actually publish a new version, or something like that:

https://github.com/jenkinsci/inedo-proget-plugin

Thanks,
Alana

@michal-roszak_0767 only what's shown there is currently supported.

I saw that @zs-dahe is is looking for a the RetentionActive setting as well (see https://forums.inedo.com/topic/5341/retention-activation-via-api), so it would probably easiest to do these at once: I'm almost certain this is done at the ProGet level, not pgutil.

Can you open a new thread w/ the requested fields/properties? This will make it much easier for people searching in the future :)

Sounds like a plan, @c4buildmasters_2588 !

In case you haven't seen it, it's "relatively easy" to load a custom extension:
https://docs.inedo.com/docs/inedosdk/extending-inedo-tools-using-the-sdk/inedosdk-extending-creating

For testing this change, I would just build/pack it with a higher version number than the official AWS extension - if all looks good on our end (i.e. won't break existing installs, etc), then we'll likely be able to accept pull request and then publish an official new version. Then you can delete your custom one.

Thanks,
Alana

Hi @michal-roszak_0767 ,

We do not have a non-Native API to assign privleges at this time, but otherwise our documented APIs are here: https://docs.inedo.com/docs/proget/api

Here is the info about the Native API:
https://docs.inedo.com/docs/proget/api/http-requests#native-api-endpoints

Thanks,
Alana

Hi @c4buildmasters_2588 ,

I haven't heard of MinIO before, but they claim to be "S3 Compatible Storage" so maybe it'll work?

I also don't know what "virtual path support" or "path style" means, but I can say that several users have had success with other S3-compatible services, while other services have had bugs and/or aren't actually S3-compatible. Best I can say, give it a shot and see what happens.

You can see how we use the AWS SDK and what our code looks like over here:
https://github.com/Inedo/inedox-aws/blob/master/AWS/InedoExtension/FileSystems/S3FileSystem.cs

No idea what to enter in any of the fields, but I suppose you would just enter the CustomServiceUrl ("Advanced" tab).

If you get it working let us know, would be nice to have a follow-up post here in case anywone else searches for it :)

Cheers,
Alana

Hi @michal-roszak_0767 ,

I assume you mean, why don't we recommend using the Native API?

We don't recommend using the Native API because we do not document usage and the methods are subject to removal or change, even in maintenance versions.

In this case, we do not have an alternative API for getting security privileges, so the only option is Security_GetPrivileges if you want to do that programmatically.

Thanks,
Alana

Hi @zs-dahe ,

This isn't currently supported via the api / pgutil feeds update command. Let us know if you'd like us to consider this a feature request and we could likely add it an upcoming maintenance release :)

Cheers,
Alana