Hi @c4buildmasters_2588

Short answer yes, and you'd probably see a bit better than 15 -> 5 TB reductions with those artifacts. We usually see 90-95% storage space reduction. Pair it with ProGet's retention rules and I wouldn't be surprised to see that drop to 500GB.

Long answer, file deduplication is something you want handled by the operating system (e.g. Windows Data Deduplication, RHEL VDO, etc), not the application. It's way too complex -- you have to routinely index a fileset, centralize chunks in a compressed store, and then rebuild those files with reparse points.

Maybe this wasn't the case a couple decades ago. But these days, rolling your own file deduplication would be like implementing your own hacky encryption or compression. Pointless and a bad idea.

That being said, you may be using a tool by our friends at JFrog. They advertise a feature called "data deduplication", which IMHO is something between deceptive and a clever marketing flex.

Because they store files by their hash instead of file names, the files are automatically "deduplicated"... so long as it's the exact same contents. Which, in most case, it will not be.

Here’s an article that digs into how things are stored in Artifactory, and also should give you an idea of their “file-based” approach: https://blog.inedo.com/proget-migration/how-files-and-packages-work-in-proget-for-artifactory-users/

As for the package count, 5M is obviously a lot of packages. Obviously it's not going to be as fast as 5 packages - but probably not that much noticeably slower. There's lots of database indexes, etc.

Hope that helps.

-- Dean

@michal-roszak_0767 just a heads up we're a bit slammed with ProGet 2025 release but will respond soon!

@lukas-christel_6718 just a heads up we're a bit slammed with ProGet 2025 release but will respond soon!

@layfield_8963 no plans, as you're the first to ask :)

I don't know much about ARM/MacOS builds.... do you think it's just as easy as adding a new publish target?

See our build script here:
https://buildmaster.inedo.com/applications/132/scripts/all?global=False

Hi @parthu-reddy ,

Nothing to worry about - there are a few ways this can happen, and unless it's happening a lot and/or causing problems with your end-users / pipelines / etc., you can ignore the message.

-- Dean

Hi @alex_6102 ,

It sounds like you're trying to do a kind of "manual" or "custom" installation on Linux? That's the impression I got when you mentioned, "using the file system provided..."

We don't support this kind of installation,; you should really just run the Docker image like this:

docker run -d --name=proget --restart=unless-stopped \
  -v proget-packages:/var/proget/packages -p 80:80 --net=inedo \
  -e PROGET_SQL_CONNECTION_STRING='Data Source=inedo-sql; Initial Catalog=ProGet; User ID=sa; Password=«YourStrong!Passw0rd»' \
  proget.inedo.com/productimages/inedo/proget:latest

As for the error... it seems that ProGet is failing to read the configuration file, which isn't used on a Linux installation. Instead, environment variables are used, since that's the Docker way.

-- Dean

Hi @darren-sloper_5044 ,

That's great to see you're giving it a shot! We'll fix this via PG-2992 in the next maintenance release, but in the meantime... it looks like the bug is in the download statistics procedure, so if you disable that feature on the feed it should work.

Let us know what else you find,

-- Dean

@michal-roszak_0767 said in Pull Maven artifacts - invalid version:

Next victim:
https://repo1.maven.org/maven2/org/springframework/data/spring-data-releasetrain/

jeeze, what a mess!

Well, there goes any hope of using v[0-9]+ -- they just straight up use random strings as version numbers.

• https://repo1.maven.org/maven2/org/springframework/data/spring-data-releasetrain/.
• https://repo1.maven.org/maven2/org/springframework/data/spring-data-releasetrain/Neumann-SR9/

Open to ideas, but based on the URLs alone... I don't see a good way to identify one of these as an artifact and the other as a version of an artifact.

Hi @michal-roszak_0767 ,

ProGet does not support wildcards searching for artifacts.

Licenses are declared in the manifest (i.e. .pom file):
https://maven.apache.org/pom.html#Licenses

You cannot really override this. If an artifact does not have a license, you will be given a chance to pick a license for it. If you ever need to change that, you'd have to go to the SCA > Licenses > License Types, and remove the package-specific assignment from there.

-- Dean

Hi @sergio-gonzalez_0157 ,

Typically, Deployment Targeting is done in the Pipeline:
https://docs.inedo.com/docs/buildmaster/deployment-continuous-delivery/buildmaster-pipelines#deployment-targets

This way, you don't need to put for server or anything in your script.

So, my guess is that your Pipeline is actually targeting the ProductionServer (but not running anything on it, except intializing the agent), but your script is targeting the BuildMasterServer.

-- Dean

@michal-roszak_0767 ProGet is not a file server. Metadata files like maven-metadata.xml are generated upon demand, based on artifacts stored in the feed

Hi @michal-roszak_0767,

After looking into this further, I'm afraid we simply can't support this artifact/package at this time. I don't really see a good path for supporting this without adding significant complexity and risk of breaking proper artifacts / versions.

The problem is that this version breaks the basic rules that Maven repositories need to follow:

1. Artifact Names / Groups cannot start with a digit
2. Versions must start with a digit

These rules resolve the ambiguity of determining what /com/google/javascript/closure-compiler/v20250407 means. For example, is it:

• Files for version v20250407 of com.google.javascript.closure-compiler artifact?
• All versions of the com.google.javascript.closure-compiler.v20250407 artifact?

I'm not even sure how this was uploaded to Maven central. I have no idea why the developers ignored the warnings that Maven spat out for legal version numbers. This has been a specification for like 20 years. Heck, here's a discussion from like 2008 on how the "must start with a digit" rules needed clarification: https://cwiki.apache.org/confluence/display/MAVENOLD/Versioning

If you encounter other artifacts like this, maybe we can consider some kind of very limited exception, but until we figure something else out this artifact version is simply not supported in ProGet.

I can't imagine there are many other artifacts like this, but let us know if there are.

-- Dean

Hi @michal-roszak_0767,

Oh that's a whole lot of connectors and this is most definitely going to cause some performance issues.

Remember that ProGet needs to forward every request you make to all 10 of those servers, and some of these repositories will not respond very quickly. Like JCenter was deprecated/retired a few years ago, and I can't imagine is performant at all. Especially for things like metadata requests.

Maven is not a very patient client and will time out while waiting for ProGet.

There is really no way around this. You'll need to use less connectors.

-- Dean

Hi @michal-roszak_0767 ,

That error is unrelated to invalid versions being allowed/disallowed in ProGet. Maven is just saying that it can't find a snapshot (i.e. prerelease) version of a dependency.

'io.github.java-diff-utils:java-diff-utils' is a public library, published to Maven Central:
https://repo1.maven.org/maven2/io/github/java-diff-utils/java-diff-utils/

Snapshot versions are not published to Maven Central. I don't know where it's published.

In any case you should not be using snapshots of public libraries unless you have a very specific need to; they're only intended for development of related public libraries and are in a special repository. Check w/ the devs behind that build with their intents... it might be a mistake?

-- Dean

@michal-roszak_0767 those are also invalid versions and you should never upload them to a feed (repository) directly; see https://docs.inedo.com/docs/proget/feeds/maven#snapshot-versions

The code has already been fixed and the maintenance release is scheduled for next week. We could get you a prerelease, but I don't think your developers are manually uploading artifacts using the Web UI?

If you want to upload artifacts with bad versions now you can just use the maven client (e.g. maven-deploy) or just do a basic PUT of the file to the desired group/artifact-id.

Hi @michal-roszak_0767 ,

It looks like the manual upload page does not consider that setting; we'll fix it via PG-2977 in the next maintenance release.

-- Dean

Hi @michal-roszak_0767 , @steviecoaster ,

Maven versioning is a total mess. v*** is indeed an invalid version, per the which means that (among other things) it must be lexicographically sorted for determining latest version.

It looks like they knew that enough to use a string... but not enough to use a valid version number. Oh well. /rant

If you go to Manage Feed Settings, you can enable invalid versions in the feed. The message should make this more clear, so I will clarify that via PG-2977 in the next maintenance release.

-- Dean

Hi @steviecoaster,

The current package import tool uses the NuGet API. It's not really easy to use, and I'm afraid our API access code isn't really "portable" -- it's tightly integrated into Connectors, which are tightly integrated into Feeds, etc.

Here's a guide on how to query all published packages from a NuGet feed:
https://learn.microsoft.com/en-us/nuget/guides/api/query-for-all-published-packages

That said, next week we will be releasing a brand-new package importer that will connect to Sona Nexus, Artifactory, AzureDevOps, ProGet, GitHub, and GitLab. These use the provider-specific APIs and work much better than what we have now.

Functionality it's the same, but now your credentials are stored in ProGet. You can also run it multiple times, and it will only import new packages. This is useful for the cases where you are transitioning usage.

-- Dean

Hi @michal-roszak_0767 ,

Thanks for letting us know!

Unfortunately the Feed Wizard seems to have some quirky behavior when configuring certain combinations of options, as you've noticed. We are actually in the process of rewriting the new feed wizard to be a bit more simpler (esapecially behind-the-scenes), hopefully in the next couple weeks it'll be in a new maintenance release.

In the meantime, if you encounter these errors... I would just create the connectors on the MAnage Feed page. Which it sounds like you've done :)

-- Dean

Hi @daniel-pardo_5658 ,

The UI-based package editor is intended for small packages, up to 50 MB or so. It looks like there is a platform-enforced limit of 2GB. For now, you will need to download, edit, re-upload.

That said, I switched the stream that we're using to something that can accommodate larger packages, but I did not test it so I really don't know if it will actually work on these packages. It'll be in the upcoming maintenance release via PG-2964

-- Dean