Thanks, looking forward to the next release!

With the 'constrains' option working, we can really use ProGet as our main method of distributing (python)packages within our company. It currently works great for most packages, except for so-called meta-packages with optional dependencies.

Meta-packages are nothing more than a list of dependencies. The great thing about them is that they can very effectively "lock" environments meaning that users cannot (either by accident or on purpose) install different versions of packages critical to our workflows. The anaconda package is also a meta-package. If you want to install some package with a different version than stated in the meta-package, you will need to uninstall that meta-package first.

But there are plenty of cases where you don't want users/developers to have to install all 400+ packages of interest each time they create a new conda environment. For that, the "constrains" option comes in handy: we still strongly fixate the versions of certain packages, but they do not have to be installed directly. Only when a user needs such a package later on, he will get the version we want him to have. This is great when someone starts a new project: initially they only need pandas and/or numpy, but later they want to try some machine learning and need scikit-learn or pytorch.

Good luck with the development of version 2023, I'll keep an eye on new releases.

Thanks for the quick reply! We'll await the next release then.

In Proget 2024.25, when I upload the "pint" package from this webshare:
https://nextcloud.marin.nl/index.php/s/nmPCLszf6PNMoQC

It shows up in the WebGUI with a *.tar.bz2 extension rather than with a .conda extension. The file I upload - as you can see in the webshare folder - has the .conda extension (and it should stay that way when uploaded to ProGet).

This is only a problem with the WebGUI I think, because when I use conda search to search for this pint package on my channel, I get the proper URL:

But in the WebGUI it looks like this:

And when clicking on the file to download, it will try to download http://proget.marin.local/conda/conda_erik_2/noarch/pint-0.24.4-pyhd8ed1ab_1.tar.bz2 which does not work. But when I change the extension to .conda it will work.

We have proget 2024.25.
I have a conda feed where I uploaded the package files in this folder:

https://nextcloud.marin.nl/index.php/s/nmPCLszf6PNMoQC

However, only the "pint" package is visible in the WebGUI, the others are not. My feeling is that there's something about the version normalization that prevents the packages from showing up in the GUI.

The repodata.json files in the win-64 and noarch subfolders look OK though. So when using the Proget channels for installing packages with conda, all seems to work fine.

repodata.json in noarch subfolder:

{"info":{"subdir":"noarch"},"packages":{},"packages.conda":{"pint-0.24.4-pyhd8ed1ab_1.conda":{"build":"pyhd8ed1ab_1","build_number":1,"depends":["flexcache \u003E=0.3","flexparser \u003E=0.4","platformdirs \u003E=2.1.0","python \u003E=3.9","typing_extensions \u003E=4.0.0"],"constrains":["numpy \u003E=1.23"],"license":"BSD-3-Clause","license_family":"BSD","md5":"a566694ac0ab8f25e7f40a5d24070a1a","name":"pint","sha256":"4595b54c19a46a8fc320d01e71000cee8bbfa47d9494fd2c8041d5c86f721b09","size":230217,"subdir":"noarch","summary":"Operate and manipulate physical quantities in Python","timestamp":1733663596173,"version":"0.24.4"},"tzdata-2024b-h04d1e81_0.conda":{"build":"h04d1e81_0","build_number":0,"depends":[],"constrains":[],"license":"CC-PDDC OR BSD-3-Clause","license_family":"BSD","md5":"9be694715c6a65f9631bb1b242125e9d","name":"tzdata","sha256":"9fdd287b55be4c475789a69d3b94cdb73f756583a6d7306da1706e43eee573da","size":117432,"subdir":"noarch","summary":"The Time Zone Database (called tz, tzdb or zoneinfo)","timestamp":1728062827250,"version":"2024b"}},"repodata_version":1}

repodata.json in win-64 subfolder:

{"info":{"subdir":"win-64"},"packages":{},"packages.conda":{"re2-2022.04.01-hd77b12b_0.conda":{"build":"hd77b12b_0","build_number":0,"depends":["vc \u003E=14.1,\u003C15.0a0","vs2015_runtime \u003E=14.16.27012,\u003C15.0a0"],"constrains":[],"license":"BSD-3-Clause","md5":"6c4e835222173ecd5bf0a39896690470","name":"re2","sha256":"367717268b970471cd38cc0435fcede36b1cc74343e3245c48d05239f6b94479","size":382871,"subdir":"win-64","summary":"RE2 is a fast, safe, thread-friendly alternative to backtracking regular expression\nengines like those used in PCRE, Perl, and Python. It is a C\u002B\u002B library.\n","timestamp":1652432364793,"version":"2022.04.01"}},"repodata_version":1}

Thanks for looking into this!

Thanks for putting this on the issue tracker!

As for "app_own_environment": I only found it in the sourcecode of Anaconda Navigator - a GUI package manager for conda environments. It seems a useful feature but I cannot find any other package that uses it.

So if you cannot add "app_own_environment" without having extra info, you can just leave it out. It was just a nice-to-have.

Looking forward to a new version with "track_features" added to the index output!

A good example for "track_features" is this package: https://anaconda.org/main/_low_priority, which directly explains the idea of "track_features": packages that "activate" features (either because they have "track_features" specified themselves or because their dependencies do so) will get a lower priority. The attribute is useful when you have a GPU build and a CPU build of the same package, and you want to prioritize one over the other.

In our case, we wish to do exactly that: use track_features to prioritize a CPU version of a certain package over a GPU version because the former runs on any PC, while the latter only works with a proper GPU available.

Some additional reading material is here: https://docs.conda.io/projects/conda/en/stable/user-guide/concepts/packages.html#track-features

In any case, the "track_features" attribute can be found in the index.json file that is in a conda package.

About the meta.yaml: that is a specification/configuration when building a package. The build process then generates the index.json file (and the about.json) and puts it in the package (under the info folder) for repositories to extract useful information from. I think that for ProGet, the index.json is more important than the about.json.

Finally, this method gives a quite complete overview of all keys that may be in the index.json. They are not all neccessarily there, though. Here is the method/code:
https://github.com/conda/conda-build/blob/8df6493565dae517165754bd75d7d74de1522dc3/conda_build/metadata.py#L1803

Hope this helps!

First of all; we've been using ProGet for our more-and-more internal package distribution for more than a year now, and pretty happy about it!

I recently tried to add some more metadata to a package, but two items are not taken over by ProGet: "track_features" and "app_own_environment". The first is simply a string, the second is a boolean. Both are available in a conda package's "index.json" file.

Can these be added? Thanks!

Sorry for my late reply, but we have since installed version 2023.16 and repodata.json works perfectly fine now! Thanks for including this in your updates.

Conda uses the "If-Modified-Since" header to know if the repodata should be downloaded (or not), but it seems the ProGet server does not provide the correct value for the "Last-Modified" response header.

I just uploaded a new package to a channel (both via API and a different one via manual upload), but the Last-Modified header says repodata.json (and channeldata.json) were last modified on February 23....

Thanks for looking into this!
We're currently on ProGet 2023.16

Thanks for the effort again!

I just tried with one of the numpy packages on the main conda channel:
https://repo.anaconda.com/pkgs/main/win-64/numpy-1.23.5-py39h6917f2d_1.tar.bz2

And I get the same problem with the win-64/repodata.json URL.

It went okay with one of our own packages though (but not with another). But even after comparing the metadata for both packages in detail I was not able to find problematic data or characters (none that I could think of).

Looking forward to the fix!

Conda has the concept of package builds on top of package versions. I can image other package managers offer similar features. The idea is that you can build your package in different ways, linking it to different versions of binary libraries, for instance. You keep the same version because your source code is the same, but the build-string (conda generates a hash from your dependencies) differs, and you produce multiple packages.

This seems to work OK-ish now in ProGet. Using the manual upload in the WebGUI, I can upload a package with a version that is already on the server, but with a different build-number/string. But if I try this via CURL (and an API key), it does not work.

I uploaded some example files here:
https://nextcloud.marin.nl/index.php/s/7zBHSDx8QYLCzfJ

If you create an empty feed, then first upload proget-package-2023.8.0-1.tar.bz2, that will work fine using either CURL or manual upload in the WebGUI. But if you then try to upload proget-package-2023.8.0-3.tar.bz2, it will work in the WebGUI but not via CURL. Via CURL, I get the error:

The package win-64/proget-package.2023.8.0 already exists and the current user or service does not have the Feeds_OverwritePackage privilege.

I am using a feed API Key with CURL (so --user api:API_KEY) for which all permissions have been enabled. But there should be no message of 'overwriting' a package at all, because it is a different build.

Thanks again for looking into this. If you need more information, I'm happy to help.