Posts made by MaxCascone

@matt-janda-kingston_9734 my old friend! I am just now seeing this. I hope you are well! Hit me up at my email in my profile if you see this!

Hi, I like the idea of Tile View, but it doesn't seem to work in my install. I am on Version 2023.22 (Build 6).

In an RPM feed, it shows tiles, but no icon, just text.

In an Asset feed, it has no effect - the Asset directory is always shown in List view.

Settings:

RPM Feed:

List view has an icon:

Asset dir:

It appears this has been implemented and released!
https://inedo.myjetbrains.com/youtrack/issue/PG-2530

@hwittenborn Thanks! I had to install binutils first, and then got what looks like a valid tree in both packages I had:

root@e6c7c176bb0d:/tmp# ar tf NessusAgent.deb 
debian-binary
control.tar.xz
data.tar.xz

root@e6c7c176bb0d:/tmp# ar tf NessusAgent-10.4.2-ubuntu1404_amd64.deb 
debian-binary
control.tar.gz
data.tar.gz

The former is a repackaged version of the latter; using @luke_1024's mechanism above.

FWIW, ar tf was not a valid command on MacOS (bash 5.2) with binutils installed, but works as expected on the Ubuntu container (above).

I too am having this problem right now, and I don't even know how to look into the package to see if @hwittenborn 's fix is possible.

The file in question is not a cargo rust artifact but pulled from a provider's website.

I know the package still installs successfully, so as a workaround, I plan to put the deb pkg in ProGet as a simple file to be downloaded to the target.

@atripp I'm sorry to revive this old thread but I am having this exact same problem. Old email is <<redacted>>, I no longer work there; new email <<redacted>>. I think two new accounts got created somehow due to my failing efforts. Please help?

On my MacBook Pro M1, I was trying to follow the ProGet Docker-Compose instructions, but the SQL container was continuously restarting.

I found the answer here: https://github.com/microsoft/mssql-docker/issues/668#issuecomment-1436802153

TL;DR: In the Docker Desktop app, enable "Use Rosetta..." in the "Features In Development" settings pane.

Voila', it works!

@stevedennis , is there perhaps a switch or param that can enable no-ssl-checking in the upack cli? Somethink like -k --insecure in curl?

Hi, I'm just using curl to workaround the connectivity issue.

It works fine locally but I found that one of our Jenkins agents is rather old and doesn't have curl on it. As a workaround to that, for the time being, I'm using the curl installed as part of git:

This is the Jenkins code:

def publishNewPackage(String path, String dest, String feed) {
  // can't use the plugin to upload since we already have a upack
  // so just push it as-is.
  // as mentioned elsewhere there's an issue with upack connectivity
  // (https://forums.inedo.com/topic/3626/upack-repack-doesn-t-use-complete-version-string-from-cli)
  // so just use curl for now.
  // specifically pointing to the git curl, it's not in all versions of windows
  echo "Publishing ${path} to ${dest}/${feed}"
  String curlCmd = ". 'C:/Program Files/Git/mingw64/bin/curl.exe'"
  retry(3) {
    withCredentials([usernameColonPassword(credentialsId: 'proget-creds', variable: 'userpass')]) {
      pwsh "${curlCmd} ${dest}/${feed} --user \$env:userpass --upload-file ${path} --silent"
    }
  }
}

(The retry is because proget sometimes fails on the first LDAP lookup, but always works on a retry. I think it's more of an issue with our internal AD setup than with Proget.)

I've got a wireshark capture of the conversation between my local and proget via upack. I can't seem to upload it here, which is probably for the best, can i email it to you?

Any ETA on the filename fix? (every engineer's favorite question)

Hi, I'm playing around with upack, and seeing some confusing behavior.

If i repack a package with version 1.0.11-develop-2041 to 1.0.11-develop-2043-repack, the new upack filename does not represent that version, nor the original version:

> ls | select name
myApp-1.0.11-develop-2041.upack

> upack repack .\myApp-1.0.11-develop-2041.upack --newVersion='1.0.11-develop-2043-repack' --note='repackaged from old version'
Package: myGroup/myApp
Version: 1.0.11-develop-2043-repack

> ls | select name
myApp-1.0.11-develop-2041.upack
myApp-1.0.11.upack    <-- shouldn't this be '1.0.11-develop-2043-repack'?

The new version is correctly shown in the upack.json:

{
  "group": "myGroup",
  "name": "myApp",
  "version": "1.0.11-develop-2043-repack",
  "description": "myApp",
  "_git_commit": "<a hash>",
  "_built_by": "Jenkins",
  "_build_link": "<a jenkins url>",
  "repackageHistory": [
    {
      "id": "myGroup/myApp:1.0.11-develop-2041:<a hash>",
      "date": "2022-10-11 21:23:51Z",
      "using": "upack/2.2.6",
      "by": "mcascone1",
      "reason": "repackaged from old version"
    }
  ]
}

Am I doing something wrong, or is this an issue with upack? Would you rather I submit this as a upack github repo issue?

> upack version
upack 2.2.6.1

One more thing! I can't push or pull with upack:

> upack install myGroup/myApp 1.0.13-POLLY-Test-75 --source=https://proget.myOrg.com/upack/Platdev_dev --user=mcascone1@myOrg.com:<redacted pw>
The underlying connection was closed: An unexpected error occurred on a send.

Hi,
I'm wondering if there's a way to pull only non-prerelease versions from the Proget Api.

In the UI, in the All Versions tab, you can click Hide Prerelease, and this only shows versions with Major.Minor.Patch and no attached metadata. The URL bar shows it tacks on the param ?HidePrerelease=True to the URL.

I can't seem to duplicate this behavior in the api:

Invoke-WebRequest "https://proget.myorg.com/upack/$FeedName/versions?group=$GroupName&name=$PackageName&HidePrerelease=True"

doesn't filter the prerelease versions out for me.

I can build a thing to parse for only the MMP versions, but is there an accessible API for this?

This doesn't work:

iwr "https://proget.myorg.com/upack/$FeedName/$GroupName/$PackageName/versions/all?HidePrerelease=True"

Just thought I'd drop in and describe where we ended up with this. It works for us, YMMV.

This is running in the context of a Jenkins deploy run, but the concepts should apply to most other use cases.

There are a lot of env vars floating around in all of this; i know it's not ideal, but it's really the easiest way to move values around in Jenkins without going crazy with OO classes that are really hard to get working in the Jenkins context. I have recently come up with a pattern to store more complex objects in env vars as json strings, and convert them when needed.

Happy to answer any questions from anyone interested in using this admittedly obscure functionality.

// this is called from vars/deploy.groovy when appropriate
clickOnceDeploy([clickAppName:env.clickOnceName, clickAppPath:env.clickOnceDir, targetEnv:targetMap.key])

// defined in vars/deploy.groovy
def clickOnceDeploy(HashMap clickParms) {
  // find the zip in the download folder
  String dlFolder         = "${WORKSPACE}/${env.packageDownloadDir}"
  String clickOncePackage = "${dlFolder}/${env.releasePath}/${clickParms.clickAppPath}.zip" 
  String targetEnv        = clickParms.targetEnv
  String zipFileName      = "${clickOncePackage.replace('.zip', '')}-${targetEnv}.zip"

  // extract the zip - select the target env folder
  unzip zipFile: clickOncePackage, glob: "${targetEnv}/**", quiet: true

  // zip that folder up, use powershell to get the dir included
  pwsh "Compress-Archive -Path ${targetEnv} -DestinationPath ${zipFileName} -Force"

  // publish it to proget asset dir
  HashMap publishParams = [assetDir:"${clickParms.clickAppName}", zipFilePath:zipFileName]
  addToInfoMessages("Publishing ClickOnce App: ${clickParms.clickAppName}/${targetEnv}")
  publish.publishProGetAssetFromZip(publishParams)

// defined in vars/publish.groovy
def publishProGetAssetFromZip(HashMap publishParams) {
  // check for required params
  assert publishParams.assetDir != null
  assert publishParams.zipFilePath != null

  // sane defaults
  publishParams.get('progetBase', env.progetUrl)
  publishParams.get('overwrite', true)
  publishParams.get('format', 'zip')

  // build the api endpoint
  String progetAssetImportApi = "endpoints/${publishParams.assetDir}/import/?format=${publishParams.format}&overwrite=${publishParams.overwrite.toString()}"
  String assetUrl = "${publishParams.progetBase}/${progetAssetImportApi}"

  // need to pass the api token, so pull it from the jenkins cred store
  withCredentials([string(credentialsId: 'proget-api-key', variable: 'token')]) {
    retry(3) {
      httpRequest httpMode: 'POST', 
        url: assetUrl, 
        uploadFile: publishParams.zipFilePath,
        customHeaders: [[maskValue: true, name: 'X-APIKey', value: token]],
        ignoreSslErrors: true,
        responseHandle: 'NONE',
        contentType: 'APPLICATION_OCTETSTREAM',
        wrapAsMultipart: false,
        quiet: true
    }
  }

  addToInfoMessages("Published to ${publishParams.progetBase}/assets/${publishParams.assetDir}")
}

Thanks Alana, I continue to be impressed with your and Inedo's responses to these issues!

We are not on v2022 yet, we are on the latest v6 as of a few weeks ago, but now we have a real reason to do it!

Thanks again!
Max

Hi,
A few of our users have reported that they are unable to create Asset folders with the same name as a recently-deleted folder:

1. I had to clear out a Proget asset folder so I deleted it and then I tried to add it back both through the UI and the API. It says it’s created successfully but not showing up. Can you please assist?

2. I created a folder ‘astest’, deleted it successfully earlier. However, now if I try to recreate the folder ‘astest’ it does nothing, no error no nothing.
   It feels like a proget issue where it remembers old folders for some reason.

I suppose you could alternatively name this "unable to re-create an Asset folder".

The devs have been working around the issue by using varied names, but it might become more of an issue if we have to automate it.

Is this expected behavior? We are on Version 6.0.18 (Build 2-proget6)

Thanks!

cool, you guys are the best!

I've updated the instructions for feeds to make it copy-pasteable for users, rather than having to edit out the options/variables. Naturally I hide the originals and only show the updated ones.

Example: the builtin CURL instructions for a package:

To download $PackageFullName from cURL, run the following command:

curl $DownloadUrl[?contentOnly=[zip|tgz]] --output $PackageName.$PackageVersion.[zip|tgz] --user <user>:<password>

I simplified it to:

To download $PackageFullName from cURL, run the following command:

curl $DownloadUrl?contentOnly=zip --output $PackageName.$PackageVersion.zip

Powershell example:

To download $PackageFullName from PowerShell, run the following command:

Invoke-WebRequest $DownloadUrl[?contentOnly=[zip|tgz]] -Headers @{"AUTHORIZATION"="Basic " + [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes("<user>:<password>"))} -OutFile $PackageName.$PackageVersion.[zip|tgz]

simplified:

To download $PackageFullName from PowerShell, run the following command:

Invoke-WebRequest $DownloadUrl?contentOnly=zip -OutFile $PackageName.$PackageVersion.zip

Hi! I love being able to create our own download instructions for packages. This doesn't seem to be available for Assets. Am I missing something? Can it be added?

thanks!

based on https://forums.inedo.com/topic/2612/docker-container-documentation?_=1651176207029

I'm interested in this feature too.

We are adding LABELs when building docker images of the jenkins_url and commit_hash. We do the same thing for our regular packages.

The LABEL metadata is buried in the rest of the image's json. Is there any way to expose our own labels in the ProGet UI?

thanks!

Thanks, yeah, that seems to work.

I changed my pipeline code to not send any title metadata to ProGet, which returned that field in the UI to its original behavior.

I also found that rebuilding an existing package, and sent to ProGet with the updated pipeline, has the same effect.

Hi there!

I just updated our instance to 6.0.12, and this change appears to have been made. Thanks!

...however, now a lot of the packages show "place title here" in the package listing, which is causing a lot of confusion:

I will have to update the jenkins pipeline that sends the title metadata to send the package name, so this stops happening.

Is there any way to fix the existing data? I don't seem to be able to alter any of the metadata in the UI - which is by design, I'm sure, but in this case it's a bit of an issue.

thanks
max

I just noticed this in the ProGet Jenkins plugin docs:

dependencies (optional)

An array of strings, each consisting of a package identification string; this string is formatted as follows:

�group�:�package-name�

�group�:�package-name�:�version�

When the version is not specified, the latest is used.

Type: String

This is for the upload parameters. I don't remember seeing this before, is this new, and/or related to my consumes pattern?

Just wanted to come back to say this worked; I did it during work hours and there weren't any downtime issues. (I made sure to communicate that there might be some, no one complained.)

I did have to revert the config change after my first attempt, because it couldn't find any files. I think I did the manual drag-n-drop of the Packages folder to the new location incorrectly - creating Packages/Packages/. On my second run I used RoboCopy and it worked great.

I also had to set the permissions on the new folder location to include the NETWORK SERVICE user, which ProGet is running as, with full access. Maybe I should have used the robocopy command that copies perms as well, but it all seems to be working now.

And also to clarify again:
all except the most recent 3 versions matching mybranch from the feed

means:
in this feed, delete matches of 'mybranch', except the latest 3 versions of those matches, which would only impact the versions matching mybranch and leave all other non-matches untouched;

and not:
in this feed, delete everything except the latest three versions of 'mybranch', which would leave ONLY the latest three versions of mybranch, and delete eveything else, which would obviously be a major problem.

Is that correct? I mean, it has to be, but I just want to be sure.

This means mybranch and mybranch2 can be reduced to mybranch2.

Sorry but wouldn't this be the reverse: *mybranch* will match *mybranch2*?

It seems to work normally now with the homepage setting to none. The default category is support. So I'm happy! Thanks!

Thanks Alana!

Would my Drop Path idea work for this, and reduce the manual steps?

Also - if we change the path without changing anything else, will proget just start keeping new packages in the new path, and still be able to access the old path?

Here's another example. If I have two rules:

1. all except the most recent 3 versions matching *mybranch* from the feed
2. all except the most recent 3 versions matching *mybranch2* from the feed

This is achieved by creating two rules, each set up with 'delete all except the latest 3 versions' and 'delete packages with matching versions' with *mybranch* and *mybranch2*

Which will the behavior be?

1. It will remove everything except the last 3 versions of both mybranch and mybranch2, OR
2. it will not remove anything except the last 3 versions of both mybranch and mybranch2

I want the behavior of #2. My plan is upon publishing from Jenkins, it will also set up a rule, if it does not exist already, for the branch name it is pushing to only keep 3 versions of that branch. I don't want any given rule to affect packages with any other branch names other than what is in that rule. So if I have a rule to delete branch1, I don't ever want it to delete anything from branch2.

After reading over this 18942 times, I believe the mechanism requires all criteria of each rule to pass before executing the delete - so i think the behavior will be what I want, that of #2 above, and only packages with versions matching mybranch or mybranch2 will be deleted (except the most recent 3 of each).

Please confirm! Thanks!

To avoid this kind of uncertainty, maybe the UI text for each rule could be clarified to something like, "Package versions matching *mybranch2*, except the most recent 3 of that package, will be deleted". I think i am reading all except the most recent 3 versions matching mybranch from the feed to mean, "everything else will be deleted except the most recent 3 versions matching mybranch". Which I don't think is right, but it's just not clear enough to be sure.

Thanks!!

My apologies if this isn't covered elsewhere, i searched pretty deeply in the docs and this forum and haven't found anything.

We originally set up ProGet to keep its packages in C:\ProgramData\ProGet\Packages\..., but we want to move the data storage to the D: drive, as that's a better practice, and it also has 1TB space on it vs 100GB on C:, which we have hit a few times.

What is the least friction way to do this? This doc says,

When you change a feed's disk directory... the package files will not be moved.

One thing I can think of is to move the storage root path to D:/something, and then set up a Drop Path of the existing location. Would that result in all of the packages being copied correctly into the new location? Would any metadata be lost? Is there a better way?

Thanks!

Changed it to "none" and it's back to normal. But if i select a specific category (i've only tried Support tbh) it fails with the same error as before.

I don't remember seeing this before. Clearly something's off.

It's empty; it says "select category".

FWIW, i also see this unrelated bug at the home page:

I always forget to set the topic to "support" when creating a new forum thread. Is there a way to set it to default to this setting?

Hi! We are trying to set up a coherent retention policy for our development feeds. We are currently using a process that designed with a lot of input from Inedo, so i hope you might have some insight on this detail of managing it.

For a given repo/package, we push all ci builds from Jenkins to a <feedname>_dev feed. Only packages that are cleared for production deploy are repackaged into a <feedname> feed, which is considered the production feed.

So we end up with a ton of branch builds littering the _dev feeds, most of which are never needed. I've been unable to come up with a generic retention policy for the dev feeds that prunes the unneeded builds, but always leaves at least one build from each branch available, until it can fade out after say 30 days.

What I am considering is adding a step in the publish process to use the Retention API to create a new retention policy for every new branch published to ProGet. That way we can set up a specific policy for each new branch.

The problem is obviously that we will have to clean up the retention policies at some point.

An example might help:

• package 1
  • versions
    1. branch1-v1
    2. branch1-v2
    3. branch2-v1
    4. branch2-v2

If I want to keep only the latest 5 versions, it will not guarantee that branch2 will keep its latest version available if branch1 keeps publishing builds.

Without the api approach, I can't see a way to mitigate this using the existing tools available: deleting/keeping versions or names that match/don't match a wildcard string. Maybe I am just not seeing it!

Thanks in advance!

Also relates to https://forums.inedo.com/post/12249

Hi, thanks - yep, it appears that plugin is assuming that field is not null. Good find!

There aren't any errors from proget either in the output of the plugin upload in the jenkins console, or in the proget error logs, as far as i can tell.

Plugin issue: https://issues.jenkins.io/browse/JENKINS-67923

Thanks again for all your great help!

Hi, i like having the option of organizing packages in a feed with GroupNames (I'd still like the UI to organize them better, see this post)

But what if i don't want a groupname? i just want feedname/packages?

At least when using the jenkins plugin, this doesn't seem to be an option. I either have to give "null" as a group, which isn't the answer, or if it's just not present, like feedname/packagename, the plugin upload succeeds but the packages is nowhere to be found. The resultant upload URL gives a 404.

That makes sense. Is there any way to import files into an Asset Dir from a URL - so I don't have to download the file and import it to ProGet manually?

Hi, thanks. I'm not sure I've done it correctly, but it doesn't appear to be working.

I have the connector pointing at https://services.gradle.org/distributions/ and am using the connector in a new feed, public-gradle.

If I search for "gradle" it finds no matches.

I'm also getting a certificate error when running gradle, but i don't know if that's coming from my connection to proget or proget's connection to gradle.org.

Downloading https://proget.mycompany.com/maven2/public-gradle/gradle-7.3-bin.zip

Exception in thread "main" javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

Hi, i'm reopening this thread as the public docker connector has been available for a while now, but I can't figure out how to use it.

I have the connector set up to point at https://registry.hub.docker.com
The setting Do not attempt to search the container registry was automatically enabled.

I have a feed public-docker set up to use the connector. The connector shows Active and Healthy, although it shows no known package count (I believe this is expected).

Overall, I'm not sure how to use it. The documentation reads as self-conflicting, although I don't know enough about it to say that for sure. That's why I'm asking.

From https://docs.inedo.com/docs/proget-docker-connectors:

You can also create a connector to an external (non-ProGet) registry such as Docker Hub. This could be useful to whitelist certain base images, or to cache them on your local infrastructure.

but then...

Many external registries such as Docker Hub do not expose an API for searching and discovery of images, so the ProGet web UI cannot be used to search or browse for images stored in one of these registries via connector.

So, i see that I can't search in the UI for images. That's alright, but how can i make sure my docker pull or docker run commands go through the ProGet feed, so the images can be cached, instead of directly out to the Docker Hub?

Thanks!

I have connectors set up for Chocolatey, nuget, and Docker public feeds (docker isn't working). I would like to set up a connector to services.gradle.org so i can point my gradle builds' distributionUrl at a proget feed instead of the external feed. I don't see a gradle type in the connector setup pane, is there a way to do this?
Thanks!

It is pretty easy; I just wanted to know. Thanks for the reply!

Hi,
I'm looking at uploading files to the Asset Library from my Jenkins pipeline. I know there's a good API but i thought i'd try the plugin first. It looks like the plugin isn't meant to work with Assets?

It seems that the plugin is always going to try to upload to https://proget.myorg.com/upack/<my feed info>, but the Asset Directory API specifies https://proget.myorg.com/endpoints/<my dir name> - note the lack of the upack node in there.

Is there any way to get the jenkins plugin to publish to an Asset Directory, or is the API the only way?

thanks!

Our package promotion process involves repackaging versions from a dev feed into a prod feed.

Users have found it confusing to have to remember to use Repackage instead of Promote, as "promote" is kind of the language we use for the process.

I would like to remove the option to Promote packages, while leaving the Repackage option. In the permissions scheme, as far as i can tell, the two mechanisms are considered to be the same thing. The Task is literally Promote/Repackage Packages.

Is there any way to do this currently? We're still on v5 if that matters.

thanks!

Hi, just wanted to say that i happened to look at the new releases available for v5, and saw that this is fixed in the latest release, 5.3.44.

Awesome! Thanks!

Hi, thanks for the ideas. At least in our process, we really only have three happy-path steps:

1. a pre-release package is published into the dev feed with a version 1.2.3-branchname-weightedprereleasenumber.
2. That package is deployed to all lower environment tiers, eventually being QA/UAT passed and ready to ship.
3. The package is Repackaged as 1.2.3 and promoted into the prod feed, to be released into Production.

What i'd like to see is a little up-arrow or similar UI 'hint' on the version's row in the package's main version list that it has been repackaged and/or promoted into another feed. So if i'm looking for a package in the dev feed to see which one's been promoted, i don't have to dig into the metadata or history of every version to find it - i just look in the main list.

Extra bonus points for putting the version and feed it was promoted/repackaged to in the hover-text of the UI hint!

You're right that the info is available in the package record json, the history in the UI, and the package metadata, it's just a hassle to have to dig into them in each version to find whether or not it even exists.

Let me know if i can clear anything else up!

Thanks! I see on that YouTrack page that it's planned for v6.0.6. Will it not be ported back to v5? This would be motivation enough for me to upgrade. Not that I need motivation to upgrade, but it would certainly increase the priority.

ProGet version: Version 5.3.43 (Build 4-proget53)

Hi,

We've seen twice now that the latest build package published to ProGet is showing at the bottom of the list of versions.

Additionally, the "Latest Version" shown on the feed main page does not reflect the actual latest version.

I have a feeling we are not understanding something about semantic versioning and pre-release version semantics.

We are using separate Dev and Prod feeds. Ready-To-Ship packages are manually (for now) repackaged into the Prod feed. So by design we'll never have a "stable" release version in the Dev feed. Part of me feels this is overcomplicating things, but it feels "safer" to have an Approved Packages Only feed to deploy to Production with.

Example:

• 2.1.1-PLATDEV-11647-1017 is shown at the top of the list, after 2.1.1-PLATDEV-11647-1016, 2.1.1-PLATDEV-11647-1015, etc.
  At the bottom of the list is 2.1.1-develop-2018, which is the most recent build of this package.
  On the main feed page, Latest Version: 2.1.1-PLATDEV-11647-1017 is displayed.

I just saw this post which seems to be related. I tried to repackage a existing package with a lower, 3-digit version, and it didn't work, possibly because it was just a repackage and not an upload; regardless, this doesn't feel like a valid workaround.

Thanks for the insight into your process on this. It's interesting.
For our part, I put everything I can on Confluence. I probably like Confluence as much as I hate Sharepoint and OneNote. (At least with OneNote I can actually create a document successfully.)

Flow charts help a lot, which are super easy to create and maintain with the Draw.io integration into Confluence.

I like confluence for its deep-linking into Jira as well.

When I'm looking at a package's version list in the ProGet UI,
I would love to see a little icon or flag on an entry if that version has been promoted or repackaged,
So I know right away that that has happened,
Without having to dig into the history, potentially finding nothing.

thanks!
max

Yuck, sharepoint?

I hear you; SOPs tend to wither and gather dust as the processes evolve, though, in my experience. That's why, while we get some real-world use of the manual promotion and versioning processes, in the hope that repeatable, reliable patterns emerge that we can then automate, we want to remind people in-flight what the process is supposed to be, so they don't make mistakes.

thanks for the reply!

Thanks, I knew this one was a bit of a stretch. It'll be tough if not impossible to automate this in Jenkins, and is another tick on the "check out buildmaster" list.