Interacting with releases from otterscript, especially from within a scheduled job

[brandon_owensby_2976]

Buildmaster has some nice features when it comes to git and even monitoring git. Unfortunately it falls a hair short for what I need and I'm trying to bridge the gap with an otter script. This is having me fetch request info to make other calls. I'll give more details but the overall goal is to see if I'm not going about this in the way you'd recommend as well to see if there is any changes planned in the next major release that might help with this.

First some specifics on where my troubles are. When writing a script I prefer OtterScript because of all the built-in commands and the help you can get from the GUI as you go. I also figured that when interacting specifically with BuildMaster it would provide the most easy to use functionality. Unfortunately that didn't prove to be true in all cases, especially when running outside the context of a specific app, like in a scheduled job. I did find, however, that you can call the Buildmaster rest API to get extra functionality in some cases. Where I can't seem to bridge the gap is I'm getting a list of applications in the otterscript and I can get a set of release numbers but the API I want (which is Releases_GetRelease) in this case requires application id which I don't have, I just have name.

What I'm after here is to get the branch the release is linked to. Otterscript seems to have a way to do it for a build not a release. I tried treating the build as an optional parameter but it just threw an error for an invalid application name. I thought the error was odd but didn't know how to get around it so I pivoted. If there is a way I can use that function that would be great.

The ultimate thing I'm after (which I'm incredibly close on) is go through the active applications that have a git monitor on them and check the releases on that app to see if there are any created off a pr branch where the pr is no longer open so I can purge those releases. If there is something in an upcoming major release that might help with I'd appreciate learning about that too.

Thank you,
Brandon

[stevedennis]

Hi @brandon_owensby_2976 ,

You are correct, the $BranchName function is a build-information function that provides which branch a build was created from. It's a slightly different concept than the "default branch" that is associated with a release.

If changing your workflow to align with more standard practices isn't in the cards, then you may want to create a custom extension that will allow you to create your own variable functions and operations. This will make adding things like this easy and if it's something we think is generally useful, then we can generally add it in.

Although it's not supported in the SDK, if you take a reference to the core assembly (buildmaster.dll), you can invoke database commands like Releases_GetRelease to find that information easily.

Thanks,
steve

[brandon_owensby_2976]

Hi @stevedennis,
Can you provide more detail in your response? You mention, for example, standard practices. You don't however mention for what, what those are, or what features BuildMaster has to support them. Maybe you mean in terms of doing things on a build bases. If that is what you meant it might be helpful if you knew what I was doing which I unfortunately didn't provide in my original message.

The goal of my script is a clean up script for PRs since you guys do not have that feature built in. The purpose behind this is I want a PR to kick off a release/build to prove that it will actually build and set a marker on the PR to allow it to be merged. Once the PR is merged those releases/builds are no longer useful. Unfortunately, none of your retention rules take that in to consideration (part of why I asked in the previous message if you had any enhancements coming in the next major build that might be helpful for me). On your note about standard practices, I'm assuming this might be where it is targeted (the PR workflow). One of our colleagues mentioned in another post trying things differently and even doing releaseless builds but they did support the idea of having some build verification on a PR before allowing it to be merged. I'm not entirely sure, however, how a releaseless build might help. I did think about it but my fear is making the UI more cumbersome in finding what you are looking for. What I mean by that is everything is listed together (although I haven't yet tried a mix of releases and releaseless builds but I will) so naturally the PRs would continue pushing the main release down. Also if PR releases aren't being removed as soon as the PR is merged then the main release stays pushed down and there are more PRs to go through to find yours.

In my original message I left it a little more open to hopefully provide more flexibility in your response. This time going to try a list of specific questions.

• Do you have any upcoming features that might help with any of this?
• The call to Releases_GetRelease requires a release id but fetching the releases just gives me release numbers. How do I bridge that gap?
• What is the standard practices you referred to for the workflow?

I'm not sure I'll be able to before you respond but I do plan on investigating writing an extension for a different purpose anyways so I'll also look at one for this PR management too. In addition to the bullet points feel free to add anything you might find useful.

Thank you,
Brandon

[atripp]

Hi @brandon_owensby_2976 ,

Here is the standard practice / pattern that we support for Feature Branches:

The idea is that developers start by creating a well-named feature branch (i.e. following a strict naming standard), adding commits to it, testing it, and then finally submitting a pull request for it. See our Feature Branches article to learn more about this workflow, including how retention rules can be configured.

In other words, the Pull Request comes only once the branch is ready (validated) for merging, and BuildMaster is submitting (or facilitating) that request. We take the term literally ("a request to pull the changes from the branch into into the master"):

• The branch owner is responsible for creating a "mergeable branch" and submitting that for review; a KPI here is is submitting timely/acceptable requests
• The master owner is responsible for promptly reviewing the request, then merging, closing, and deleting the branch; a KPI here is timely review/merging

In this workflow, there is no need for "commit verification" before submitting a PR. That's the responsibility of the branch owner, and they will have already done that before submitting.

I am aware that others do not use Git repositories this way, but our tooling is a generally opinionated. For example, we consider "pull requests as a backlog" as an anti-pattern for several reasons, but mostly that it's an inappropriate tool for work item management and lacks any semblance of prioritization and related feature. Issue tracking is appropriate for backlog tracking. So thaht's why there's not first-class support for it in BuildMaster.

All this said, I don't think the "Feature Branch" workflow is what you're describing. I don't quite understand it, but long story short... if it doesn't really fit into one of our use cases, so it's isn't something that's going to be easy to configure in the software - and you'll likely need to start customizing with your own Extension.

Instead, we recommend to just adjust the workflow.

Thanks,
Alana

[brandon_owensby_2976]

@atripp,
I'll be honest I'm not sure how we here completely. What I mean is that it appears that some how I've led you to the conclusion that we are using PRs but not feature branches which made it hard to get the help I needed. I appreciate the information you provided as that finally helped us figure out the disconnect.

We are indeed using feature branches. The primary difference in what we are doing and what you are doing is the timing in which an automated build is generated which leads to a few other smaller differences.

When working on a ticket we create a feature branch for that ticket. At this point everything we do is local to our personal computer and we commit to save our work. Once we feel we are code complete we generate the PR and mark it ready for code review. The PR generates a build and one of the requirements on git repo is that a PR must have a successful build before it can be merged. Once the build finishes and the reviewers have approved the code it can be merged in to master which generates the official build which is first deployed to our dev/test (basically the QA environment but the call it dev) environment and then if everything is good it eventually goes to production.

Unfortunately my lack of understanding in this area of your application can me from asking direct clear questions. I had 2 core issues I was trying to figure out.

• Best way to set up the creation of these builds of a PR
• How to clean up the builds when no longer needed

Even though they are 2 distinct issues the second one is affected by the first one as I'm come to find out. Originally I thought the best thing to do was to have it create a release based on PR # so you could see the history of the builds that occurred for that PR. For example if a unit test failed so you had to do a subsequent build to fix it. The other part was to make it easy for developers to find the build they generated in case it failed so they could see why as there could be multiple developers with feature branches on the same application. This method created the issue of a release that then needed to be purged once the PR and branch were gone. I saw no built in feature for this. You have a scheduled job feature but I didn't want to have to set that up individually for every app so I tried to create a generic one but your otter commands are set up more for running in the context of app/release/build then globally and that is where this post came from.

After some experimenting I realized I could set the monitor to just create a build with a specific pipeline and that seemed to be the ticket. It doesn't clutter up the app with extra releases you have to purge and the you can still see the history of your tickets build by going to the branch and viewing builds for the branch.

I'm still not entirely sure how the clean up will work there but I'm going to cover that in a different post as I plan on having a post specifically about just the retention policy feature.

I hope that clarifies things a bit. I'm not sure if my colleagues will mind looking for things by branch of PR # but in some ways I personally think it might be nicer.

Thank you,
Brandon

[atripp]

Hi @brandon_owensby_2976 ,

Thanks for clarifying; it got a little distracting with the desire to have PR cleanups and specialized variables! We also see a lot of anti-patterns in Git repositories, such as using PRs for workflows, because that's how open source projects tend to work.

Anyway, it sounds like you figured it the first part, which is to set the monitor to just create a build with a specific pipeline (e.g. "Feature Branch" pipeline).

The next step, just use a retention policy that targets builds that have completed the pipeline. For example:

https://docs.inedo.com/docs/buildmaster/builds-continuous-integration/buildmaster-ci-git-workflows/buildmaster-git-feature-branches#cleaning-up-feature-branch-builds-with-retention-rules