Changing server context in the middle of a script

jimbobmcgee

Is there any practical reason why an OtterScript script cannot issue a for server call in a job that does not use custom targeting? The limitation seems arbitrary at first glance.

Say, for example, as part of my orchestration, I want to run a job against a list of servers, or an environment and for each one of those resolved servers I want to run an isolated step on a common ancillary server, I simply can't.

I can either...

set the job to target servers by name (or role / env), and lose the ability to switch to the ancillary in the script
set the job to custom targeting, and lose the ability to easily select against which servers I want to run the main script

For example, I could want to run the following contrived example script against a subset of my machines...

set $hostname = $EnvironmentVariable(COMPUTERNAME);

set $macaddr = $PSEval((Get-NetRoute -DestinationPrefix '0.0.0.0/0' | Sort { $_.RouteMetric + $_.ifMetric } | Select -First 1 | Get-NetAdapter).MacAddress);

for server 'common-dhcp' {
  set $nextip = $SHEval(/opt/dhcp/dhcpctl get-next-ip);

  Exec(
    FileName: /opt/dhcp/dhcpctl
    Arguments: set-reservation $macaddr $nextip
  );
}

for server 'common-dc' {
  Create-File(
    Name: "X:\known-servers\${hostname}.txt",
    Text: $macaddr
  );
}

I could create an input variable of type Text, and manually type the server names into that prompt; then @Split and $Trim that value, and hope I got the names right -- not fun for more than a few Servers.

I could create a script-specific role and require that I pre-populate it before running the script. However, I can easily see that leading to mistakes, as the management of the role is separate to the running of the script -- re-running from the History view would be fraught, for example.

I could create an input variable of type List, then compose a separate script, with the sole purpose of peforming some Rafts_GetRaftItems / (some horrid JSON manipulation) / Rafts_CreateOrUpdateRaftItem to periodically update the list with selectable server names -- but this is borderline insanity.

Otter already has a decent server / role / environment selector -- I just lose access to it if my script needs to for server.

Assuming there really is a good reason why we can't change context in a targeted script, then could we perhaps have a dedicated Input Variable type (e.g. Object List), which could be automatically populated with servers, roles or environments (with single- or multi-select), so we can emulate the existing, user-friendly targeting in for server scripts?

(I note there seems to be a specific input variable type for Universal Packages, so I assume the concept of custom controls exists. I'd look at maybe extending this myself, but I don't think custom variable types are an extensible point in the SDK.)

atripp

Hi @jimbobmcgee,

This restriction is for security purposes, specifically to enable the usecase of Otter enabling end-users to create/edit/run scripts, but not decide where they are run. So the for server is locked unless the targeting is set to None.

The solution is to indeed add a variable that allows you to select a server... but as you noticed, you'd have to type in a list. We simply ran out of time to bring those over from BuildMaster unfortunately, and this is not really a popular Otter requirement.

But sure it's possible, it's called a VariableTemplateType. Here is the code from BuildMaster that you could probably copy/paste into a custom extension.

But the issue is that you don't have access to DB in the SDK. Kind of a pain, but you could either reference Otter.Core.dll in your Nuget Package, use reflection, call DB directrly, etc.

[Category("Infrastructure")]
[DisplayName("Servers")]
[Description("Servers configured in BuildMaster, optionally filtered by one or more environments")]
public sealed class ServerListVariableSource : BuildMasterDynamicListVariableType
{
    [Persistent]
    [DisplayName("Environment filter")]
    [PlaceholderText("Any environment")]
    [Inedo.Web.SuggestableValue(typeof(EnvironmentNameSuggestionProvider))]
    public string EnvironmentsFilter { get; set; }

    private class EnvironmentNameSuggestionProvider : ISuggestionProvider
    {
        public async Task<IEnumerable<string>> GetSuggestionsAsync(IComponentConfiguration config) =>
            (await DB
                .Environments_GetEnvironmentsAsync().ConfigureAwait(false))
                .Select(e => e.Environment_Name);
    }

    public async override Task<IEnumerable<string>> EnumerateListValuesAsync(VariableTemplateContext context)
    {
        var values = (await DB.Environments_GetEnvironmentsAndServersAsync(false).ConfigureAwait(false))
            .EnvironmentServers_Extended
            .Where(es => es.Server_Active_Indicator)
            .Where(es => string.IsNullOrEmpty(this.EnvironmentsFilter) || string.Equals(this.EnvironmentsFilter, es.Environment_Name, StringComparison.OrdinalIgnoreCase))
            .Select(es => es.Server_Name)
            .Distinct();

        return values;
    }

    public override RichDescription GetDescription()
    {
        if (this.EnvironmentsFilter?.Length > 0)
            return new RichDescription("Servers in ", new ListHilite(this.EnvironmentsFilter), " environments.");
        else
            return new RichDescription("Servers in all environments.");
    }
}

It is on our list to "Make Job Template Varible Editor Closer to Pipeline Variable Editor", it's just not trivial and not a huge priority on our products roadmap (https://inedo.com/products/roadmap)

Cheers,
Alana

jimbobmcgee

Thanks @atripp -- I can't believe I missed VariableTemplateType in the SDK!

Thansk for posting the BuildMaster equivalent -- I'll look to see if I can adapt. As you say, the lack of DB may make this more challenging.

Out of interest, how resilient are extensions to mismatched versions of InedoSdk and/or Otter.Core? Would I have recompile/redistribute an extension each time Otter is updated?

Regarding the security issue, I'm not entirely certain I agree – perhaps, as the sole administrator / runner, my use case is vastly different from others. Agree to disagree

Maybe, in a multi-user scenario, a valid middle ground might be to consider allowing Job authors to limit the servers that can be for server'd into, i.e. as a property of the job, rather than cutting off all access to for server.

Or maybe allowing defining a for server privilege on servers / roles / environments themselves, so that certain users are allowed to issue that command in jobs scripts, and resolving the permissions at runtime.

(I'm sure you've considered this before.)

atripp

Hi @jimbobmcgee,

Good question on versioning... to be honest I kind of forgot how assembly binding would work in this particular context

The Otter.Core assembly version changes in each release. I don't think we have special handling for it, but you could probably add in an assembly resolver for now, until we officially add support. The Job Template / Variables is something we have on our 2025 roadmap for Otter, but other products are priority.

Security... BuildMaster handles it a little better, and will do runtime checks, and does allow for more flexibility. This serverlock we added in Otter was a stopgap in 2022 (I think?) to resolve a customer's issue. The issue you are encountering is relatively easy to work-around, by reimplementing server targeting using variables. It's also on our roadmap, but you know... priorities :)

jimbobmcgee

@atripp

OK, I've tried, but I'm getting nowhere.

I've added a very rudimentary implementation of DynamicListVariableType, based on your code. I've used Inedo.SDK.GetServers(true); rather than linking directly to Otter.Core, but I think that might be irrelevant.

Custom variable types just seem not to work at all.

Any time you add one to the Job (including trying to add the built-in Universal Packages variable type), the front-end just throws an HTTP/500.

My variable type appears in the list, I select it; I give it a variable name and set the list properties (restrict, multi-select, etc); I click Save Variable and an dialog-style iframe pops up with the standard error page. The new variable's JSON is never written to the raft file.

An error occurred in the web application: Value cannot be null. (Parameter 'type')

URL: http://172.31.15.125:8626/jobs/templates/edit-variable?templateId=Default%3A%3AJobTemplate%3A%3AInitialization%2FRun Sysprep for target servers
Referrer: http://172.31.15.125:8626/jobs/templates/edit-variable?templateId=Default%3A%3AJobTemplate%3A%3AInitialization%2FRun Sysprep for target servers
User: Admin
User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:133.0) Gecko/20100101 Firefox/133.0
Stack trace:    at System.ArgumentNullException.Throw(String paramName)
   at System.Activator.CreateInstance(Type type, Boolean nonPublic, Boolean wrapExceptions)
   at Inedo.Otter.WebApplication.Pages.Jobs.JobTemplates.EditJobTemplateVariablePage.<>c__DisplayClass5_0.<CreateChildControls>b__1()
   at Inedo.Web.Controls.ButtonLinks.PostBackButtonLink.Inedo.Web.Controls.ISimpleEventProcessor.ProcessEventAsync(String eventName, String eventArgument)
   at Inedo.Web.PageFree.SimplePageBase.ExecutePageLifeCycleAsync()
   at Inedo.Web.PageFree.SimplePageBase.ProcessRequestAsync(AhHttpContext context)
   at Inedo.Web.AhWebMiddleware.InvokeAsync(HttpContext context)

::HTTP Error on 20/01/2025 18:46:39::

Attaching a debugger traces the error to code which decompiles horribly (and very likely does not match the codebase exactly). I won't paste it all here, but it inside the EditJobTemplateVariablePage, there is a CreateChildControls method, partially containing:

PostBackButtonLink postBackButtonLink = new PostBackButtonLink("Save Variable", delegate {
    // ... omitted for brevity
    string selectedValue = CS$<>8__locals1.ddlType.SelectedValue;
    VariableTemplateType variableTemplateType;
    if (!(selectedValue == "Constant"))
    {
        if (!(selectedValue == "Text"))
        {
            if (!(selectedValue == "List"))
            {
                if (!(selectedValue == "Checkbox"))
                {
                    // *** NEXT LINE THROWS... (parameter 'type' cannot be null) ***
                    variableTemplateType = (VariableTemplateType)Activator.CreateInstance(Type.GetType(CS$<>8__locals1.ddlType.SelectedValue));
                }
                else
                {
                    variableTemplateType = VariableTemplateType.Checkbox;
                }
            }
            else
            {
                variableTemplateType = VariableTemplateType.List;
            }
        }
        else
        {
            variableTemplateType = VariableTemplateType.Text;
        }
    }
    else
    {
        variableTemplateType = VariableTemplateType.Constant;
    }
    // ... omitted for brevity
});

It looks like Type.GetType call can't resolve the custom VariableTemplateType class name.

The value of ddlType.SelectedValue looks (at first glance) to match the correct class and assembly name of my type (ServerSelector.ServerListVariableType, ServerSelector).

I've also tried to manually enter the variable definition directly into the raw raft content, but I must be missing something in the syntax, because the result just turns red in the /jobs page (no errors are logged in Diagnostics Centre.).

"JobVariables": [
    {
      "Name": "TargetServers",
      "Description": "Execute against these servers",
      "InitialValue": "",
      "Type": "ServerSelector.ServerListVariableType, ServerSelector",
      "Usage": "Input",
      "ListValues": [],
      "ListMultiple": true,
      "ListRestrict": true
    },
...

For what it is worth, my custom class is...

[DisplayName("Specific servers")]
public class ServerListVariableType : DynamicListVariableType
{
    [Persistent]
    [DisplayName("Include inactive")]
    public bool IncludeInactive { get; } = false;

    public override async Task<IEnumerable<string>> EnumerateListValuesAsync(VariableTemplateContext context)
    {
        IEnumerable<string> GetServerNames()
        {
            foreach (var s in Inedo.SDK.GetServers(IncludeInactive).Select(s => s.Name))
                yield return s;
        }

        return await Task.FromResult(GetServerNames());
    }

    public override RichDescription GetDescription()
    {
        return new RichDescription("Allows selection of ", 
                                   new Hilite("Servers"), 
                                   " outside of Job targeting");

    }
}

atripp

@jimbobmcgee that's too bad it didn't work :(

That's probably why we didn't update the page in Otter 2024 to work with those types from BuildMaster... and as you saw from the code, it's probably not trivial. I know that some of the other platform modernization efforts (especially with HTTP/S support) took a lot longer.

I guess the only option for now is to just add a list of server names. This is still on our roadmap, but rewriting the page is more than we can do in a scope of a fix like this.