jimbobmcgee

jimbobmcgee

@dean-houston I'm sure that fixing the syntax would not be straightforward, but introducing @ListSet and %MapSet variable functions in lieu is probably a good enough workaround for nearly all use cases where someone would want to do this.

jimbobmcgee

I've been trying to get working some of the behaviours described in Augmented Help, so that Otter might generate rudimentary input UI for my ad-hoc scripts. However, I must be reading it wrong, because nothing I seem to do with these makes any difference.

Consider the rudimentary OtterScript...

/*  .AHPARAMETERS
    .AHPARAMETER TestA(input, text)
    This is a simple test input

    .AHPARAMETER TestB(input, list, values='optA,optB' default='optA')
    This is another test input, with allowed values limited to specific items

    .AHPARAMETER TestC(input, text, sensitive)
    This is a third test input, which might render as a password field
*/
Log-Information "A test -- this is unimportant";

...or a similar rudimentary PowerShell...

<#
.AHPARAMETER TestA(input, text)
.AHPARAMETER TestB(input, list, values='optA,optB' default='optA')
.AHPARAMETER TestC(input, text, sensitive)
#>
Param(
    [String]  $TestA,
    [String]  $TestB,
    [String]  $TestC,
    [Switch]  $TestD
)
Write-Output "A test -- this is unimportant"

When creating an ad-hoc job to run these (i.e. from the button in the list within the Scripts screen), I anticipate that input fields should be shown for $Test1, $Test2, $Test3, etc., with certain special attributes applied.

However...

the PowerShell ad-hoc job shows the input parameters derived from the Param block (including a checkbox for $TestD), but does not present a list for $TestB, nor does it mask the sensitive $TestC.
the OtterScript ad-hoc job never shows any input parameters at all.

The only way I can get input parameters to show reliably is to create an associated Job Template and define them there (using the Job Template GUI or directly in the Raft JSON), but that obviously requires keeping two different parameter sets in sync (i.e. the one in the script and the one in the template); and these never apply to an ad-hoc script execution.

The implication in the documentation is that Augmented Help is designed to keep this in one place, and is the preferred way to apply these attributes.

What am I missing here? What is the "right" way to document my parameters such that the UI might pick it up?

Some more-targeted stuff I've tried

(this may or may not be relevant...)

A cursory scan of the decompiled PowerShellScriptParameterInfo.Parse(...) from Scripting.dll suggests that the pattern used to extract values from a comment-based help block only picks these up if they are in the form...

.AHPARAMETER TestC
input, text, sensitive

(i.e. not .AHPARAMETER TestC(input, text, sensitive) as in the documentation.) I've tried this form, both with and without the surrounding brackets, but it makes no difference.

The same decompilation does not refer to AHPARAMETER in the code, although AHEXECMODE, SYNOPSIS, DESCRIPTION and PARAMETER are referenced.

The nearest equivalent I can find for OtterScript may be in the decompilation of OtterScriptLanguage in OtterCoreEx.dll, which has methods GetParameters(...) and ParseScriptInfo(...). Neither of these seem to mention AHPARAMETER either (although the former has a check for if (!additionalHeader.StartsWith("Parameter")) {...}. I can't follow it back far enough to see how additionalHeader is populated though.

I can see in similar decompilations that the script engines for Batch, Shell and Python seem to favour # AhParameters (plural) or # AhArgFormat -- I've tried similar strings in my OtterScript and PowerShell scripts (e.g. .AHPARAMETERS and .AHARGFORMAT).

I note very similar documentation for BuildMaster (2,3) as well, but I haven't got one of those in place with which to test.

jimbobmcgee

Looks like I pasted the SecureCredential... file twice (and can't edit/update now). The SecureResource... version is:

SecureResourcePropertiesVariableFunction.cs

using System.Collections;
using System.ComponentModel;
using Inedo.ExecutionEngine.Executer;
using Inedo.Extensibility;
using Inedo.Extensibility.SecureResources;
using Inedo.Extensibility.VariableFunctions;
using Inedo.Serialization;

namespace Inedo.Extensions.VariableFunctions.SecureResources
{
    [ScriptAlias("SecureResourceProperties")]
    [Description("Gets the properties available to a named Secure Resource.  Use `$SecureResourceProperty()` to obtain the value.")]
    public sealed class SecureResourcePropertiesVariableFunction : VectorVariableFunction
    {
        [VariableFunctionParameter(0)]
        [ScriptAlias("resource")]
        [Description("The name of the Secure Resource for which to fetch property names.")]
        public string? ResourceName { get; set; }

        [Description("The type of the Secure Resource for which to fetch property names.")]
        [ScriptAlias("type")]
        [VariableFunctionParameter(1, Optional=true)]
        public SecureResourceType? ResourceType { get; set; }

        protected override IEnumerable? EvaluateVector(IVariableFunctionContext context)
            => GetPropertyNames(ResourceName, ResourceType, context);


        internal static IEnumerable<string> GetPropertyNames(string? resourceName, SecureResourceType? resourceType, IVariableFunctionContext context)
        {
            var resourceResolutionContext = new ResourceResolutionContext(context.ProjectId);
            var secureResource = SecureResource.TryCreate(resourceType.GetValueOrDefault(), resourceName, resourceResolutionContext) 
                ?? throw BadResourceName(resourceName ?? "<null>");
                    
            return Persistence.GetPersistentProperties(secureResource.GetType(), true)
                              .Select(p => p.Name);
        }

        internal static Exception BadResourceName(string resourceName)
        {
            return new ExecutionFailureException(string.Concat(
                "Could not find a Secure Resource named \"",
                resourceName,
                "\"; this error may occur if you renamed a resource, " +
                "or the application in context does not match any " +
                "existing resources. To resolve, edit this item, " +
                "property, or operation's configuration, ensure a " +
                "valid credential for the application in context is " +
                "selected, and then save."));
        }
    }

    [ScriptAlias("SecureResourceHasProperty")]
    [Description("Checks if the named property can be read from the named Secure Resource")]
    public sealed class SecureResourceHasPropertyVariableFunction : ScalarVariableFunction
    {
        [VariableFunctionParameter(0)]
        [ScriptAlias("resource")]
        [Description("The name of the Secure Resource to test.")]
        public string? ResourceName { get; set; }

        [Description("The name of the property to test.")]
        [ScriptAlias("property")]
        [VariableFunctionParameter(1)]
        public string? PropertyName { get; set; }

        [Description("The type of the resource property to test.")]
        [ScriptAlias("type")]
        [VariableFunctionParameter(2, Optional=true)]
        public SecureResourceType? ResourceType { get; set; }

        protected override object? EvaluateScalar(IVariableFunctionContext context)
        {
            if (string.IsNullOrWhiteSpace(PropertyName)) return false;

            return SecureResourcePropertiesVariableFunction
                    .GetPropertyNames(ResourceName, ResourceType, context)
                    .Contains(PropertyName, StringComparer.InvariantCultureIgnoreCase);
        }
    }


    [ScriptAlias("SecureResourceCredential")]
    [Description("Gets the name of the Secure Credential assigned to a Secure Resource")]
    public sealed class SecureResourceCredentialVariableFunction : ScalarVariableFunction
    {
        [VariableFunctionParameter(0)]
        [ScriptAlias("resource")]
        [Description("Literal text value")]
        public string? ResourceName { get; set; }

        [Description("The type of the resource property to get.")]
        [ScriptAlias("type")]
        [VariableFunctionParameter(1, Optional=true)]
        public SecureResourceType? ResourceType { get; set; }

        protected override object EvaluateScalar(IVariableFunctionContext context)
        {
            var resourceResolutionContext = new ResourceResolutionContext(context.ProjectId);
            var resourceType = ResourceType;
            var secureResource = SecureResource.TryCreate(
                resourceType.GetValueOrDefault(), ResourceName, resourceResolutionContext)
                ?? throw SecureResourcePropertiesVariableFunction.BadResourceName(ResourceName ?? "<null>");

            return secureResource?.CredentialName
                ?? string.Empty;
        }
    }
}

jimbobmcgee

I recently had a need to work with both a Secure Resource and Secure Credential from within an Otter script, but found it was difficult to determine what property names were available when using $SecureCredentialProperty(...) and $SecureResourceProperty(...), as they were not fully-documented and are dynamically generated based on the resource/credential type (which may themselves be provided by plugins).

To compensate, I knocked up a couple of variable functions in an scratch extension of my own. I figured they might be useful for others, and so am providing them below.

Long term, they are possibly best-served living in OtterEx.dll, alongside the existing $SecureCredentialProperty function, but I don't think that it available via your GitHub for pull-requests (I had to run OtterEx.dll through a decompiler to figure out what was needed).

In any case, if they are of any use to you, you are more than welcome to them...

SecureResourcePropertiesVariableFunction.cs

using System.Collections;
using System.ComponentModel;
using System.Reflection;
using Inedo.ExecutionEngine.Executer;
using Inedo.Extensibility;
using Inedo.Extensibility.Credentials;
using Inedo.Extensibility.VariableFunctions;
using Inedo.Serialization;

using SecureCreds = Inedo.Extensibility.Credentials.SecureCredentials;

namespace Inedo.Extensions.VariableFunctions.SecureCredentials
{
    [ScriptAlias("SecureCredentialProperties")]
    [Description("Gets the properties available to a named Secure Credential.  Use `$SecureCredentialProperty()` to obtain the value.")]
    public sealed class SecureCredentialPropertiesVariableFunction : VectorVariableFunction
    {
        [VariableFunctionParameter(0)]
        [ScriptAlias("credential")]
        [Description("The name of the credential for which to fetch property names")]
        public string? CredentialName { get; set; }

        protected override IEnumerable? EvaluateVector(IVariableFunctionContext context)
            => GetProperties(CredentialName, context).Select(p => p.Name);


        internal static IEnumerable<PropertyInfo> GetProperties(string? credentialName, IVariableFunctionContext context)
        {
            var credentialResolutionContext = new CredentialResolutionContext(context.ProjectId, context.EnvironmentId);
            var secureCredential = SecureCreds.TryCreate(credentialName, credentialResolutionContext) 
                ?? throw new ExecutionFailureException(string.Concat(
                    "Could not find a Secure Credential named \"",
                    credentialName,
                    "\"; this error may occur if you renamed a credential, " +
                    "or the application or environment in context does not " +
                    "match any existing credentials. To resolve, edit this " +
                    "item, property, or operation's configuration, ensure a " +
                    "valid credential for the application/environment in " +
                    "context is selected, and then save."));
                    
            return Persistence.GetPersistentProperties(secureCredential.GetType(), true);
        }
    }

    [ScriptAlias("SecureCredentialHasProperty")]
    [Description("Checks if the named property can be read from the named Secure Credential")]
    public sealed class SecureCredentialHasPropertyVariableFunction : ScalarVariableFunction
    {
        [VariableFunctionParameter(0)]
        [ScriptAlias("credential")]
        [Description("The name of the Secure Credential to test.")]
        public string? CredentialName { get; set; }

        [Description("The name of the property to test.")]
        [ScriptAlias("property")]
        [VariableFunctionParameter(1)]
        public string? PropertyName { get; set; }

        [Description("Set to `false` to return `false` if the property is found but encrypted " +
                     "and/or requires additional script access permissions.  Set to `true` " + 
                     "(i.e. the default) to test only whether the property exists.")]
        [ScriptAlias("whenEncrypted")]
        [VariableFunctionParameter(2, Optional=true)]
        public bool WhenEncrypted { get; set; } = true;

        protected override object? EvaluateScalar(IVariableFunctionContext context)
        {
            if (string.IsNullOrWhiteSpace(PropertyName)) return false;

            var prop = SecureCredentialPropertiesVariableFunction
                        .GetProperties(CredentialName, context)
                        .FirstOrDefault(p => string.Equals(p.Name, PropertyName, StringComparison.InvariantCultureIgnoreCase));
            
            if (prop != null)
            {
                if (WhenEncrypted == false)
                {
                    var encrypted = prop.GetCustomAttribute<PersistentAttribute>()?.Encrypted ?? false;
                    if (encrypted) return false;
                }
                return true;
            }
            return false;
        }
    }
}

SecureCredentialPropertiesVariableFunction.cs

using System.Collections;
using System.ComponentModel;
using System.Reflection;
using Inedo.ExecutionEngine.Executer;
using Inedo.Extensibility;
using Inedo.Extensibility.Credentials;
using Inedo.Extensibility.VariableFunctions;
using Inedo.Serialization;

using SecureCreds = Inedo.Extensibility.Credentials.SecureCredentials;

namespace Inedo.Extensions.VariableFunctions.SecureCredentials
{
    [ScriptAlias("SecureCredentialProperties")]
    [Description("Gets the properties available to a named Secure Credential.  Use `$SecureCredentialProperty()` to obtain the value.")]
    public sealed class SecureCredentialPropertiesVariableFunction : VectorVariableFunction
    {
        [VariableFunctionParameter(0)]
        [ScriptAlias("credential")]
        [Description("The name of the credential for which to fetch property names")]
        public string? CredentialName { get; set; }

        protected override IEnumerable? EvaluateVector(IVariableFunctionContext context)
            => GetProperties(CredentialName, context).Select(p => p.Name);


        internal static IEnumerable<PropertyInfo> GetProperties(string? credentialName, IVariableFunctionContext context)
        {
            var credentialResolutionContext = new CredentialResolutionContext(context.ProjectId, context.EnvironmentId);
            var secureCredential = SecureCreds.TryCreate(credentialName, credentialResolutionContext) 
                ?? throw new ExecutionFailureException(string.Concat(
                    "Could not find a Secure Credential named \"",
                    credentialName,
                    "\"; this error may occur if you renamed a credential, " +
                    "or the application or environment in context does not " +
                    "match any existing credentials. To resolve, edit this " +
                    "item, property, or operation's configuration, ensure a " +
                    "valid credential for the application/environment in " +
                    "context is selected, and then save."));
                    
            return Persistence.GetPersistentProperties(secureCredential.GetType(), true);
        }
    }

    [ScriptAlias("SecureCredentialHasProperty")]
    [Description("Checks if the named property can be read from the named Secure Credential")]
    public sealed class SecureCredentialHasPropertyVariableFunction : ScalarVariableFunction
    {
        [VariableFunctionParameter(0)]
        [ScriptAlias("credential")]
        [Description("The name of the Secure Credential to test.")]
        public string? CredentialName { get; set; }

        [Description("The name of the property to test.")]
        [ScriptAlias("property")]
        [VariableFunctionParameter(1)]
        public string? PropertyName { get; set; }

        [Description("Set to `false` to return `false` if the property is found but encrypted " +
                     "and/or requires additional script access permissions.  Set to `true` " + 
                     "(i.e. the default) to test only whether the property exists.")]
        [ScriptAlias("whenEncrypted")]
        [VariableFunctionParameter(2, Optional=true)]
        public bool WhenEncrypted { get; set; } = true;

        protected override object? EvaluateScalar(IVariableFunctionContext context)
        {
            if (string.IsNullOrWhiteSpace(PropertyName)) return false;

            var prop = SecureCredentialPropertiesVariableFunction
                        .GetProperties(CredentialName, context)
                        .FirstOrDefault(p => string.Equals(p.Name, PropertyName, StringComparison.InvariantCultureIgnoreCase));
            
            if (prop != null)
            {
                if (WhenEncrypted == false)
                {
                    var encrypted = prop.GetCustomAttribute<PersistentAttribute>()?.Encrypted ?? false;

                    // TODO: this should probably actually check if script access has been granted, but
                    // this is buried in Inedo.Otter.Data.Tables.Credentials_Extended, and I'm not sure
                    // if that is safe to wrap in an Extension

                    // For now, we just determine if it would be encrypted, not whether we can access
                    if (encrypted) return false;
                }
                return true;
            }
            return false;
        }
    }
}

jimbobmcgee

@stevedennis I understand not wanting to go in blind on something so low-level. For what it is worth, the monkey-patch I applied has been stable, so far, in my lab environment.

Let me know if you need me to test a build, prior to Otter 2025.

jimbobmcgee

@dean-houston said in PSEval can be called as $PSEval, @PSEval or %PSEval, but null/empty returns only make sense for $PSEval:

a variable prefix ($, @, %) is more of a convenience/convention, and the prefix isn't really available in any useful context. I'm almost certain you can do stuff like $MyVar = @(1,2,3) for example.

For what it is worth, if this is the intent, then it does not match what actually occurs. The execution engine throws exceptions when you mismatch the variable types:

# mixed sigils
{
    set $ok = ""; set $no = "";
    try { set $a = "blah";          set $ok = $ok: scalar;            } catch { set $no = $no: scalar;           force normal; }
    try { set $b = @(1,2,3);        set $ok = $ok: vector-as-scalar;  } catch { set $no = $no: vector-as-scalar; force normal; }
    try { set $c = %(a: 1, b: 2);   set $ok = $ok: map-as-scalar;     } catch { set $no = $no: map-as-scalar;    force normal; }
    try { set @d = "blah";          set $ok = $ok: scalar-as-vector;  } catch { set $no = $no: scalar-as-vector; force normal; }
    try { set @e = @(1,2,3);        set $ok = $ok: vector;            } catch { set $no = $no: vector;           force normal; }
    try { set @f = %(a: 1, b: 2);   set $ok = $ok: map-as-vector;     } catch { set $no = $no: map-as-vector;    force normal; }
    try { set %g = "blah";          set $ok = $ok: scalar-as-map;     } catch { set $no = $no: scalar-as-map;    force normal; }
    try { set %h = @(1,2,3);        set $ok = $ok: vector-as-map;     } catch { set $no = $no: vector-as-map;    force normal; }
    try { set %i = %(a: 1, b: 2);   set $ok = $ok: map;               } catch { set $no = $no: map;              force normal; }

    Log-Information Mixed sigils: success${ok}, fail${no};
}

DEBUG: Beginning execution run...
ERROR: Unhandled exception: System.ArgumentException: Cannot assign a Vector value to a Scalar variable.
   at Inedo.ExecutionEngine.Executer.ExecuterThread.InitializeVariable(RuntimeVariableName name, RuntimeValue value, VariableAssignmentMode mode)
   at Inedo.ExecutionEngine.Executer.ExecuterThread.ExecuteAsync(AssignVariableStatement assignVariableStatement)
   at Inedo.ExecutionEngine.Executer.ExecuterThread.ExecuteNextAsync()
ERROR: Unhandled exception: System.ArgumentException: Cannot assign a Map value to a Scalar variable.
   at Inedo.ExecutionEngine.Executer.ExecuterThread.InitializeVariable(RuntimeVariableName name, RuntimeValue value, VariableAssignmentMode mode)
   at Inedo.ExecutionEngine.Executer.ExecuterThread.ExecuteAsync(AssignVariableStatement assignVariableStatement)
   at Inedo.ExecutionEngine.Executer.ExecuterThread.ExecuteNextAsync()
ERROR: Unhandled exception: System.ArgumentException: Cannot assign a Scalar value to a Vector variable.
   at Inedo.ExecutionEngine.Executer.ExecuterThread.InitializeVariable(RuntimeVariableName name, RuntimeValue value, VariableAssignmentMode mode)
   at Inedo.ExecutionEngine.Executer.ExecuterThread.ExecuteAsync(AssignVariableStatement assignVariableStatement)
   at Inedo.ExecutionEngine.Executer.ExecuterThread.ExecuteNextAsync()
ERROR: Unhandled exception: System.ArgumentException: Cannot assign a Map value to a Vector variable.
   at Inedo.ExecutionEngine.Executer.ExecuterThread.InitializeVariable(RuntimeVariableName name, RuntimeValue value, VariableAssignmentMode mode)
   at Inedo.ExecutionEngine.Executer.ExecuterThread.ExecuteAsync(AssignVariableStatement assignVariableStatement)
   at Inedo.ExecutionEngine.Executer.ExecuterThread.ExecuteNextAsync()
ERROR: Unhandled exception: System.ArgumentException: Cannot assign a Scalar value to a Map variable.
   at Inedo.ExecutionEngine.Executer.ExecuterThread.InitializeVariable(RuntimeVariableName name, RuntimeValue value, VariableAssignmentMode mode)
   at Inedo.ExecutionEngine.Executer.ExecuterThread.ExecuteAsync(AssignVariableStatement assignVariableStatement)
   at Inedo.ExecutionEngine.Executer.ExecuterThread.ExecuteNextAsync()
ERROR: Unhandled exception: System.ArgumentException: Cannot assign a Vector value to a Map variable.
   at Inedo.ExecutionEngine.Executer.ExecuterThread.InitializeVariable(RuntimeVariableName name, RuntimeValue value, VariableAssignmentMode mode)
   at Inedo.ExecutionEngine.Executer.ExecuterThread.ExecuteAsync(AssignVariableStatement assignVariableStatement)
   at Inedo.ExecutionEngine.Executer.ExecuterThread.ExecuteNextAsync()
INFO : Mixed sigils: success: scalar: vector: map, fail: vector-as-scalar: map-as-scalar: scalar-as-vector: map-as-vector: scalar-as-map: vector-as-map

There are also syntax elements which require specific context, such as foreach requiring a @vec (Iteration source must be a vector value).

I can certainly understand why you would not want to update the base classes so they provide the context (scalar, vector, map) to implementations but I expect it is probably the safest solution for maintaining backwards compatibility with existing authored scripts (if that information is available to you at parse-time).

The alternative is to let the script author pass it along as an optional property to $PSEval() (similar to $GetVariableValue()), but this introduces another character which would then need to be escaped within the embedded Powershell (i.e. ,), and that probably would break authored scripts.

jimbobmcgee

@dean-houston

PSExec (i.e. Execute-Powershell) can capture variables, but not output streams.

Understood with regards to output stream.

However, no joy with this (i.e. capturing variables) either...

set $In = 12345;
set $Out = '<unset>';

Execute-PowerShell
(
	Text: >-|>
		Write-Verbose "Got some input: $In here...";
		$Out = "This is nice";
		Write-Verbose "inside the script, we have: $Out";
	>-|>,
	Verbose: true
);

Log-Information Outside the script, we have $Out;

DEBUG: Using Windows PowerShell 5.1...
DEBUG: Importing Out...
DEBUG: Importing In...
DEBUG: Got some input: 12345 here...
DEBUG: inside the script, we have: This is nice
INFO : Outside the script, we have <unset>

So something like this:

set $hello = world;
$PSExec >>
  $hello = 'dears';
>>;
Log-Information Hello $hello;

Calling $PSExec like this throws Unexpected token $. I suspect you meant to write PSExec (the operation) or $PSEval (the variable function).

However, substituting PSExec >> does not capture the output variable.

set $hello = world;
PSExec >>
    $hello = 'dears';
>>;
Log-Information 1: Hello $hello;

DEBUG  Using Windows PowerShell 5.1...
DEBUG  Importing hello...
INFO   1: Hello world

Substituting set $x = $PSEval(>>...>>) does not even execute: The term '>>' is not recognized as the name of a cmdlet, function, script file, or operable program

set $hello = world;
set $x = $PSEval(>>
   $hello = 'dears';
>>);
Log-Information 2: Hello $hello;

DEBUG: Using Windows PowerShell 5.1...
ERROR: The term '>>' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
ERROR: The term 'world' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
ERROR: The term '>>' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
ERROR: PSEVal: PowerShell script failed with an error (see previous log messages).

set $hello = world;
set $x = $PSEval(>>
   `$hello = 'dears';
>>);
Log-Information 3: Hello $hello;

DEBUG: Using Windows PowerShell 5.1...
DEBUG: Importing hello...
ERROR: The term '>>' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
ERROR: The term '>>' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
ERROR: PSEVal: PowerShell script failed with an error (see previous log messages).

set $hello = world;
set $x = $PSEval(>>`$hello = 'dears';>>);
Log-Information 4: Hello $hello;

DEBUG: Using Windows PowerShell 5.1...
DEBUG: Importing hello...
ERROR: The term '>>' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
ERROR: The term '>>' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
ERROR: PSEVal: PowerShell script failed with an error (see previous log messages).

Substituting set $x = $PSEval("...") does not capture the output variable:

set $hello = world;
set $x = $PSEval("
    $hello = 'dears';
");
Log-Information 5: Hello $hello;

DEBUG: Using Windows PowerShell 5.1...
DEBUG: Importing hello...
INFO : 5: Hello world

set $hello = world;
set $x = $PSEval("
    `$hello = 'dears';
");
Log-Information 6: Hello $hello;

DEBUG: Using Windows PowerShell 5.1...
DEBUG: Importing hello...
INFO : 6: Hello world

I think this might actually make my point for me. If you can get this wrong instinctively, what hope do I have (or my less code-savvy colleagues)?

jimbobmcgee

@dean-houston Submitted as https://github.com/Inedo/inedox-inedocore/pull/180.

jimbobmcgee

@dean-houston I hadn't noticed Execute-Powershell automatically capturing output variables. I'll do some more tests and see if it works for me.

This is probably a reasonable candidate for the /literal proposal described in the documentation, if you are still taking votes on that:

We're considering adding this using a /literal decorator at the end of a quoted or swim string. For example, "Not $escaped"/literal.

If you have any interest in this, please submit a ticket with how you would plan to use this, and how it would be helpful; this is currently tracked as IEE-20, and we will link your ticket and seek to prioritize this.

jimbobmcgee

@dean-houston I'm sure that fixing the syntax would not be straightforward, but introducing @ListSet and %MapSet variable functions in lieu is probably a good enough workaround for nearly all use cases where someone would want to do this.

jimbobmcgee

There does not appear to be a means to capture the output stream from Execute-Powershell, nor does it appear possible to obtain the value of a variable set inside the Powershell script back to the enclosing OtterScript.

It is possible to get a result back from $PSEval() but the options to both $PSEval and Execute-Powershell are not equivalent. For instance:

Execute-Powershell uses a Text property which does not interpolate OtterScript variables at the script level (it extracts and dispatches them), which means it better handles the $variables that are specific to the Powershell script
$PSEval() does interpolate OtterScript variables, which means all the variables in the Powershell script have to be backtick-escaped, whether they are specific to the Powershell script or defined in the OtterScript context
$PSEval() does not like parentheses; many of these have to be escaped as well, and it is not always clear which ones
$PSEval() does not like newlines; these can be within "swim" strings, but these still require escaping at least the variables
$PSEval() is not (currently) particularly supportive of scripts with varying output (see #4920).

Correctly escaping all the parentheses and variables in any Powershell longer than a couple of lines is an exercise in torture.

Execute-Powershell is clearly the better choice for more complex scripts, but seems to lack the means to return anything back to the caller.

As such, could Execute-Powershell be at least augmented with an output parameter, to capture anything in the Powershell output stream back to target variable? (noting it should be made aware of scalar, vector or map context)

Alternatively/additionally, it would be useful for Execute-Powershell to export variables back to the calling OtterScript context. Clobbering existing variables across the board might not be the right approach (so as not to break existing scripts), but perhaps Execute-Powershell could be given an optional input parameter which is a list of variable names to export, so the capture is opt-in?

(I assume this was the intent of including them in ExecutePowerShellJob+Result, and that Result is correctly populated...)

# context aware output..
Execute-Powershell (
  Text: >>
    Write-Output "abc"
  >>, 
  OutputVariable => $foo
);

Execute-Powershell (
  Text: >>
    Write-Output "abc"
    Write-Output "def"
    Write-Output "ghi"
  >>, 
  OutputVariable => @bar
);

Execute-Powershell (
  Text: >>
    Write-Output @{a = 1; b = 2; c = 3} 
  >>, 
  OutputVariable => %baz
);

# or, with capture...
Execute-Powershell (
  Text: >>
    $a = 10 + 5
    $b = @($a, 10)
    $c = @{a = $a; b = $b}
  >>,
  CaptureVariables: @(a, b, c)
);
Log-Information $a;            # -> 15
Log-Information $ToJson(@b);   # -> ["15", "10"]
Log-Information $ToJson(%c);   # -> { "a": "15", "b": ["15", "10"] }

PS: various crimes against humanity, trying to escape properly are below, to demonstrate the difficulties...

set $foo = "hello 'world'";
set @bar = @();

# 'natural' approach, newlines, no escapes: (otter compile error: 'Expected ;')
{
    set @a = @PSEval(
        for ($i = 1; $i -le 5; $i++) {
            Write-Output ('{0} {1}' -f $foo,$i)
        }
    );
    Log-Information `@a: $ToJson(@a);
}

# flatten newlines, no escapes: (otter runtime error: 'cannot resolve variable $i')
set @a = @PSEval(for ($i = 1; $i -le 5; $i++) { Write-Output ('{0} {1}' -f $foo,$i) });

# flatten newlines, no escapes: (otter runtime error: 'invalid use of vector expression in scalar')
set $a = $PSEval(for ($i = 1; $i -le 5; $i++) { Write-Output ('{0} {1}' -f $foo,$i) });

# flatten newlines, escape parens: (otter runtime error: 'cannot resolve variable $i')
set @a = @PSEval(for `($i = 1; $i -le 5; $i++`) { Write-Output `('{0} {1}' -f $foo,$i`) });

# flatten newlines, escape Powershell sigils and parens: ($foo is interpolated, not captured; Powershell syntax error at '-f hello')
set @a = @PSEval(for `(`$i = 1; `$i -le 5; `$i++`) { Write-Output `('{0} {1}' -f $foo,`$i`) });

# flatten newlines, escape all sigils and parens: ($foo is captured; powershell runtime error: 'missing closing ")"')
set @a = @PSEval(for (`$i = 1; `$i -le 5; `$i++`) { Write-Output ('{0} {1}' -f `$foo,`$i`) });

# flatten newlines, escape all sigils and parens: ($foo is captured; powershell runtime error: 'missing closing ")"')
set @a = @PSEval(for `(`$i = 1; `$i -le 5; `$i++`) { Write-Output `('{0`} {1`}' -f `$foo,`$i`) });

# as a swim string; same as above: ('cannot resolve $i' when unescaped; 'missing closing ")"' when escaped)
set @a = @PSEval(>-|>
    for ($i = 1; $i -le 5; $i++) { 
        Write-Output ('{0} {1}' -f $foo,$i)
    }
>-|>);

# as a variable, loaded by swim string; escaping all sigils: (captures $foo, this is the first one that works)
set $ps = >-|>
    for (`$i = 1; `$i -le 5; `$i++) { 
        Write-Output ('{0} {1}' -f `$foo,`$i)
    }
>-|>;
set @a = @PSEval($ps);

# executes with the 'natural' syntactic approach; captures $foo but has no means to return output
Execute-Powershell(
    Text: >-|>
        for ($i = 1; $i -le 5; $i++) { 
            Write-Output ('{0} {1}' -f $foo,$i)
        }
    >-|>
);

# similarly 'natural'; captures $foo and $bar, but does not populate @bar
Execute-Powershell(
    Text: >-|>
        $bar = @()
        for ($i = 1; $i -le 5; $i++) { 
            $bar += ('{0} {1}' -f $foo,$i)
        }
    >-|>
);

(Reposted from Github on request)

jimbobmcgee

@jimbobmcgee

Best posts made by jimbobmcgee

Latest posts made by jimbobmcgee

Some more-targeted stuff I've tried

SecureResourcePropertiesVariableFunction.cs

SecureResourcePropertiesVariableFunction.cs

SecureCredentialPropertiesVariableFunction.cs