@dean-houston I'm sure that fixing the syntax would not be straightforward, but introducing @ListSet and %MapSet variable functions in lieu is probably a good enough workaround for nearly all use cases where someone would want to do this.

I recently had a need to work with both a Secure Resource and Secure Credential from within an Otter script, but found it was difficult to determine what property names were available when using $SecureCredentialProperty(...) and $SecureResourceProperty(...), as they were not fully-documented and are dynamically generated based on the resource/credential type (which may themselves be provided by plugins).

To compensate, I knocked up a couple of variable functions in an scratch extension of my own. I figured they might be useful for others, and so am providing them below.

Long term, they are possibly best-served living in OtterEx.dll, alongside the existing $SecureCredentialProperty function, but I don't think that it available via your GitHub for pull-requests (I had to run OtterEx.dll through a decompiler to figure out what was needed).

In any case, if they are of any use to you, you are more than welcome to them...

SecureResourcePropertiesVariableFunction.cs

using System.Collections;
using System.ComponentModel;
using System.Reflection;
using Inedo.ExecutionEngine.Executer;
using Inedo.Extensibility;
using Inedo.Extensibility.Credentials;
using Inedo.Extensibility.VariableFunctions;
using Inedo.Serialization;

using SecureCreds = Inedo.Extensibility.Credentials.SecureCredentials;

namespace Inedo.Extensions.VariableFunctions.SecureCredentials
{
    [ScriptAlias("SecureCredentialProperties")]
    [Description("Gets the properties available to a named Secure Credential.  Use `$SecureCredentialProperty()` to obtain the value.")]
    public sealed class SecureCredentialPropertiesVariableFunction : VectorVariableFunction
    {
        [VariableFunctionParameter(0)]
        [ScriptAlias("credential")]
        [Description("The name of the credential for which to fetch property names")]
        public string? CredentialName { get; set; }

        protected override IEnumerable? EvaluateVector(IVariableFunctionContext context)
            => GetProperties(CredentialName, context).Select(p => p.Name);


        internal static IEnumerable<PropertyInfo> GetProperties(string? credentialName, IVariableFunctionContext context)
        {
            var credentialResolutionContext = new CredentialResolutionContext(context.ProjectId, context.EnvironmentId);
            var secureCredential = SecureCreds.TryCreate(credentialName, credentialResolutionContext) 
                ?? throw new ExecutionFailureException(string.Concat(
                    "Could not find a Secure Credential named \"",
                    credentialName,
                    "\"; this error may occur if you renamed a credential, " +
                    "or the application or environment in context does not " +
                    "match any existing credentials. To resolve, edit this " +
                    "item, property, or operation's configuration, ensure a " +
                    "valid credential for the application/environment in " +
                    "context is selected, and then save."));
                    
            return Persistence.GetPersistentProperties(secureCredential.GetType(), true);
        }
    }

    [ScriptAlias("SecureCredentialHasProperty")]
    [Description("Checks if the named property can be read from the named Secure Credential")]
    public sealed class SecureCredentialHasPropertyVariableFunction : ScalarVariableFunction
    {
        [VariableFunctionParameter(0)]
        [ScriptAlias("credential")]
        [Description("The name of the Secure Credential to test.")]
        public string? CredentialName { get; set; }

        [Description("The name of the property to test.")]
        [ScriptAlias("property")]
        [VariableFunctionParameter(1)]
        public string? PropertyName { get; set; }

        [Description("Set to `false` to return `false` if the property is found but encrypted " +
                     "and/or requires additional script access permissions.  Set to `true` " + 
                     "(i.e. the default) to test only whether the property exists.")]
        [ScriptAlias("whenEncrypted")]
        [VariableFunctionParameter(2, Optional=true)]
        public bool WhenEncrypted { get; set; } = true;

        protected override object? EvaluateScalar(IVariableFunctionContext context)
        {
            if (string.IsNullOrWhiteSpace(PropertyName)) return false;

            var prop = SecureCredentialPropertiesVariableFunction
                        .GetProperties(CredentialName, context)
                        .FirstOrDefault(p => string.Equals(p.Name, PropertyName, StringComparison.InvariantCultureIgnoreCase));
            
            if (prop != null)
            {
                if (WhenEncrypted == false)
                {
                    var encrypted = prop.GetCustomAttribute<PersistentAttribute>()?.Encrypted ?? false;
                    if (encrypted) return false;
                }
                return true;
            }
            return false;
        }
    }
}

SecureCredentialPropertiesVariableFunction.cs

using System.Collections;
using System.ComponentModel;
using System.Reflection;
using Inedo.ExecutionEngine.Executer;
using Inedo.Extensibility;
using Inedo.Extensibility.Credentials;
using Inedo.Extensibility.VariableFunctions;
using Inedo.Serialization;

using SecureCreds = Inedo.Extensibility.Credentials.SecureCredentials;

namespace Inedo.Extensions.VariableFunctions.SecureCredentials
{
    [ScriptAlias("SecureCredentialProperties")]
    [Description("Gets the properties available to a named Secure Credential.  Use `$SecureCredentialProperty()` to obtain the value.")]
    public sealed class SecureCredentialPropertiesVariableFunction : VectorVariableFunction
    {
        [VariableFunctionParameter(0)]
        [ScriptAlias("credential")]
        [Description("The name of the credential for which to fetch property names")]
        public string? CredentialName { get; set; }

        protected override IEnumerable? EvaluateVector(IVariableFunctionContext context)
            => GetProperties(CredentialName, context).Select(p => p.Name);


        internal static IEnumerable<PropertyInfo> GetProperties(string? credentialName, IVariableFunctionContext context)
        {
            var credentialResolutionContext = new CredentialResolutionContext(context.ProjectId, context.EnvironmentId);
            var secureCredential = SecureCreds.TryCreate(credentialName, credentialResolutionContext) 
                ?? throw new ExecutionFailureException(string.Concat(
                    "Could not find a Secure Credential named \"",
                    credentialName,
                    "\"; this error may occur if you renamed a credential, " +
                    "or the application or environment in context does not " +
                    "match any existing credentials. To resolve, edit this " +
                    "item, property, or operation's configuration, ensure a " +
                    "valid credential for the application/environment in " +
                    "context is selected, and then save."));
                    
            return Persistence.GetPersistentProperties(secureCredential.GetType(), true);
        }
    }

    [ScriptAlias("SecureCredentialHasProperty")]
    [Description("Checks if the named property can be read from the named Secure Credential")]
    public sealed class SecureCredentialHasPropertyVariableFunction : ScalarVariableFunction
    {
        [VariableFunctionParameter(0)]
        [ScriptAlias("credential")]
        [Description("The name of the Secure Credential to test.")]
        public string? CredentialName { get; set; }

        [Description("The name of the property to test.")]
        [ScriptAlias("property")]
        [VariableFunctionParameter(1)]
        public string? PropertyName { get; set; }

        [Description("Set to `false` to return `false` if the property is found but encrypted " +
                     "and/or requires additional script access permissions.  Set to `true` " + 
                     "(i.e. the default) to test only whether the property exists.")]
        [ScriptAlias("whenEncrypted")]
        [VariableFunctionParameter(2, Optional=true)]
        public bool WhenEncrypted { get; set; } = true;

        protected override object? EvaluateScalar(IVariableFunctionContext context)
        {
            if (string.IsNullOrWhiteSpace(PropertyName)) return false;

            var prop = SecureCredentialPropertiesVariableFunction
                        .GetProperties(CredentialName, context)
                        .FirstOrDefault(p => string.Equals(p.Name, PropertyName, StringComparison.InvariantCultureIgnoreCase));
            
            if (prop != null)
            {
                if (WhenEncrypted == false)
                {
                    var encrypted = prop.GetCustomAttribute<PersistentAttribute>()?.Encrypted ?? false;

                    // TODO: this should probably actually check if script access has been granted, but
                    // this is buried in Inedo.Otter.Data.Tables.Credentials_Extended, and I'm not sure
                    // if that is safe to wrap in an Extension

                    // For now, we just determine if it would be encrypted, not whether we can access
                    if (encrypted) return false;
                }
                return true;
            }
            return false;
        }
    }
}

@stevedennis I understand not wanting to go in blind on something so low-level. For what it is worth, the monkey-patch I applied has been stable, so far, in my lab environment.

Let me know if you need me to test a build, prior to Otter 2025.

@dean-houston said in PSEval can be called as $PSEval, @PSEval or %PSEval, but null/empty returns only make sense for $PSEval:

a variable prefix ($, @, %) is more of a convenience/convention, and the prefix isn't really available in any useful context. I'm almost certain you can do stuff like $MyVar = @(1,2,3) for example.

For what it is worth, if this is the intent, then it does not match what actually occurs. The execution engine throws exceptions when you mismatch the variable types:

# mixed sigils
{
    set $ok = ""; set $no = "";
    try { set $a = "blah";          set $ok = $ok: scalar;            } catch { set $no = $no: scalar;           force normal; }
    try { set $b = @(1,2,3);        set $ok = $ok: vector-as-scalar;  } catch { set $no = $no: vector-as-scalar; force normal; }
    try { set $c = %(a: 1, b: 2);   set $ok = $ok: map-as-scalar;     } catch { set $no = $no: map-as-scalar;    force normal; }
    try { set @d = "blah";          set $ok = $ok: scalar-as-vector;  } catch { set $no = $no: scalar-as-vector; force normal; }
    try { set @e = @(1,2,3);        set $ok = $ok: vector;            } catch { set $no = $no: vector;           force normal; }
    try { set @f = %(a: 1, b: 2);   set $ok = $ok: map-as-vector;     } catch { set $no = $no: map-as-vector;    force normal; }
    try { set %g = "blah";          set $ok = $ok: scalar-as-map;     } catch { set $no = $no: scalar-as-map;    force normal; }
    try { set %h = @(1,2,3);        set $ok = $ok: vector-as-map;     } catch { set $no = $no: vector-as-map;    force normal; }
    try { set %i = %(a: 1, b: 2);   set $ok = $ok: map;               } catch { set $no = $no: map;              force normal; }

    Log-Information Mixed sigils: success${ok}, fail${no};
}

DEBUG: Beginning execution run...
ERROR: Unhandled exception: System.ArgumentException: Cannot assign a Vector value to a Scalar variable.
   at Inedo.ExecutionEngine.Executer.ExecuterThread.InitializeVariable(RuntimeVariableName name, RuntimeValue value, VariableAssignmentMode mode)
   at Inedo.ExecutionEngine.Executer.ExecuterThread.ExecuteAsync(AssignVariableStatement assignVariableStatement)
   at Inedo.ExecutionEngine.Executer.ExecuterThread.ExecuteNextAsync()
ERROR: Unhandled exception: System.ArgumentException: Cannot assign a Map value to a Scalar variable.
   at Inedo.ExecutionEngine.Executer.ExecuterThread.InitializeVariable(RuntimeVariableName name, RuntimeValue value, VariableAssignmentMode mode)
   at Inedo.ExecutionEngine.Executer.ExecuterThread.ExecuteAsync(AssignVariableStatement assignVariableStatement)
   at Inedo.ExecutionEngine.Executer.ExecuterThread.ExecuteNextAsync()
ERROR: Unhandled exception: System.ArgumentException: Cannot assign a Scalar value to a Vector variable.
   at Inedo.ExecutionEngine.Executer.ExecuterThread.InitializeVariable(RuntimeVariableName name, RuntimeValue value, VariableAssignmentMode mode)
   at Inedo.ExecutionEngine.Executer.ExecuterThread.ExecuteAsync(AssignVariableStatement assignVariableStatement)
   at Inedo.ExecutionEngine.Executer.ExecuterThread.ExecuteNextAsync()
ERROR: Unhandled exception: System.ArgumentException: Cannot assign a Map value to a Vector variable.
   at Inedo.ExecutionEngine.Executer.ExecuterThread.InitializeVariable(RuntimeVariableName name, RuntimeValue value, VariableAssignmentMode mode)
   at Inedo.ExecutionEngine.Executer.ExecuterThread.ExecuteAsync(AssignVariableStatement assignVariableStatement)
   at Inedo.ExecutionEngine.Executer.ExecuterThread.ExecuteNextAsync()
ERROR: Unhandled exception: System.ArgumentException: Cannot assign a Scalar value to a Map variable.
   at Inedo.ExecutionEngine.Executer.ExecuterThread.InitializeVariable(RuntimeVariableName name, RuntimeValue value, VariableAssignmentMode mode)
   at Inedo.ExecutionEngine.Executer.ExecuterThread.ExecuteAsync(AssignVariableStatement assignVariableStatement)
   at Inedo.ExecutionEngine.Executer.ExecuterThread.ExecuteNextAsync()
ERROR: Unhandled exception: System.ArgumentException: Cannot assign a Vector value to a Map variable.
   at Inedo.ExecutionEngine.Executer.ExecuterThread.InitializeVariable(RuntimeVariableName name, RuntimeValue value, VariableAssignmentMode mode)
   at Inedo.ExecutionEngine.Executer.ExecuterThread.ExecuteAsync(AssignVariableStatement assignVariableStatement)
   at Inedo.ExecutionEngine.Executer.ExecuterThread.ExecuteNextAsync()
INFO : Mixed sigils: success: scalar: vector: map, fail: vector-as-scalar: map-as-scalar: scalar-as-vector: map-as-vector: scalar-as-map: vector-as-map

There are also syntax elements which require specific context, such as foreach requiring a @vec (Iteration source must be a vector value).

I can certainly understand why you would not want to update the base classes so they provide the context (scalar, vector, map) to implementations but I expect it is probably the safest solution for maintaining backwards compatibility with existing authored scripts (if that information is available to you at parse-time).

The alternative is to let the script author pass it along as an optional property to $PSEval() (similar to $GetVariableValue()), but this introduces another character which would then need to be escaped within the embedded Powershell (i.e. ,), and that probably would break authored scripts.

@dean-houston

PSExec (i.e. Execute-Powershell) can capture variables, but not output streams.

Understood with regards to output stream.

However, no joy with this (i.e. capturing variables) either...

set $In = 12345;
set $Out = '<unset>';

Execute-PowerShell
(
	Text: >-|>
		Write-Verbose "Got some input: $In here...";
		$Out = "This is nice";
		Write-Verbose "inside the script, we have: $Out";
	>-|>,
	Verbose: true
);

Log-Information Outside the script, we have $Out;

DEBUG: Using Windows PowerShell 5.1...
DEBUG: Importing Out...
DEBUG: Importing In...
DEBUG: Got some input: 12345 here...
DEBUG: inside the script, we have: This is nice
INFO : Outside the script, we have <unset>

So something like this:

set $hello = world;
$PSExec >>
  $hello = 'dears';
>>;
Log-Information Hello $hello;

Calling $PSExec like this throws Unexpected token $. I suspect you meant to write PSExec (the operation) or $PSEval (the variable function).

However, substituting PSExec >> does not capture the output variable.

set $hello = world;
PSExec >>
    $hello = 'dears';
>>;
Log-Information 1: Hello $hello;

DEBUG  Using Windows PowerShell 5.1...
DEBUG  Importing hello...
INFO   1: Hello world

Substituting set $x = $PSEval(>>...>>) does not even execute: The term '>>' is not recognized as the name of a cmdlet, function, script file, or operable program

set $hello = world;
set $x = $PSEval(>>
   $hello = 'dears';
>>);
Log-Information 2: Hello $hello;

DEBUG: Using Windows PowerShell 5.1...
ERROR: The term '>>' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
ERROR: The term 'world' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
ERROR: The term '>>' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
ERROR: PSEVal: PowerShell script failed with an error (see previous log messages).

set $hello = world;
set $x = $PSEval(>>
   `$hello = 'dears';
>>);
Log-Information 3: Hello $hello;

DEBUG: Using Windows PowerShell 5.1...
DEBUG: Importing hello...
ERROR: The term '>>' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
ERROR: The term '>>' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
ERROR: PSEVal: PowerShell script failed with an error (see previous log messages).

set $hello = world;
set $x = $PSEval(>>`$hello = 'dears';>>);
Log-Information 4: Hello $hello;

DEBUG: Using Windows PowerShell 5.1...
DEBUG: Importing hello...
ERROR: The term '>>' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
ERROR: The term '>>' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
ERROR: PSEVal: PowerShell script failed with an error (see previous log messages).

Substituting set $x = $PSEval("...") does not capture the output variable:

set $hello = world;
set $x = $PSEval("
    $hello = 'dears';
");
Log-Information 5: Hello $hello;

DEBUG: Using Windows PowerShell 5.1...
DEBUG: Importing hello...
INFO : 5: Hello world

set $hello = world;
set $x = $PSEval("
    `$hello = 'dears';
");
Log-Information 6: Hello $hello;

DEBUG: Using Windows PowerShell 5.1...
DEBUG: Importing hello...
INFO : 6: Hello world

I think this might actually make my point for me. If you can get this wrong instinctively, what hope do I have (or my less code-savvy colleagues)?

@dean-houston Submitted as https://github.com/Inedo/inedox-inedocore/pull/180.

@dean-houston I hadn't noticed Execute-Powershell automatically capturing output variables. I'll do some more tests and see if it works for me.

This is probably a reasonable candidate for the /literal proposal described in the documentation, if you are still taking votes on that:

We're considering adding this using a /literal decorator at the end of a quoted or swim string. For example, "Not $escaped"/literal.

If you have any interest in this, please submit a ticket with how you would plan to use this, and how it would be helpful; this is currently tracked as IEE-20, and we will link your ticket and seek to prioritize this.

@dean-houston I'm sure that fixing the syntax would not be straightforward, but introducing @ListSet and %MapSet variable functions in lieu is probably a good enough workaround for nearly all use cases where someone would want to do this.

There does not appear to be a means to capture the output stream from Execute-Powershell, nor does it appear possible to obtain the value of a variable set inside the Powershell script back to the enclosing OtterScript.

It is possible to get a result back from $PSEval() but the options to both $PSEval and Execute-Powershell are not equivalent. For instance:

1. Execute-Powershell uses a Text property which does not interpolate OtterScript variables at the script level (it extracts and dispatches them), which means it better handles the $variables that are specific to the Powershell script
2. $PSEval() does interpolate OtterScript variables, which means all the variables in the Powershell script have to be backtick-escaped, whether they are specific to the Powershell script or defined in the OtterScript context
3. $PSEval() does not like parentheses; many of these have to be escaped as well, and it is not always clear which ones
4. $PSEval() does not like newlines; these can be within "swim" strings, but these still require escaping at least the variables
5. $PSEval() is not (currently) particularly supportive of scripts with varying output (see #4920).

Correctly escaping all the parentheses and variables in any Powershell longer than a couple of lines is an exercise in torture.

Execute-Powershell is clearly the better choice for more complex scripts, but seems to lack the means to return anything back to the caller.

As such, could Execute-Powershell be at least augmented with an output parameter, to capture anything in the Powershell output stream back to target variable? (noting it should be made aware of scalar, vector or map context)

Alternatively/additionally, it would be useful for Execute-Powershell to export variables back to the calling OtterScript context. Clobbering existing variables across the board might not be the right approach (so as not to break existing scripts), but perhaps Execute-Powershell could be given an optional input parameter which is a list of variable names to export, so the capture is opt-in?

(I assume this was the intent of including them in ExecutePowerShellJob+Result, and that Result is correctly populated...)

# context aware output..
Execute-Powershell (
  Text: >>
    Write-Output "abc"
  >>, 
  OutputVariable => $foo
);

Execute-Powershell (
  Text: >>
    Write-Output "abc"
    Write-Output "def"
    Write-Output "ghi"
  >>, 
  OutputVariable => @bar
);

Execute-Powershell (
  Text: >>
    Write-Output @{a = 1; b = 2; c = 3} 
  >>, 
  OutputVariable => %baz
);

# or, with capture...
Execute-Powershell (
  Text: >>
    $a = 10 + 5
    $b = @($a, 10)
    $c = @{a = $a; b = $b}
  >>,
  CaptureVariables: @(a, b, c)
);
Log-Information $a;            # -> 15
Log-Information $ToJson(@b);   # -> ["15", "10"]
Log-Information $ToJson(%c);   # -> { "a": "15", "b": ["15", "10"] }

PS: various crimes against humanity, trying to escape properly are below, to demonstrate the difficulties...

set $foo = "hello 'world'";
set @bar = @();

# 'natural' approach, newlines, no escapes: (otter compile error: 'Expected ;')
{
    set @a = @PSEval(
        for ($i = 1; $i -le 5; $i++) {
            Write-Output ('{0} {1}' -f $foo,$i)
        }
    );
    Log-Information `@a: $ToJson(@a);
}

# flatten newlines, no escapes: (otter runtime error: 'cannot resolve variable $i')
set @a = @PSEval(for ($i = 1; $i -le 5; $i++) { Write-Output ('{0} {1}' -f $foo,$i) });

# flatten newlines, no escapes: (otter runtime error: 'invalid use of vector expression in scalar')
set $a = $PSEval(for ($i = 1; $i -le 5; $i++) { Write-Output ('{0} {1}' -f $foo,$i) });

# flatten newlines, escape parens: (otter runtime error: 'cannot resolve variable $i')
set @a = @PSEval(for `($i = 1; $i -le 5; $i++`) { Write-Output `('{0} {1}' -f $foo,$i`) });

# flatten newlines, escape Powershell sigils and parens: ($foo is interpolated, not captured; Powershell syntax error at '-f hello')
set @a = @PSEval(for `(`$i = 1; `$i -le 5; `$i++`) { Write-Output `('{0} {1}' -f $foo,`$i`) });

# flatten newlines, escape all sigils and parens: ($foo is captured; powershell runtime error: 'missing closing ")"')
set @a = @PSEval(for (`$i = 1; `$i -le 5; `$i++`) { Write-Output ('{0} {1}' -f `$foo,`$i`) });

# flatten newlines, escape all sigils and parens: ($foo is captured; powershell runtime error: 'missing closing ")"')
set @a = @PSEval(for `(`$i = 1; `$i -le 5; `$i++`) { Write-Output `('{0`} {1`}' -f `$foo,`$i`) });

# as a swim string; same as above: ('cannot resolve $i' when unescaped; 'missing closing ")"' when escaped)
set @a = @PSEval(>-|>
    for ($i = 1; $i -le 5; $i++) { 
        Write-Output ('{0} {1}' -f $foo,$i)
    }
>-|>);

# as a variable, loaded by swim string; escaping all sigils: (captures $foo, this is the first one that works)
set $ps = >-|>
    for (`$i = 1; `$i -le 5; `$i++) { 
        Write-Output ('{0} {1}' -f `$foo,`$i)
    }
>-|>;
set @a = @PSEval($ps);

# executes with the 'natural' syntactic approach; captures $foo but has no means to return output
Execute-Powershell(
    Text: >-|>
        for ($i = 1; $i -le 5; $i++) { 
            Write-Output ('{0} {1}' -f $foo,$i)
        }
    >-|>
);

# similarly 'natural'; captures $foo and $bar, but does not populate @bar
Execute-Powershell(
    Text: >-|>
        $bar = @()
        for ($i = 1; $i -le 5; $i++) { 
            $bar += ('{0} {1}' -f $foo,$i)
        }
    >-|>
);

(Reposted from Github on request)

Consider the arbitrary OtterScript...

set @fileLines = @PSEval(Get-Content -LiteralPath 'X:\PathTo\File')
if $ListCount(@fileLines) != 0 {
   Log-Debug "OK"
}

If the Get-Content PowerShell cmdlet returns no lines, then the current approach is to attempt to set the return value of @PSEval to be an empty string, rather than an empty array. This in turn causes Otter to throw a rather nasty exception:

Unhandled exception: System.ArgumentException: Cannot assign a Scalar value to a Vector variable.
   at Inedo.ExecutionEngine.Executer.ExecuterThread.InitializeVariable(RuntimeVariableName name, RuntimeValue value, VariableAssignmentMode mode)
   at Inedo.ExecutionEngine.Executer.ExecuterThread.ExecuteAsync(AssignVariableStatement assignVariableStatement)
   at Inedo.ExecutionEngine.Executer.ExecuterThread.ExecuteNextAsync()

From an end-user perspective, trying to resolve whether a PSEval would return scalar, vector or hash is therefore fraught -- you have to know in advance whether the script will return 0, 1 or many items, and cannot safely deduce this by simply calling in vector context and testing the resulting length.

The best I can think of is try { set global @r = @PSEval(...); } catch { set global @r = @(); }, which technically proceeds, but the exception is still thrown and the overall job still results in an error state (assuming no subsequent force normal).

Ideally, PSEval should behave appropriately on a null/empty value, depending on whether it was called as $PSEval, @PSEval or %PSEval -- in scalar context, it is fine to return the empty string; in vector context, it would be better if it returned an empty vector; in hash context, an empty hash.

I appreciate that this might not be resolvable straight away, as you probably need assistance from whatever parses and invokes the script line upstream (maybe ExecuterThread?) to pass the desired RuntimeValueType to you. I expect it either needs to be suppled by extending IVariableFunctionContext (which may have side-effects on implementors of that interface); or—more likely—be introduced as a protected internal settable property of VariableFunction itself.

Alternatively, if it is possible to alter RuntimeValue so it has a dedicated constructor/static sentinel value for empty values, ExecuterThread.InitializeVariable might be better-placed to handle the scalar/vector/hash decision, and you can just return the sentinel from EvaluateAsync.

(Reposted from Github on request)

It does not appear to be possible to set the value of a list or map element, where the index or key is stored in a variable.

The formal grammar suggests...

<assign_variable_statement> ::=
   set [local | global] ( /variable_expression/ | /indexed_expression/ ) = /literal_expression/;

variable_expression:
A variable type identifier ($, @, or %) immediately followed by one of:

• simple name - follows same rules as any_name
• explicit name - a left curly brace ({), followed by of characters with the same rules as any_name but that also allow spaces, followed by a right curly brace (})

indexed_expression:
A variable_expression for a vector (@) or map (%) type, immediately followed one of:

• left bracket ([), scalar_expression, then right bracket (])
• dot (.) then scalar_expression

...which implies that both...

set %Map.$k = something;
set @Vec[$i] = something;

...should be possible, but these throw various errors at the execution engine level, e.g.:

Unhandled exception: System.InvalidCastException: Unable to cast object of type 'System.ArrayEnumerator' to type 'System.Collections.Generic.IEnumerator`1[Inedo.ExecutionEngine.RuntimeValue]'.
  at Inedo.ExecutionEngine.RuntimeListValue.GetEnumerator()
  at Inedo.ExecutionEngine.Mapping.CoreScriptPropertyMapper.CoerceValue(RuntimeValue value, PropertyInfo property, Type type)
  at Inedo.Otter.Service.PlanExecuter.ExecutionVariableEvaluationContext.GetVariableFunctionInternal(RuntimeVariableName functionName, IList`1 arguments)
  at Inedo.Otter.Service.PlanExecuter.ExecutionVariableEvaluationContext.TryEvaluateFunctionAsync(RuntimeVariableName functionName, IList`1 arguments)
  at Inedo.ExecutionEngine.Variables.FunctionTextValue.EvaluateAsync(IVariableEvaluationContext context)
  at Inedo.ExecutionEngine.Variables.ProcessedString.EvaluateValueAsync(IVariableEvaluationContext context)
  at Inedo.ExecutionEngine.Executer.ExecuterThread.EvaluateAsync(EqualityPredicate equalityPredicate)
  at Inedo.ExecutionEngine.Executer.ExecuterThread.ExecuteAsync(PredicateStatement predicateStatement)
  at Inedo.ExecutionEngine.Executer.ExecuterThread.ExecuteNextAsync()

In the absence of syntax support, the sanest workaround is probably a pair of variable functions -- e.g. @ListSet(@Vec, $i, value) and %MapSet(%Map, $k, value), which could perform this operation.

My current workaround is to...

set %Map = %MapAdd(%MapRemove(%Map, $key), $key, value);

set @copy = @();
foreach $j in @Range(0, $ListCount(@Vec)) {
    if ($i == $j) { set @copy = @ListInsert(@copy, value); }
    else { set @copy = @ListInsert(@copy, $($ListItem(@Vec, $j))); }
}
set @Vec = @copy;

...the latter of which is so tortuous I've probably got it wrong just typing it in here (and does not handle anything other than lists of scalars).

Looking at the existing similar %MapAdd and @ListInsert functions, I expect the meat of these is probably fairly straightforward:

ListSetVariableFunction

        protected override IEnumerable EvaluateVector(IVariableFunctionContext context)
        {
            var list = this.List.ToList();
            var index = this.Index;

            // bounds checking
            if (index >= list.Count)
            {
                // allow for growing the list to fit new index
                list.AddRange(Enumerable.Range(0, 1+index-list.Count).Select(_ => string.Empty));
            }
            else if (index < 0)
            {
                // allow for negative indexing from end of array (but not growth)
                if (-index >= list.Count) throw new ArgumentOutOfRangeException(nameof(this.Index));
                index = list.Count + (index % list.Count);
            }

            list[index] = this.Value;
            return list;
        }

MapSetVariableFunction

        public override RuntimeValue Evaluate(IVariableFunctionContext context)
        {
            if (String.IsNullOrEmpty(this.Key)) throw new ArgumentNullException(nameof(this.Key));

            var map = new Dictionary<string, RuntimeValue>(this.Map);
            map[this.Key] = this.Value;
            return new RuntimeValue(map);
        }

(Reposted from GitHub on request)