Posts made by Jonathan.Engstrom

Yes, something like that. The master/main server additionally would be set to listen to collector servers. The collector/mini/secondary server would communicate with the host servers like Otter normally does. The exception is that because the clients are unaware of the master/main server, they only send the data to the collector. The collector then sends the data back to the master server. Really, the only difference is adding the ability to one Otter Server to send server configuration data upstream so one can see all of the otter farm (Otter pond? :) )

I would be more than happy to help! Let me know what and I can start coming up with something.

So do I set up Otter to the same location as the original raft? Should I have any concerns or gotchas I need to worry about? Thanks!

Hello Alana, thanks for the update. I understand the double hop issue with credssp and the challenges using Kerberos. I was thinking of a remote machine that sends it's data to a collector, or a mini Otter server of sorts. This mini server would then send the collected data to the master (mega? :) ) server witch adds the results to it's own. So the client would never attempt to communicate with the master server, only the collector/mini server thus bypassing any double hop complexity. Almost the same principal of a secondary server if you are familiar with Microsoft SMS/ System Center Configuration Manager. Does this make sense?

Hello, I am writing to see how this is progressing. Are there any rough ETA's available? Thanks!

So, any thoughts on a PowerShell Module for Otter? I would like to do something like:

Get-OtterServer | Where-object {$_.variables.vcpu -eq 4}

where I have a variable for servers called vcpu. I want to be able to just ask otter for information quickly. If the API is as quick or quicker, or easier, then what would be a good example of this? I am being asked to provide data to managers, and I need to be able to provide them that. Thanks!

So are there any other thoughts with this? Does this sound doable at all? Thanks!

I have two Otter servers in different domains. As of right now, they cannot communicate with each other. Can I share a Raft in Otter, or can I make it so they are copies of each other? Looking to avoid copying all of the code manually between each instance. I am using BitBucket Git as code repository for the raft. Thanks!

Product: Otter
Version: 2.1.3

Yes, that is somewhat what I was thinking, except for I am not using agents; I am wanting to do this with PowerShell (WinRM).

I have Otter installed in/on one domain, and would like to run a "relay service" on a set of servers that already has access to multiple tenants, and relay the communication from the Otter Server, to the Relay, to the client. Data from Client will start from the client, travel to the relay, then be communicated to the Otter server.

Keep in mind my problem is related to firewall issues and politics, hence the use of WinRM and not agents, thanks!

Hello, I have a question for a possible feature request. I am trying to avoid having different installation instances of Otter in separate domains. I would like to run the same code for each servers, but each tenant has their own set of "unique" (/sarcasm) requirements that are prohibiting communications to the existing Otter server. Would it be possible to create a minimal "collection" node that runs as a relayin other environments (AD domains/ forests) to run jobs, collections, executions and reports the data back to the Otter Server? Does this make sense?

Product: Otter
Version: 2.1.3

As I had stated before, I have copied the text of the server name itself from a failed imported server, and created one using the exact same name pasted from the clipboard, and then it would start working. I have not seen any formatting issues, and it appears to be correct. It may be incorrect though, it is a possibility. I have a large project I have been trying to get to work in Otter, and unfortunately this importing issue, and the fact there is no way to work with servers, environments, or roles by addressing them on a command line language like PowerShell. My options are:

-Trying to change the servers by import/export server data to change them. Since my imports have not been rock solid, and have caused me many issues as of late, I have concerns of late with that method.

-Try to access the API to manipulate that.(?) Yeah, since I can't call individual servers, I am working with a glob of data and alot of "noise/clutter". Maybe someone can share with me a good way to do this in PowerShell?

-Add 1000+ servers into Otter using the web interface. Well......, there's that. PowerShell Module please? :D

It would help if there was more verbosity to the logging and/or give me more indications of what the issues are. I am not able to glean anything from the "Hostname not found" error when I can clearly find it without issue. Again, lack of data makes troubleshooting more difficult.

As far as my configuration, it is almost 10,000 lines long, and I have a long way to go yet. I am not sure what is easy or quick about picking 10 servers or so out of that output and changing things.

So after adding about 200 servers in our environment already, I added the other 1000 or so but adding them to their own separate environments and roles that are empty, as well as having basic configurations. They are doing ReportOnly, default raft, and using powershell. Now the screen pulls up an IIS error page when I try to list the servers out. My question is how do I get these removed via import/export or API? I wish I would of saved a copy of the config before I imported it. This is another good reason why it would be nice to have powershell access and not rely on an web/db combo to work with it. Server resource utilization is still pretty low. Help would be appreciated, thanks!

Server Error in '/' Application.
The wait operation timed out
Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.

Exception Details: System.ComponentModel.Win32Exception: The wait operation timed out

Source Error:

An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below.

Stack Trace:

[Win32Exception (0x80004005): The wait operation timed out]

[SqlException (0x80131904): Execution Timeout Expired. The timeout period elapsed prior to completion of the operation or the server is not responding.]
System.Data.SqlClient.<>c.<ExecuteDbDataReaderAsync>b__180_0(Task1 result) +1219418 System.Threading.Tasks.ContinuationResultTaskFromResultTask2.InnerInvoke() +101
System.Threading.Tasks.Task.Execute() +70
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() +31
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) +60
Inedo.Data.<CreateAsync>d__10.MoveNext() +343
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() +31
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) +60
Inedo.Data.<ExecuteInternalAsync>d__33.MoveNext() +1237
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() +31
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) +60
Inedo.Data.<ExecuteInternalAsync>d__7.MoveNext() +364
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() +31
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) +60
Inedo.Data.<ExecuteTableAsync>d__32`1.MoveNext() +373
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() +31
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) +60
Inedo.Otter.WebApplication.Pages.<CreateChildControlsAsync>d__2.MoveNext() +377
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() +31
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) +60
Inedo.Otter.WebApplication.Pages.<InitializeAsync>d__1.MoveNext() +350
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() +31
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) +60
Inedo.Web.PageFree.<ProcessRequestAsync>d__46.MoveNext() +279
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() +31
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) +60
System.Web.TaskAsyncHelper.EndTask(IAsyncResult ar) +64
System.Web.CallHandlerExecutionStep.InvokeEndHandler(IAsyncResult ar) +212
System.Web.CallHandlerExecutionStep.OnAsyncHandlerCompletion(IAsyncResult ar) +166

Version Information: Microsoft .NET Framework Version:4.0.30319; ASP.NET Version:4.7.3163.0

Product: Otter
Version: 2.1.2

OK, that is what I was figuring. I am glad this is being worked on, as it is a "Must Have" for automation and copying files to servers. I wish I could use ProGet for something like this, but I can't install agents on the servers at this time. We have to control how access is configured for specific items anyway, and can't change the Agent Service Account on the fly for that.

So, I am looking again at the API, all the while wishing there was an Otter PowerShell Module to manage servers, roles, environments, all the good stuff..... :D

So for the roles, how do I add a role as a dependency to another role? I looked in the json server import/export, and I couldn't even tell where roles list out dependencies for other roles are?

I am doing this as I have not had good luck doing the Orchestration in Otter, and most of my jobs have failed. Plus, I do not want to modify the working code I have in a role and start supporting two different collections of code that do the same thing. So instead, I am adding the servers that I want to change with an empty nesting role that is set to auto remediate. I want to schedule the job in orchestrator to add the role that actually gets the work done at the start of our change window. Maybe Servers should have a change window added to them so they can be auto remediate during the change window, and go back to ReportOnly when they are outside the change window. This way, I don't have to write a psexec script that is just one more thing to manage and troubleshoot. Plus the variables don't seem to work in Orchestrator Jobs, so I end up hard coding all of the variables anyway. Does this make sense?

"Dear Santa, I have been a good engineer and almost always close out my change tasks on time. For Christmas, I would love a shiny new PowerShell Module for Otter, a New York Style Cheesecake, and the bestest Chocolate Chip Cookies"

Also, for anyone having performance issues in Otter, upgrade to 2.12 as soon as possible. Many of my performance issues were improved dramatically, as well has getting an executions tab for each server! Thanks Inedo support people for their great work!

Product: Otter
Version: 2.1.2

Here are the logs from the job history. I am assuming it's not working as it isn't using the permissions assigned to the server, and there is no where to configure what permissions to use. CredSSP is disabled in our environment for security concerns. My account I wish to use is already stored as a credential, so I wish to be able to leverage that to execute jobs.

I already know I have access to the file share, and can copy it without issue.

DEBUG: 2018-12-17 14:14:24Z - Job 216 directly targets 1 server(s) by name.
DEBUG: 2018-12-17 14:14:24Z - Skipping collection run; execute only.
INFO : 2018-12-17 14:14:24Z - Collection run succeeded.
DEBUG: 2018-12-17 14:14:24Z - Beginning execution run...
INFO : 2018-12-17 14:14:31Z - Copying files from \fileserver.domain.local\share\scripts to C:\Temp2...
WARN : 2018-12-17 14:14:31Z - Source directory \fileserver.domain.local\share\scripts does not exist.
DEBUG: 2018-12-17 14:14:31Z - Cleaning up temporary files on Inedo Agent (v??, TESTSERVER:46336)...
DEBUG: 2018-12-17 14:14:31Z - Execution run complete.
WARN : 2018-12-17 14:14:31Z - Execution run succeeded with warnings.

John, while on the subject of execution ID's and such, during the logging of a psensure, can we add the psensure key to the log so it always outputs what it is checking and validating? How it is now:

INFO : 2018-12-11 20:31:07Z - Configuration drift detected.
DEBUG: 2018-12-11 20:31:07Z - Adding to execution plan.
DEBUG: 2018-12-11 20:31:11Z - Collecting configuration...
DEBUG: 2018-12-11 20:31:11Z - Comparing configuration...
INFO : 2018-12-11 20:31:11Z - Configuration matches template.
DEBUG: 2018-12-11 20:31:14Z - Collecting configuration...
DEBUG: 2018-12-11 20:31:14Z - Comparing configuration...
INFO : 2018-12-11 20:31:14Z - Configuration matches template.

and so on. Maybe it could log something like "Collecting configuration <key>" and "<key> matches template". That way it would make each line meaningful and bring more value into keeping the logs. Eventually I would like to have a way to save a copy of the txt execution into servicenow so we have a record of it in change management system as well. The url you shared previously will help in doing that I think. This improvement would help as documentation for each change. Thanks!

Here is an example I tried doing UNC first, then tried both local paths, and neither worked:

##AH:UseTextMode

Copy-Files
(
Include: **,
From: C:\Temp\Agent\Extensions,
To: C:\Temp2,
Verbose: true,
Overwrite: true
);

So I have noticed that there are 'empty' executions id's that are showing an error. When I use your url provided, after putting in some of the execution id's they show an empty page. If I populate it with another execution ID that has logs, it displays that normally, so I know I am doing it right. Ideas?

John, thank you for that; that is very helpful. Is there a way to look up all of the executions that have happened for a server? I tried replacing the ExecutionID with a serverID, hoping it would list out all of the executions that have occurred for that server, but that didn't work :)

It would be awesome to have a tab on each server that listed out a complete history of all of the executions.

I have several examples; none of them are working. I would like to copy something from a UNC path (NAS Device) to copy to a location on one of the server's in Otter (say the C Drive under C:\Temp for example). Something like a file or a folder structure. Can you share an example of how this works in your environment?

Is anyone else able to shed any insight as to why thins is doing this, or share with me an alternative method to finding these logs? I need to troubleshoot this issue, thanks!

George, thanks for the reply. My point was being this page prevents one from finding out what issues are going on as it displays no relevant information. I am not sure where else I am going to find this information without digging in the SQL database, which I really don't want to do/ have time to do.

Yes, I am on 2.1.1. I usually upgrade within 48 hours of new release and a backup of previous DB.

I am having issues troubleshooting executions of servers in Otter with this screen. It has 3 buttons that don't do anything. Why can't the information just be displayed? The "Set Execution Status" button does not do anything, and the other two buttons "View Log Text" and "Download Full Log" only open up another tab in Chrome to the same screen. This does nothing to assist with troubleshooting. Where or how can I troubleshoot these systems? Thanks!

Product: Otter
Version: 2.1.1

Ticket is submitted along with additional information

Hello, I have been using multiple roles that are gathered to one role that I assign to each server as needed. I have two roles that are assigned to each server, one with about 30 roles and another with 15 roles. I do not want to assign these roles to each server individually. Over time, the performance has been getting worse. I have started performance troubleshooting, and have not found any issues (Checked usual Captain Obvious stuff), so I engaged a member of our SQL Support staff. After running a SQL trace, we saw this being executed that was taking between 3-12 seconds a piece. It is called "exec ServerRoles_GetServerRoles " in SQL Server Profiler. Is there a way to optimize this so it does not use excessive CPU? Thanks!

Product: Otter
Version: 2.1.1

Example it shows for the Execution Dispatcher times for each server. Watching one will show 1:27, 1:28,1:29, and then 2:30. Not a big deal, but a bug nonetheless.

Product: Otter
Version: 2.1.1

I get a "Service cannot be started. System.Data.SqlClient.SqlException (0x80131904): Execution Timeout Expired. The timeout period elapsed prior to completion of the operation or the server is not responding. ---> " in my event log. I have restarted the machine after trying to restart services, no change yet. SQL is working as other DB's for other apps are working fine. Ideas?

Product: Otter
Version: 2.1.1

I did not have any issues with Otter when switching from a trial to the free version.

So I am trying to figure out the $PSCredential property. I am assuming it's not as simple as just putting a password in plain text for everyone to see? That would be a total no-go. I am assuming that it is pulling the password that is stored from the user specified in "Resource Credentials"?

If not, is there a way to pull those credentials from "Resource Credentials"? I see in each user, there is a "$GetCredentialProperty() function". I tried searching for an example of that without much luck. I Found this https://inedo.com/support/questions/5094 but doesn't shed alot of light except reinforce my belief of a password in plain text. Can anyone provide an example of how that works? Thanks!

Product: Otter
Version: 2.1.0

Hello, I am trying to copy files from a UNC path (I have access to) to a group of servers. I am copying the latest windows security quality monthly rollup patch before we can install the SCCM client. This has to be done per our security policy. I have tried copy file and transfer file, neither seem to do anything. I guess I could do a psensure and do a get-hotfix to see if they are installed and see if I can copy and install from there? I am doing this with Powershell Agentless, and the creditials used are any admin on the server and full control on the nas folder.

Product: Otter
Version: 2.1.0

Alright, after messing around with that information, I was able to get it to work. I am able to update the variables. Interesting thing is that it doesn't appear I can change just one variable, I have to change all of them, or at least enter all of them as they are?

Thanks for your help!

Thanks John. I did try a few variations, and after I ended up renaming a server trying to get your method to work, I decided to stop trying. I am not very familiar with the format you are using, and it is different from the documentation. However, I agree that not adding the key to the URL string is probably a safer way to do it, and that process and examples should be added to the documentation.

I still am not able to update a variable, but if one creates a new server, then you can add variables as long as it's a create. Maybe that will help whoever is going to fix it.

I am not familiar with this way to do it, and it does not appear to work for me:

Invoke-RestMethod : The underlying connection was closed: An unexpected error occurred on a receive.

This appears to be an issue when using headers, but I have not found a reliable way to fix this.

So I am able to update all of the attributes of a server, with the exception of variables. I am able to edit roles, environments, temppath, drift, all without issue. I am not able to change server variables, though. Can anyone test this to confirm it may be a bug, or perhaps I am doing something wrong? Being I can update everything else, I am going to side with the former, but anything's possible.

$Key= 'blahblah'

$Server = 'hdarsintsv1'

$URI = "https://otterserver.test:8626/api/infrastructure/servers/update/$Server`?key=$key"

$Body =

'{

 "variables": 
 
 {
     "Viking": "True",
     
 },

}'

Invoke-RestMethod $URI -Method Post -Body $Body -Verbose

Forgot to add this too:

$URI = "https://azlmidrange.azlifem.azl.pvt:8626/api/infrastructure/servers/update?key=$key"

My bad!

So I have been working on different ways to update a server, and I am having some issues. I can create a new server, or add a new server to a role no problem with the code below. But Updating variables, I am getting this:

Invoke-RestMethod : 547160Variables_CreateOrUpdateVariable57`The INSERT statement conflicted with the FOREIGN KEY constraint "FK__Variables__Servers".

$Key= 'blahblah'
$Server = 'hdarsintsv1'

$Body =
'{
"variables": {
"Viking": "True",
},
}'

Invoke-RestMethod $URI -Method Post -Body $Body -Verbose

Any ideas? Thanks!

Is it possible to update a variable for a server based on code? For example, can I create a variable called OS and have it populate based on CIM_OperatingSystem or WMI? More specifically edit a variable on a server?

Product: Otter
Version: 2.0.13

I have a PSDsc configuration that I am having a problem with:

##AH:UseTextMode

PSDsc Registry
(
Otter_ConfigurationKey: Key,

Key : "HKLM:\SOFTWARE\Policies\Microsoft\Windows\CurrentVersion\Internet Settings\Zones\1",

ValueName : "1601",

ValueType : 'Dword',

ValueData : "1"

);

PSDsc Registry
(
Otter_ConfigurationKey: Key,

Key : "HKLM:\SOFTWARE\Policies\Microsoft\Windows\CurrentVersion\Internet Settings\Zones\1",

ValueName : "2100",

ValueType : 'Dword',

ValueData : "3"

);

PSDsc Registry
(
Otter_ConfigurationKey: Key,

Key : "HKLM:\SOFTWARE\Policies\Microsoft\Windows\CurrentVersion\Internet Settings\Zones\3",

ValueName : "1601",

ValueType : 'Dword',

ValueData : "1"

);

PSDsc Registry
(
Otter_ConfigurationKey: Key,

Key : "HKLM:\SOFTWARE\Policies\Microsoft\Windows\CurrentVersion\Internet Settings\Zones\3",

ValueName : "2100",

ValueType : 'Dword',

ValueData : "3"

);

This setting ensures IE harding of the security zones, but the trouble is two of they keys are the same, and two of the valuename or valuedata are the same as well. I have tried several combinations of trying to get the key to be more dynamic, but I can't seem to be able to work with Otter_ConfigurationKey: at all:

-Tried custom name, that doesn't work, must match a dsc property.
-tried an array like @('key','valuedata') as well as (@('key','valuedata') ), no dice there. Tried adding " " and ' ' just for kicks, no luck there.

While I love guessing, I like knowing much better. Anyone have any ideas to get this to work?

Product: Otter
Version: 2.0.12

(0x80131904): Transaction (Process ID 72) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.

Seem to be getting more of this the more servers I add. I have about 40 servers added atm. Anyone know anything about this?

Product: Otter
Version: 2.0.12

Dean, thanks for the reply. I am hoping it makes it into this release, as each server I import I have to go into the properties of each one manually and change the raft from not configured to default. While I am trying to build traction for this product, this process of touching each machine does not give a good impression. Thanks for the update!

Dean, that's great news! Thanks!

Hello, after trying for a half hour, I was not able to get the $PSEval to successfully do any math operations for me, but something I hadn't tried in my hours of troubleshooting this is try putting the variable last and the number first, and the PSEnsure worked great at that point.

OK, so I have a simple few lines of code I want to run a server checkout on to validate memory is correct. $MEM is a server variable, and the rest is just math. Anyone else any luck doing multiplication in Otter?

I know from doing a write-output of my variables that it is not doing the low or high variables:

    $MEM = 16
    $Win32_ComputerSystem = Get-WmiObject -Class Win32_ComputerSystem
    $ServerMEM = ($Win32_ComputerSystem.TotalPhysicalMemory / 1GB)
    $Low = $MEM * 0.95
    $High = $MEM * 1.05
    (($ServerMEM -ge $low) -and ($ServerMEM -le $High))

Product: Otter
Version: 2.0.10

Also, I wanted to add that I tried making an entry called "Raft" and that didn't work.

"Raft": "Default",

That doesn't do anything. Any reason why this doesn't work?

I am having problems sending emails from otter. I have configured the Smtp.FromAddress, Smtp.FromName, Smtp.Host in advanced settings. I can't make this process work at all. I can send emails using powershell without issue, so it's not a firewall or email server issue; those all work fine. If I put code to send an email in a PSEnsure code block, that works fine as well, but that is definitely less than ideal. I can't get any logs or output to help my investigation, so I really don't have anything to go off of. Thanks!

Product: Otter
Version: 2.0.10

Thanks for the reply. When I am looking at a server that I have made just for exporting, it does not contain an entry for the raft, which I thought was odd:

{
      "active": true,
      "environments": [],
      "roles": [],
      "serverType": "powershell",
      "credentialsName": "USERNAME",
      "drift": "reportOnly",
      "tempPath": "C:\\Temp",
      "wsManUrl": "",
      "name": "Template",
      "variables": {
        "DiskLabel": "%(C: OSDisk, D: DataDisk1, E: DataDisk2, F: DataDisk3, G: DataDisk4, H: DataDisk5, I: DataDisk6)"
      }
    },

I already had a raft configured on this template machine. but it does not show on the export. This doesn't make any sense to me why the raft would not show up as well on the configuration, IMHO. Is this a bug?

Hello, looking for a way without touching all of the servers I am importing to configure what raft to use. It does not appear this configuration is specified in either of the import. So if I am to understand this correctly, I import 100 servers and I have to 'touch' each one to configure a raft before I can do anything with them? I am sure there is a way to do this; I just am not aware of how.

Thanks!

Product: Otter
Version: 2.0.10

Hello,

I am using a PSensure to collect the information. I have a map expression configured on the server variable as to to check for disk labels:
%DiskLabel %(C: OSDisk, D: DataDisk1)

And this runs in a role to collect for each server it is assigned to:
##AH:UseTextMode

## Ensure Disk Label configuration
PSEnsure
(
    Key: Disk Label Check,
    Value: True,
    Collect: >>
        $ServerVariable = $DiskLabel
        $Disks = Get-WmiObject -Class Win32_LogicalDisk -Filter "DriveType=3"
        $ServerVariable.GetEnumerator() |ForEach-Object {  if ( ($Disks.DeviceID).Replace(':','') -contains ("$($_.Key)")) { $y = $disks | Where-Object -Property DeviceID -match $_.Key; $y.VolumeName -eq $($_.Value) }}
    >>,
    Configure: >>
        write-output "The fix is in"
    >>
);

But when it collects, even though the values are both true, it reports as drift. Ideas?

Product: Otter
Version: 2.0.10

Thank you, that did the trick. I was using the copy path from the Ribbon in Explorer, and it already adds quotes to it. I saw in the db query that it added a ".ps1" to the end of the file name, thus creating a hiccup.

I also noticed it had the path of where I stored the PoSh script. What is the value in having the path where I uploaded it contained there? Does it truly not add the script as an object and relies on it's path in the OS where it is uploaded from? Surely this can't be true.

Any videos or recommendations for getting started with DevOps using Inedo products? We are all infrastructure people who are learning code, and are not familiar with code repositories, etc. We want to continue using code to configure our infrastructure, but would like something that all works together. Thanks!