"Bad handshake" when listening for agent

[jimbobmcgee]

I have enabled the listening connection on my Otter install, and am trying to get a second server that has the agent installed to dial home to the Otter server.

The firewall is open, a self-sign certificate has been created on the Otter server, and its thumbprint has been configured in the Otter server's listener config. A public export of the self-signed cert has been installed in the Trusted Roots store on the server with the agent, and the windows dialog claims the certificate chain is therefore OK.

I have created a server object, with the server type of "pull", and pasted the secret key from the object into a Connections/Server node the InedoAgent.config file on the server. Also, Connections/@Enabled="true" in that file.

However, the server object in Otter is stuck in the Error state.

The Agent Listener Dashboard shows connections from the server with the agent, every 30s or so. The Diagnostics Centre shows errors in a matching timeframe, with:

Bad handshake from SERVERWITHAGENTIP:52768: System.Security.Authentication.AuthenticationException: Authentication failed, see inner exception. ---> System.ComponentModel.Win32Exception (0x8009030D): The credentials supplied to the package were not recognized at System.Net.SSPIWrapper.AcquireCredentialsHandle(ISSPIInterface secModule, String package, CredentialUse intent, SCHANNEL_CRED* scc) at System.Net.Security.SslStreamPal.AcquireCredentialsHandle(CredentialUse credUsage, SCHANNEL_CRED* secureCredential) at System.Net.Security.SslStreamPal.AcquireCredentialsHandleSchannelCred(SslStreamCertificateContext certificateContext, SslProtocols protocols, EncryptionPolicy policy, Boolean isServer) at System.Net.Security.SslStreamPal.AcquireCredentialsHandle(SslStreamCertificateContext certificateContext, SslProtocols protocols, EncryptionPolicy policy, Boolean isServer) --- End of inner exception stack trace --- at System.Net.Security.SslStreamPal.AcquireCredentialsHandle(SslStreamCertificateContext certificateContext, SslProtocols protocols, EncryptionPolicy policy, Boolean isServer) at System.Net.Security.SecureChannel.AcquireServerCredentials(Byte[]& thumbPrint) at System.Net.Security.SecureChannel.GenerateToken(ReadOnlySpan`1 inputBuffer, Byte[]& output) at System.Net.Security.SecureChannel.NextMessage(ReadOnlySpan`1 incomingBuffer) at System.Net.Security.SslStream.ProcessBlob(Int32 frameSize) at System.Net.Security.SslStream.ReceiveBlobAsync[TIOAdapter](TIOAdapter adapter) at System.Net.Security.SslStream.ForceAuthenticationAsync[TIOAdapter](TIOAdapter adapter, Boolean receiveFirst, Byte[] reAuthenticationData, Boolean isApm) at Inedo.Agents.Connections.PullServerConnection.ReceiveHandshakeAsync(CancellationToken cancellationToken) at Inedo.Agents.AgentListener`1.ProcessIncomingConnection(TConnection channel)

If I set LogFile in the InedoAgent.config file, I see repeated entries for:

07/06/2023 06:13:44 DEBUG: Attempting to establish connection with OTTERSERVER:46336...
07/06/2023 06:14:14 DEBUG: Attempting to establish connection with OTTERSERVER:46336...

DNS and firewall look fine:

> Test-NetConnection OTTERSERVER -Port 46336

ComputerName     : OTTERSERVER
RemoteAddress    : OTTERSERVERIP
RemotePort       : 46336
InterfaceAlias   : Ethernet0
SourceAddress    : SERVERWITHAGENTIP
TcpTestSucceeded : True

I can add the standard .NET trace listeners to the InedoAgentService.exe.config, but I'm not sure what I'm looking for in the massive infodump the resulting trace file then contains.

What am I missing?

[atripp]

Hi @jimbobmcgee ,

The "Bad handshake" error is occurring (as you probably guessed) while trying to establish SSL tunnel. This basically occurs at the operating system level, and is usually a total mystery. The underlying error seems to be coming from SSPIWrapper.AcquireCredentialsHandle, and is just this:

(0x8009030D): The credentials supplied to the package were not recognized

I have absolutely no idea what that means, but... there's a ton of things I found via searching. A few articles are suggesting that it means lack of access to read the private key

Maybe it's in the wrong folder (on the Otter server)? I think it needs to be in "Personal" folder of "Machine" certificates, but it sounds like it's definitely something wrong with Otter reading the certificate....

Let us now if you find out more!

Hope that helps,

Alana