Jenkins and HTTP POST authentication difficulties

With a little tinkering of the Job Token we are able to use the Jenkins Extension to run a new Job. However the API is way more capable and the extension does not have those features, so I tried using the HTTP POST action to accommodate.

The 2 common thing we need to do:

Create a new Job

Update an existing Job

They both require a file or big textblock of the config.xml and they both require authentication. I am able to accomplish these tasks with an HTTP Chrome plugin (Postman), so I know the following code works when I apply a binary file containing the config.xml. The login/password are entered into plugin and it returns the basic authentication key, which I cleaned up below to just be "xyz123"

-New Job

curl -X POST -H "Content-Type: application/xml" -H "Authorization: Basic xyz123" "http://Jenkins:8080/job/createItem?name=TestNewAPIJob" --data-binary @config.xml

-Update Job

curl -X POST -H "Content-Type: application/xml" -H "Authorization: Basic xyz123" "http://Jenkins:8080/job/TestNewAPIJob/config.xml" --data-binary @config.xml

In BuildMaster I am having trouble and always receive a 403 authentication error. I dont know if there is an issue with the syntax for RequestHeaders and how I pass in the authentication, of if there is a way for BuildMaster to create the authentication string based on login settings.

I do have everything working right now using BuildMaster to create the config, and then kicking off the cURL request as a simple bat file, but it would be nice if I dont need to install cURL, and can just do everything with the application.

Thanks, -Dan

Product: BuildMaster
Version: 5.3.5

Perhaps it'd be easy to add these Operations to the Jenkins Extension?

https://github.com/Inedo/bmx-jenkins

What did you try in BuildMaster? Note that, basic authentication works by passing in a header like this:

Authorization: Basic QWxhZGRpbjpPcGVuU2VzYW1l

The string is a base64 encoded user:password.

I'm not sure if you tried that or not.

Here is the TextMode output, reokaceubg the large xml file that would be attached in TextData with just 'body'

Post-Http http://Jenkins:8080/job/TestNewAPIJob/config.xml
(
    ContentType: application/xml,
    TextData: body,
    LogRequestData: true,
    LogResponseBody: true,
    RequestHeaders: Authorization: Basic xyz123
);

This returns a 'bad value' error with stack trace, if I remove the request header completely its a 403 http error which I expected. I cannot seem to get the syntax to work for this request to pass in the authorization in a way that does not cause the 'bad value' error.

Stack (incase it helps)

Unhandled exception: System.Exception: bad value
at Inedo.BuildMaster.PlanExecuter.ScriptPropertyMapper.CoerceValue(RuntimeValue value, Type type)
at Inedo.BuildMaster.PlanExecuter.ScriptPropertyMapper.SetPropertyValue(Object target, IVariableEvaluationContext variableContext, String propertyValue, PropertyInfo property)
at Inedo.BuildMaster.PlanExecuter.ScriptPropertyMapper.SetProperties(Object target, ActionStatement action, IVariableEvaluationContext variableContext)
at Inedo.BuildMaster.Windows.ServiceApplication.Executions.PlanExecuter.DeploymentPlanExecuter.<Inedo-ExecutionEngine-Executer-IExecutionHostEnvironment-ExecuteActionAsync>d__21.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Inedo.ExecutionEngine.Executer.ExecuterThread.<ExecuteAsync>d__46.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Inedo.ExecutionEngine.Executer.ExecuterThread.<ExecuteNextAsync>d__34.MoveNext()

Cleaning up a few Typos:

'reokaceubg' should be 'replacing'

And the authorization I actually use is a base64 string, I have been anonymizing the URL and login credentials when I post details.

If you guys want to expand the use of the Jenkins extension, there might be issues regarding the login vs using the token. We could only make a connection to the job itself by setting the job up for Remote Access and saving off a token, and provide that exact token back. We could not get the connection to work using only login credentials or the user token provided by Jenkins. And if the task is to make a new job, there is no Job Token for us to key against. (This would be a separate ticket though)

-Dan

Ah, I see... RequestHeaders is a Map type, so you need to use the appropriate syntax for that... it probably isn't documented on the visual editor / help page like that though:

Post-Http http://Jenkins:8080/job/TestNewAPIJob/config.xml
(
    ContentType: application/xml,
    TextData: body,
    LogRequestData: true,
    LogResponseBody: true,
    RequestHeaders: %(Authorization: Basic xyz123, Header1: value)
);

Thanks, that got the connection working.

Although my issue now is that I cannot provide the XML body as text, or find a way to upload a file. Is either possible with this action type?

This is what is contained in a normal Jenkins Config.xml file:

<?xml version='1.0' encoding='UTF-8'?>
<project>
  <actions/>
  <description>test</description>
  <keepDependencies>false</keepDependencies>
  <properties/>
  <scm class="hudson.scm.NullSCM"/>
  <canRoam>true</canRoam>
  <disabled>false</disabled>
  <blockBuildWhenDownstreamBuilding>false</blockBuildWhenDownstreamBuilding>   
  <blockBuildWhenUpstreamBuilding>false</blockBuildWhenUpstreamBuilding>
  <triggers/>
  <concurrentBuild>false</concurrentBuild>
  <builders/>
  <publishers/>
  <buildWrappers/>
</project>

What happens when you supply it for TextData? It will just send that as the body...

I get a 500 error, it has to do with the content I provide not properly turning into the xml code needed (exact Jenkins error is probably java.io.IOException: Failed to persist config.xml, i cant see it though with BuildMaster because the log is cut off). I also have not yet found any way to to give the POST request a text block to properly convert into the config.xml, so I dont think this is an issue with BuildMaster.

Good news is that I realized there was an action called HTTP Upload, which seems to be exactly what I want, although even then it does not like the file, which I know is good because I am providing it the exact file that came from the jenkins server.

Upload-Http C:/config.xml
(
    Url: http://jenkins:8080/job/TestNewAPIJob/config.xml,
    LogResponseBody: true,
    RequestHeaders: %(Authorization: Basic abc123)
);

I am not sure why this could return the same error as before (a 500 server error), when I provide an xml file through my HTTP tools, I dont even have to give it the ContentType, so I tried adding that on my own and get a different error, heres that code:

Upload-Http C:/config.xml
(
    Url: http://jenkins:8080/job/TestNewAPIJob/config.xml,
    LogResponseBody: true,
    RequestHeaders: %(Authorization: Basic abc123, Content-Type: application/xml)
);

and the error:

Unhandled exception: System.InvalidOperationException: Misused header name. 
Make sure request headers are used with HttpRequestMessage, response headers with 
HttpResponseMessage, and content headers with HttpContent objects.
   at System.Net.Http.Headers.HttpHeaders.CheckHeaderName(String name)
   at System.Net.Http.Headers.HttpHeaders.Add(String name, String value)
   at Inedo.Extensions.Operations.HTTP.HttpOperationBase.CreateClient()
   at Inedo.Extensions.Operations.HTTP.HttpFileUploadOperation.<ExecuteAsync>d__4.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Inedo.BuildMaster.Windows.ServiceApplication.Executions.PlanExecuter.DeploymentPlanExecuter.
   <Inedo-ExecutionEngine-Executer-IExecutionHostEnvironment-ExecuteActionAsync>d__21.MoveNext()

I get the typical 500 response if Content-Type is replaced with ContentType (without the hyphen)

Oh you definitely can't set Content Type as a RequestHeader like that.

The Upload-Http action uses "form/multi-part" as a content type in order to "attach" a file to the body. This is how file "uploading" works by convention. This is why you can't set the property on the operation. If Jenkins does not accept/work with form/multi-part attachments of xml, then this won't work.

What does your Post-HTTP look like? I would start small, and see if you can get Jenkins to respond differently.

Post-Http http://Jenkins:8080/job/TestNewAPIJob/config.xml
(
    ContentType: application/xml,
    TextData: <project />,
    LogRequestData: true,
    LogResponseBody: true,
    RequestHeaders: %(Authorization: Basic xyz123)
);

Note that you will probably want to use swim strings for the XML.

set @xml = >><project>
  <actions/>
  <description>test</description>
  <keepDependencies>false</keepDependencies>
  <properties/>
  <scm class="hudson.scm.NullSCM"/>
  <canRoam>true</canRoam>
  <disabled>false</disabled>
  <blockBuildWhenDownstreamBuilding>false</blockBuildWhenDownstreamBuilding>   
  <blockBuildWhenUpstreamBuilding>false</blockBuildWhenUpstreamBuilding>
  <triggers/>
  <concurrentBuild>false</concurrentBuild>
  <builders/>
  <publishers/>
  <buildWrappers/>
</project>>>;

Finally got it working, turns out a single quote at the start and end of TextData is enough for the configs to be fully included without error, while still accepting BuildMaster parameters,

Here's the code in case there is a desire to implement Job creation in the Jenkins extension, (anonymized a little):

Post-Http http://Jenkins:8080/job/TestNewAPIJob/config.xml
(
    ContentType: application/xml,
    TextData: '<?xml version="1.0" encoding="UTF-8"?><project> </project>',
    LogRequestData: true,
    LogResponseBody: true,
    RequestHeaders: %(Authorization: Basic xyz123)
);

Thanks for the help getting this going, this was my first time using 5.3 from 4.7, little learning curve with the new Plan editor that is really cool!

-Dan