Proget Jenkins Plugin: Not working on Slaves?

Hi,

I am trying to run the Proget Download step on a Jenkins Windows slave. But while the console log indicates everything has worked fine, no package is downloaded and/or extracted on the slave. The target path and the workspace remain empty.

I see the following three lines in the console log for the job (actual values redacted):

[ProGet] Download package to <targetpath>
[ProGet] Sending GET to <packageurl>
[ProGet] Unpack <packagefile>

When running the same job on the Jenkins master, it works as expected.

If I run the upack command on the slave, it works fine. I have also tried running the upack command as a buildstep in Jenkins, and that works fine, too.

Please indicate if the plugin's Download Package step is designed to run on Jenkins Slaves, and what needs to be done to make it work.

Product: ProGet
Version: 4.4.1

I haven't tested it on a Jenkins slave as I assumed there would be no difference in behaviour. There is nothing that I can think of that should prevent it. I'll investigate this further and get back to you.

I've tried this on a slave running on the same PC as the Jenkins server and it ran just fine. Unfortunately I don't have another PC available today to try this on a separate slave.

What are the settings you have for:

Download format
Destination folder

Is the Jenkins job able to create other files in the destination folder? eg add build step to run windows command:
echo can do > try.txt

Hi Andrew,

thanks for looking into this. To confirm, my slave is separate from the master. I have tried to use the 'Package' and 'Unpack Content' download formats, neither of which worked. My Destination folder is a folder on the D drive (D:\Foldername). I've tested a local slave and that works fine as well, but after a bit more testing, that's deceptive.

Here's what I just noticed while testing a bit more: when the job containing the Proget Unpack step is bound to a Slave (using 'Restrict where this project can run'), the console log for the build indicates that it is running on the slave, however the actual step appears to be running on the master. So when I run the step to unpack the package into D:\Targetfolder, it will download and unpack the package to D:\Targetfolder on the master, not on the slave.

I've added a step to create a file on the slave using the File Operations build step, and that works exactly as expected, creating the folder and file in the correct location on the slave.

So it seems that the plugin doesn't properly bind to a remote slave when the containing job is bound to a slave, instead performing the Proget Unpack operations on the master.

Hope this helps.

Thanks for your reply - it gave me everything I needed to track down the issue. I wasn't aware that Jenkins plugins run on the master by default and need to be written a particular way to run on a execute on a node.

I've published version 0.4 and I'd be grateful if you could try it out and let me know how it goes.

Do you use the upload buildstep in Jenkins by any chance? I'd be interested to know if that suffers from the same issue. It shouldn't but I haven't been able to test it.

Hi Andrew,

thanks for continuing to look into this. I have used the Upload build step, but only on the master.

Unfortunately the Download step isn't working on the slave with version 0.4. It still runs fine when the job is bound to the master, but when running on the remote slave, I now get this console output:

FATAL: Unable to serialize com.inedo.proget.jenkins.DownloadPackageBuilder$1@16b05cf
java.io.IOException: Unable to serialize com.inedo.proget.jenkins.DownloadPackageBuilder$1@16b05cf
at hudson.remoting.UserRequest.serialize(UserRequest.java:201)
at hudson.remoting.UserRequest.<init>(UserRequest.java:64)
at hudson.remoting.Channel.call(Channel.java:779)
at com.inedo.proget.jenkins.DownloadPackageBuilder.perform(DownloadPackageBuilder.java:123)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:779)
at hudson.model.Build$BuildExecution.build(Build.java:205)
at hudson.model.Build$BuildExecution.doRun(Build.java:162)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:534)
at hudson.model.Run.execute(Run.java:1720)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:98)
at hudson.model.Executor.run(Executor.java:404)
Caused by: java.io.NotSerializableException: com.inedo.proget.jenkins.DownloadPackageBuilder
at java.io.ObjectOutputStream.writeObject0(Unknown Source)
at java.io.ObjectOutputStream.defaultWriteFields(Unknown Source)
at java.io.ObjectOutputStream.writeSerialData(Unknown Source)
at java.io.ObjectOutputStream.writeOrdinaryObject(Unknown Source)
at java.io.ObjectOutputStream.writeObject0(Unknown Source)
at java.io.ObjectOutputStream.writeObject(Unknown Source)
at hudson.remoting.UserRequest._serialize(UserRequest.java:190)
at hudson.remoting.UserRequest.serialize(UserRequest.java:199)
... 12 more
Finished: FAILURE

I'm unfortunately not familiar with developing Jenkins plugins myself, nor do I know much about Java. I found these two links on a quick search, perhaps they are of some help?

https://wiki.jenkins-ci.org/display/JENKINS/Making+your+plugin+behave+in+distributed+Jenkins#MakingyourpluginbehaveindistributedJenkins-PassingObjectsthatarenotSerializable

http://stackoverflow.com/questions/17727054/jenkins-plugin-cant-access-file-on-slave

Thanks for the feedback, looks like I'll have to get myself a proper test environment.

Hi Andrew,

I have the resources, so if there's anything I can do, I'd be happy to assist with testing this.

Best Regards

Alex

If you're comfortable with Java and Gradle then:

Source code is at: https://github.com/jenkinsci/inedo-proget-plugin

The offending bit of code is in DownloadPackageBuilder's perform method. I have it using the callable interface which should be all it needs to run on the slave.

Easiest way to test this would be to run "gradlew clean jpi" and take the generated hpi (or jpi) file from build\libs and place in eclipse pugings folder and restart eclipse.

Alternatively running "gradlew clean server" will start up a new instance of jenkins.

I had hoped to get to this over the weekend, but being a long weekend and finally having a bit of sunshine after weeks of crap weather I chose the sunshine (sorry). I will have another crack at this in the next couple of days.

Finally got it working! Sorry it's taken so long, it was very fiddly and I haven't had much spare time.

I've only got the download working, I'll have a look at the proget upload and Jenkins buildmaster plugins sometime this week to ensure that they work on slaves as well.

New Jenkins inedo plugin version is 0.5.

There is one issue that I have noticed: for some reason if you select the download format as 'Package' then it is not getting the correct filename on a slave. Works fine with 'Content as Zip' and 'Unpack Content' options. This should not cause any issues but I am looking into the cause.

Please let me know how you get on.

Regards
Andrew

Hi Andrew,

thanks for the update! I can confirm that download to node now works as expected (using unzip contents option).

Also I apologise, I didn't see your previous answer re testing, I didn't get a notification for it. Will take a look at how this works, I need to get more familiar with Jenkins plugins at any rate.

Keep up the good work!

fillermark_5157

Alex-Hempel said in Proget Jenkins Plugin: Not working on Slaves?:

Caused by: java.io.NotSerializableException: com.inedo.proget.jenkins.DownloadPackageBuilder

java.io.NotSerializableException can occur when you serialize an inner class instance because serializing such an inner class instance will result in serialization of its associated outer class instance as well. How to solve it?

If the class is yours, make the class serializable by implementing the interface java.io.Serializable.
All non primitive members implement Serializable (or are transient instead)
If your class is an inner class it's either static or the outer class implements Serializable
If the class is 3rd party, but you don't need it in the serialized form, mark the field as transient

It is important to note that Serialization of inner classes (i.e., nested classes that are not static member classes), including local and anonymous classes, is strongly discouraged.