Jenkins pipeline uploadProgetPackage fails on large uploads?

Guest

I'm having to trial-and-error, and have found this:

if I execute uploadProgetPackage with single file (xxx.exe), it all uploads fine.
If I execute with folder containing that EXE (binDir/), or (binDir/**/), it fails with
java.io.IOException: Remote call on Channel to /10.5.50.53 failed

Is there a limit to the package size being transferred from slave to master to ProGet server to ProGet DB?? Also, I cannot find any logs on the slave... where are the logs?? I've looked in the server (via web) and also do not see logs.

Environment:

Jenkins Server: 2.32.3 (on Win10), Inedo Plugin 0.7<br>
ProGet Server: 4.7.6 build5 running on same system as Jenkins Server
using ProGet provided MySQL DB
Jenkins Slave: Win10

example success


13:31:16 [x64dbg] [ProGet] Sending POST as user 'build-auto' to http://XXXMYSERVERXXX.com:81/upack/Experiment/upload
13:31:16 [x64dbg] [ProGet] With application/zip content:
13:31:16 [x64dbg] 	... Content of file C:\Baseline_x64dbg.unpack ...
13:31:16 [x64dbg] 

[Pipeline] [x64dbg] echo
13:31:19 [x64dbg] upload succeeded

example failure


13:39:25 [x64dbg] [ProGet] Sending POST as user 'build-auto' to http://XXXMYSERVERXXX.com:81/upack/Experiment/upload
[Pipeline] [x64dbg] error
[Pipeline] [x64dbg] }
[Pipeline] [x64dbg] // dir
[Pipeline] [x64dbg] }
[Pipeline] [x64dbg] // node
[Pipeline] [x64dbg] }
[Pipeline] [x64dbg] // timestamps
[Pipeline] [x64dbg] }
[x64dbg] Failed in branch x64dbg
[Pipeline] // parallel
[Pipeline] echo
>>>>>>>>> ERROR:hudson.AbortException: java.io.IOException: Remote call on Channel to /10.5.50.53 failed

I'm not sure where the IP address 10.5.50.53 is coming from - the system running both Jenkins and Proget servers is 10.1.1.180. I will be checking that with my IT department.
In case its important, I've surrounded the Proget uploadProgetPackage with try{...} catch () {...}

Meanwhile, can you direct me to any documentation on the plugin and the upload (examples, restrictions, etc)??
thank you!

Product: ProGet
Version: 4.7.6

Guest

Are you able to upload this same package via upack.exe or the browser?

There shouldn't be any upload limits...

here are some additional links:

https://github.com/jenkinsci/inedo-proget-plugin

https://wiki.jenkins-ci.org/display/JENKINS/Inedo+ProGet+Plugin

Guest

Apologies for the lengthy time-to-respond. I've been side-tracked, but back on this now.
Update:

I've been able to setup my own ProGet Server (4.7.13), with a valid 30 day license. I have it setup with one test feed. My Jenkins Server is 2.60.2, and the Indedo plugin is 0.7. I'm using pipeline scripts (groovy) only.
tests using Jenkins plugin and uploading a folder with logs (test files) of about 8-20Kb works.
When I then use same plugin, but upload folder of binaries (2.9Gb), it packs for about 2 min, and then I see
ProGet Upload error was: java.io.IOException: Remote call on Channel to /10.5.50.98 failed
I've found the API documentation and see PUT or POST /upack/«feed-name»/upload«...» but it appears (the CURL example) that I must first make a ZIP archive on my own, and then issue the curl PUT using the name of that ZIP archive file. (yes?) [and I will be experimenting today..]

Guest

More Data:

after some experimentation with the Jenkins Plugin, I've discovered that it is not a "size" issue (eg 2.x Gb), but rather that I seem to only be able to specify a SINGLE folder to pack using the Jenkins Plugin syntax:
dir(parent) { uploadProgetPackage artifacts: kidFolder+'/**/*', caseSensitive: false, excludes: '**/*.unpack', title: myTitle, description: descrip, feedName: feedID, groupName: myGroupName, packageName: myPackageName, version: '1.2.3', metadata: myMetaAry.join('\r\n') }
the above does not work. It fails -- I've lost the log but can re-create the error pretty easily..
I've setup another experiment using the Universal Package (make a zip called myTest.upack, with a upack.json inside) and issue a curl command mostly per the Inedo docs:
curl.exe" -X PUT "http://myProGetSrvr:81/upack/myUnivFeed/upload" --upload-file "C:\Jenkins\workspace\experiment\myTest.upack" --user "****"
this works (sortof).
I see the curl transfer stats, and there does not appear to be an error, but when I echo the STDout, I only see the command (above) and nothing else.
when I look on the ProGet server, I see no new packages anywhere.
Aha!! I looked at the Event Log, and see that the package was created at 12:47 today, and modified at 5:59 today (when I ran the test script one last time)!! and indeed, I see the correct contents when I download that package.

I think I'm better, but it sure would help to know what responses I get from a cURL -X PUT... and do I need to use PUT, or can I use POST(??)

Guest

even more data;

the curl -X PUT... works, but only for zipped files smaller then 1Gb

as I mentioned in earlier posts, the data is usually close to 2.x Gb, so using a web-ish upload (eg curl) is problematic on many levels.

Is there an equivalent to FTP for uploads/downloads to handle file transfers between the ProGet server and a client system?

Guest

Final results for curl -X PUT...

I was able to change C:\Program Files\ProGet\WebApp\Web.config (yes, my server is on Windows 10), and increase the transfer size from 1Gb to 3Gb
I then restarted the ProGet Web Service
I then tried to upload a zip file 2.02Gb in size..and, it worked.