Inquiry about buildmaster FTP plugin

nkerifacclaud_6931

Hello guys, is it possible to download folders (instead of files) using FTP plugin? If so how can this be done please?

rhessinger

Can you give me a little more detail in what you are trying to do? From what I can see you could specify your ServerPath as the folder you are looking to pull and include all files. Are you specifically looking for the include/exclude masks to work with folders? Or are you looking for something else entirely?

Thanks,
Rich

nkerifacclaud_6931

@rhessinger I am looking for something different from what you suggested. Consider for example we have the below folder structure:
Folder
Folder/Subfolder1/file1
Folder/Subfolder1/file2
Folder/Subfolder2/file3
Folder/Subfolder3/file4.

What I am looking for is to download Folder with its sub directories irrespective of their names and contents (I might not even know what is in it).

rhessinger

Hi @nkerifacclaud_6931,

I'm not sure I fully follow. Are you trying to download the entire contents of Folder? If so, use the Get-Files and set your ServerPath to /Folder and then Include to *. This should download the entire contents of the folder and its child folders and files. Is that what you are doing now?

Thanks,
Rich

nkerifacclaud_6931

Hello @rhessinger, when i try ServerPath = Folder and Include = * it does not download any files instead it download all subdirectories (with none of their contents) adding a '\' prefix to the name. Eventually we get something like below on the server running the script:
\Subfolder1
\Subfolder2
\Subfolder3

When I keep ServerPath=Folder and change Include to ** we still have the same outcome as above except that this time there is error in the log saying network name is no longer available.

The second method seems to be what i need but it is failing and i think it is due to the '\' being added infront of folder names created

rhessinger

Hi @nkerifacclaud_6931,

Good catch, I'm sorry for that, I actually did mean ** for the include property. The error is an interesting error though. I have seen this error commonly occur in two scenarios; 1. The file is deleted prior to the copy step (unlikely but possible) or 2. You have the permissions to list folder contents, but you do not have the permission to read the files. Can you verify that the account you are using for the FTP server has access to read the files on the FTP server? I would also check that your Get-Files operation is not happening until after the files have been put on the FTP server.

Thanks,
Rich

nkerifacclaud_6931

Hi @rhessinger,
Thanks for the feedback.
I can assure you that permission is not an issue because using the same credentials i can manually download same files/folders from the FTP server.
Like i mentioned earlier, I strongly think the issue is being caused by '\' being added infront of folder names. Please can you check why it is happening? Because I think it is not a normal.

Concerning the phrase "I would also check that your Get-Files operation is not happening until after the files have been put on the FTP server." i don't quite understand. Maybe you can explain again what you are expecting from my side.

rhessinger

Hi @nkerifacclaud_6931,

Thanks for confirming the permissions for me. As for adding the '' character in front of the folder names, I cannot see anywhere in our code that would do that. I do see that ServerPath will prepend a '/' character, but it is clear that the file list is working, so that would not be an issue.

I have been searching for the server returned 550 the specified network name is no longer available and I am seeing quite a lot of people saying this is actually an issue with Anti-Virus on servers or web protection in a Firewall or Network Proxy. This is an error coming directly from the FTP server. Do you have an anti-virus installed on the FTP server? If so, which one? Do you have BuildMaster configured to use a proxy? I see a lot of people complaining about this issue with AVG's resident shield.

@nkerifacclaud_6931 said in Inquiry about buildmaster FTP plugin:

Concerning the phrase "I would also check that your Get-Files operation is not happening until after the files have been put on the FTP server." i don't quite understand. Maybe you can explain again what you are expecting from my side.

Sorry about the confusion on this. I meant whatever tool is copying the files to the FTP server, just making sure the transfer has completed prior to trying to pull the files. Some FTP servers will list a file if a file is currently in transit, but not yet complete. Scenarios this can happen is with CI/CD tools (like BuildMaaster, TeamCity, Jenkins, Azure DevOps, etc...). When those tools are executing operations, scripts, pipelines, etc... in parallel, it can start the FTP file transfer and kick off a pull of these files at the same time. This is more to just rule that out as a possibility.

nkerifacclaud_6931

@rhessinger
The FTP server is running on windows 7 and the anti virus running should be BitDefender (to be confirmed). No proxy is configured on the buildmaster

rhessinger

Hi @nkerifacclaud_6931,

Could you try disabling BitDefender temporarily to rule that out? Also, this could simply be related to the fact that Windows 7 is no longer in support and their may be a security patch to the .Net Framework which is causing the error. Since Windows 7 no longer gets updates, it would not get any new patches for the .Net Framework or the FTP server. We leverage the classes built-in to the .Net Framework for FTP connections.

Thanks,
Rich