MinIO Support

[c4buildmasters_2588]

I've been looking into ProGet with my company and wanted to see if there was support for MinIO? I know there is AWS S3 storage support, but it looks like the only way is using virtual path support, whereas MinIO requires forcing path style.

[atripp]

Hi @c4buildmasters_2588 ,

I haven't heard of MinIO before, but they claim to be "S3 Compatible Storage" so maybe it'll work?

I also don't know what "virtual path support" or "path style" means, but I can say that several users have had success with other S3-compatible services, while other services have had bugs and/or aren't actually S3-compatible. Best I can say, give it a shot and see what happens.

You can see how we use the AWS SDK and what our code looks like over here:
https://github.com/Inedo/inedox-aws/blob/master/AWS/InedoExtension/FileSystems/S3FileSystem.cs

No idea what to enter in any of the fields, but I suppose you would just enter the CustomServiceUrl ("Advanced" tab).

If you get it working let us know, would be nice to have a follow-up post here in case anywone else searches for it :)

Cheers,
Alana

[c4buildmasters_2588]

As we are needing to run S3 on Premise, we elected to go with MinIO
So after testing, it looks like there are compatibility issues with the AWS SDK. The issue being the default connector using the AWS SDK is using what is called virtual style rather than path style. In this case, I believe a change would be needed to support path style URL's. eg. with aws, you could use either <bucket name>.s3.amazonaws.com or s3.amazonaws.com/bucket-name. For reference from AWS, here is their documentation on the difference between Virtual Hosted & Path Style Requests.
In the situation of how the SDK works by default, it is using virtual path style which would not work with the implementation style that we have used with MinIO.

From what I can tell, the modification to the AWS SDK should be relatively simple to implement as it is adding another option within the web interface, as well as adding a line in the AmazonS3Config class to "ForcePathStyle" when selected. If it would be easier, I can draft up a PR that can implement those changes, however my C# is a bit rusty.

I will say that by implementing this, it should support the rest of the S3 Compatible providers others have looked at.

[atripp]

Sounds like a plan, @c4buildmasters_2588 !

In case you haven't seen it, it's "relatively easy" to load a custom extension:
https://docs.inedo.com/docs/inedosdk/extending-inedo-tools-using-the-sdk/inedosdk-extending-creating

For testing this change, I would just build/pack it with a higher version number than the official AWS extension - if all looks good on our end (i.e. won't break existing installs, etc), then we'll likely be able to accept pull request and then publish an official new version. Then you can delete your custom one.

Thanks,
Alana

[c4buildmasters_2588]

Thank you so much for that information. I ended up putting in this PR with the changes. This change should allow for full support of most (if not all) other S3 Compatible services like Backblaze & MinIO. I did test this specifically with MinIO & AWS with no issues presenting themselves.

If there needs to be anything else done from my end to perform testing, I would be more than happy to do that to get this over the finish line.

[Dan_Woolf]

Hi @c4buildmasters_2588,

Thanks for the PR! We are currently reviewing your PR and we will be in touch.

Thanks,
Dan

[gdivis]

Just to let you know - the PR has been merged and we'll release AWS 3.1.2 on Friday with the fix included. Thank you!