Add support for Terraform Public Registry in ProGet (offline/air-gapped)

[Stholm]

Background / Use case:
We consume public Terraform providers in an air-gapped environment by importing them offline. We have a homebuilt solution that handles Terraform packages only.

Requested capability:
Add ProGet feed type (or feature) that mirrors the Terraform Public Registry protocol for providers.

Implementation notes (based on our research):
Minimal surface: a few JSON files for provider/package/version indices plus the binary payloads

References:
Happy to share our current homebuilt approach and example JSON/binaries for validation.

[atripp]

Hi @Stholm ,

I assume you saw the Terraform Modules documentation in ProGet?

While updating the Support for Terraform Backends to link to this discussion, I noticed we had some internal notes. So I'll transfer them here:

This would require implementing both the Provider Registry Protocol (for first-party plugins) and the Provider Network Mirror Protocol (for connectors). Both seem relatively simple, though there appear to be some complexities involving signature files.

In either case, we ought to not package these because they are quite large. For example, the hashcorp\aws provider for Windows is just a zip file with a single, 628mb .exe. They also have no metadata whatsoever that's returned from the API.

One option is just to store these as manifest-less packages. For example, hashicorp/aws packages could be pkg:tfprovider/hashicorp@5.75.0?os=windows&arch=amd64. This would be two purls in one feed, which might not work, so it might require a new feed.

Don't ask me what that all means, I'm just the copy/paster

But based on my read of that, it sounds like a big effort (i.e. a new feed type) to try to fit a round peg in a square hole. And honestly your homebuilt solution might work better.

I think we'd need to see how much of a demand there is in the offline/air-gapped Terraform userbase for this. But feel free to add more thoughts as you have them.

Thanks,
Alana