Add support for Terraform Public Registry in ProGet (offline/air-gapped)

[Stholm]

Background / Use case:
We consume public Terraform providers in an air-gapped environment by importing them offline. We have a homebuilt solution that handles Terraform packages only.

Requested capability:
Add ProGet feed type (or feature) that mirrors the Terraform Public Registry protocol for providers.

Implementation notes (based on our research):
Minimal surface: a few JSON files for provider/package/version indices plus the binary payloads

References:
Happy to share our current homebuilt approach and example JSON/binaries for validation.

[atripp]

Hi @Stholm ,

I assume you saw the Terraform Modules documentation in ProGet?

While updating the Support for Terraform Backends to link to this discussion, I noticed we had some internal notes. So I'll transfer them here:

This would require implementing both the Provider Registry Protocol (for first-party plugins) and the Provider Network Mirror Protocol (for connectors). Both seem relatively simple, though there appear to be some complexities involving signature files.

In either case, we ought to not package these because they are quite large. For example, the hashcorp\aws provider for Windows is just a zip file with a single, 628mb .exe. They also have no metadata whatsoever that's returned from the API.

One option is just to store these as manifest-less packages. For example, hashicorp/aws packages could be pkg:tfprovider/hashicorp@5.75.0?os=windows&arch=amd64. This would be two purls in one feed, which might not work, so it might require a new feed.

Don't ask me what that all means, I'm just the copy/paster

But based on my read of that, it sounds like a big effort (i.e. a new feed type) to try to fit a round peg in a square hole. And honestly your homebuilt solution might work better.

I think we'd need to see how much of a demand there is in the offline/air-gapped Terraform userbase for this. But feel free to add more thoughts as you have them.

Thanks,
Alana

[mikael]

I’d like to add some practical experience to this discussion.

We are currently using Terraform providers extensively, and have done so for several years, against self-hosted, offline installations of HashiCorp Vault, Grafana, our Git platform, ArgoCD, and Artifactory (which we are in the process of replacing, hence our interest in ProGet). None of these environments require outbound internet access at runtime.

Based on that, I don’t fully agree with the documentation saying "In our research, users [...]. They also saw no value in proxying these plugins from Hashicorp's public registry, as internet access was inherently required to use Terraform." In our case, Terraform is very much an offline IaC tool once providers are available. Providers are fetched through proxy, and then used entirely against on-prem/self-hosted services.

For organizations operating in restricted or air-gapped environments, proxying or mirroring the public Terraform Registry is about ensuring controlled, repeatable, and compliant access to upstream providers. This enables a proven, production-grade Terraform workflow, using standard, well-maintained providers, while still meeting strict network and security requirements that are common in enterprise and regulated environments.

[atripp]

Hi @mikael ,

Thanks for the additional insight.

The use case you describe (offline/air-gapped usage of Terraform) does seem rather niche, and is different than the traditional "proxying" use case we describe. Very few organizations will restrict internet access like that, and Proxying is more about controlling versions and limiting what developers are able to use.

Anyway, it may not be a good fit for investment on our end. But we'll see if anyone else joins this thread :)

That said, a Provider is basically just a executable file in a zip file. I wonder if you could simply use Asset directories somehow.

Thanks,
Alana

[davidroberts63]

We are considering the usage of Terraform internally. It hasn't happened yet as we are mostly in one specific hosting solution, and a few other internal reasons. That being said, this is a very pertinent discussion as we would be using ProGet to proxy all Terraform items, such as modules and providers. I echo what @mikael points out about proxying enabling a "controlled, repeatable, and compliant ... upstream". Its a primary reason we use products such as ProGet. While I would agree many other organizations would allow general internet access, those in governmental, medical and or financial industries would be far more likely to utilize proxying capabilities for those reasons.

Our servers that would run the Terraform plans would inherently be cutoff from general internet access. Especially cut off from github.com and similar domains given the broad nature of what is available within the github.com domain. Meaning a Terraform provider that required our servers to reach out to github would be extremely difficult to get past security's approval and would be far more welcome if made available in an internally controlled source, such as ProGet.

[atripp]

Thanks @davidroberts63

I looked into this a little more, so just as an FYI...

• Terraform Modules are relatively simple templates for code/configuration that you write and maintain (think of it like a Helm chart)
• Terraform Providers are executable files invoked by Modules, and many of them simply embed and wrap a CLI tool; the most popular providers are AWS, Azure, GCP, but there are many niche ones by other vendors

No one really writes their own Terraform Providers, and it's highly unlikely you would ever do that. They should be thought of as SDK/CLIs you downloaded from a marketplace... except they run as admin/root and have your most "sacred" credentials.

So just to clarify... you can currently host Terraform Modules in ProGet and you can also most certainly host Terraform Providers in ProGet (just using an Asset Directory). The question comes down to user experience and convenience.

The issue here is that Terraform Providers do not fit into ProGet's "package" or "connector" mindset; so it's not just a new feed type, but creating a whole new feature in ProGet that won't really work like other feeds.

We were able to "force" Terraform Modules into Packages/Feeds... but as you can see, it requires a lot of hoops to jump through, since Terraform is a "terraform.io-first" tool. However, many users create their own Modules, so it adds a lot of value.

The same isn't true of Providers. From a "proxying" standpoint, high-control organizations would likely not want to "proxy" the Terraform Registry for Providers -- it's way to much risk, considering anyone can just upload whatever they'd like.

Instead, they'd likely have a review process for adding and upgrading providers. And once that's in place.... how much time are we saving by using a specialized feed type over an asset directory?

I don't have the answer to that --- but that's what we're considering :)

Thanks,
Alana

[davidroberts63]

Thank you @atripp for looking into that a bit more, it is appreciated. If hosting Terraform providers in ProGet's assets would work I can see that being sufficient. We've done something like that with puppeteer's headless chrome browser where we set env vars to have the node package pull from ProGet rather than the internet. I'll keep that in mind as we consider Terraform and how we'd implement our internal security controls surrounding it.

[mikael]

Thanks for the clarification on the differences between Terraform Modules and Providers, @atripp, and for highlighting the current options in ProGet such as Asset Directories. I completely understand the security concerns you're raising around proxying the public Terraform Registry for Providers. In high-control environments, the risk of unvetted uploads is a valid concern, and prioritizing curated approaches to reduce supply-chain risk makes sense.

That said, I don't fully agree that proxying Providers inherently introduces excessive risk for high-control organizations when combined with appropriate internal controls. In our setup, we use around 15 Providers across the organization and rely on automated update tools like Dependabot or Renovate, configured to use a custom internal registry. Proxying the registry enables these tools to surface updates automatically for the Providers we actually use, which is valuable both from a security perspective (timely visibility into fixes and CVEs) and from a developer-experience standpoint. All update suggestions are reviewed and approved by our platform team before being merged, so nothing is consumed unchecked.

Manually downloading and uploading Providers into Asset Directories is certainly a viable option, and I appreciate you calling it out. For us, however, maintaining that process across multiple Providers and frequent releases doesn't scale well long-term and adds operational overhead that automation can handle more reliably.

[davidroberts63]

We have a similar setup to @mikael . While we do manually update some assets in ProGet, we also have automated processes using the ProGet Assets API. Between that, and tools such as Renovate/Dependabot it works okay.

I do agree with @mikael that proxying Terraform providers would not introduce risk. It actually helps reduce risk by ensuring the organization can be confident of which version of the providers are being used internally. Its many of the same reasons that organizations use ProGet in the first place. While I didn't mention it in my previous reply, the idea that "anyone can just upload whatever they'd like" is a major point of proxying things such as Terraform Providers. That's not likely to be open for all other teams to upload. It's quite likely, such as in our case, to be tightly controlled by a smaller team to validate what gets uploaded.

Again, I understand the balance of a specialized feed type over using the assets directory with complexity in mind. And that is something we, as a customer, do leverage with success. I just wanted to echo the thoughts @mikael as we may be in a similar environment.