Force Terraform to install providers from local disk only, disabling Terraform Registry

Question

Since 1995, we have used an update mechanism which

• cleanly updates and removes software
• centrally stores all software meta-data internally to manage needs and artifacts from a single source of truth
• NEVER triggers itself arbitrarily.

While we understand terraform has begun reaching out to a registry in a brave reinvention of that wheel without any of those features, we wish to disable it completely. Our current kit includes only one plugin:

terraform-0.13.0-1.el7.harbottle.x86_64
golang-github-terraform-provider-vsphere-1.13.0-0.1.x86_64

The goal is

1. never check the registry
2. return an error if the given module is not installed

and I'd be very grateful for good suggestions toward that end. Is there a setting I've overlooked, or can we fake it by telling it to look somewhere empty? Is there a -stay-in-your-lane switch?

Clarification:

• the add-on package is a go-build package which delivers a single artifact /usr/bin/terraform-provider-vsphere and nothing else. This has worked wonderfully for all previous incarnations and may have only begun to act up in v13.

Update: These things failed:

• terraform init -plugin-dir=/dev/shm
• terraform init -get-plugins=false
• terraform init -get=false
• setting terraform::required_providers::vsphere::source=""
• echo "disable_checkpoint = true" > ~/.terraformrc

$ terraform init -get-plugins=false

Initializing the backend...

Initializing provider plugins...
- Finding latest version of -/vsphere...
- Finding latest version of hashicorp/vsphere...

Update: I'm still a bit off:

rpm -qlp golang-github-terraform-provider-vsphere
/usr/share/terraform/plugins/registry.terraform.io/hashicorp/vsphere/1.14.0/linux_amd64/terraform-provider-vsphere

I feel I'm really close. /usr/share/ is in the XDG default search path, and it DOES seem to find the location, but it seems to check the registry first/at-all, which is unexpected.

Initializing provider plugins...
- Finding latest version of hashicorp/vsphere...
- Finding latest version of -/vsphere...
- Installing hashicorp/vsphere v1.14.0...
- Installed hashicorp/vsphere v1.14.0 (unauthenticated)

Error: Failed to query available provider packages

Are we sure it stops checking if it has something local, and that it does that by default? Did I read that right?

Answer 1

What you are describing here sounds like the intention of the Provider Installation settings in Terraform's CLI configuration file.

Specifically, you can put your provider files in a local filesystem directory of your choice -- for the sake of this example, I'm going to arbitrarily choose /usr/local/lib/terraform, and then write the following in the CLI configuration file:

provider_installation {
  filesystem_mirror {
    path = "/usr/local/lib/terraform"
  }
}

If you don't already have a CLI configuration file, you can put this in the file ~/.terraformrc.

With the above configuration, your golang-github-terraform-provider-vsphere-1.13.0-0.1.x86_64 package would need to place the provider's executable at the following path (assuming that you're working with a Linux system):

/usr/local/lib/terraform/registry.terraform.io/hashicorp/vsphere/1.30.0/linux_amd64/terraform-provider-vsphere_v1.13.0_x4

(The filename above is the one in the official vSphere provider release, but if you're building this yourself from source then it doesn't matter what exactly it's called as long as it starts with terraform-provider-vsphere.)

It looks like you are in the process of completing an upgrade from Terraform v0.12, and so Terraform is also trying to install the legacy (un-namespaced) version of this provider, -/vsphere. Since you won't have that in your local directory the installation of that would fail, but with the knowledge that this provider is now published at hashicorp/vsphere we can avoid that by manually migrating it in the state, thus avoiding the need for Terraform to infer this automatically on the next terraform apply:

terraform state replace-provider 'registry.terraform.io/-/vsphere' 'registry.terraform.io/hashicorp/vsphere'

After you run this command your latest state snapshot will not be compatible with Terraform 0.12 anymore, so if you elect to abort your upgrade and return to 0.12 you will need to restore the previous version from a backup. If your state is not stored in a location that naturally retains historical versions, one way to get such a backup is to run terraform state pull with a Terraform 0.12 executable and save the result to a file. (By default, Terraform defers taking this action until terraform apply to avoid upgrading the state format until it would've been making other changes anyway.)

---

The provider_installation configuraton above is an answer if you want to make this true for all future use of Terraform, which seems to be your goal here, but for completeness I also want to note that the following command should behave in an equivalent way to the result of the above configuration if you want to force a local directory only for one particular invocation of terraform init:

terraform init -plugin-dir=/usr/local/lib/terraform

---

Since you seem to be upgrading from Terraform 0.12, it might also interest you to know that Terraform 0.13's default installation behavior (without any special configuration) is the same as Terraform 0.12 with the exception of now expecting a different local directory structure than before, to represent the hierarchical provider namespace. (That is, to distinguish hashicorp/vsphere from a hypothetical othernamespace/vsphere.)

Specifically, Terraform 0.13 (as with Terraform 0.12) will skip contacting the remote registry for any provider for which it can discover at least one version available in the local filesystem.

It sounds like your package representing the provider was previously placing a terraform-provider-vsphere executable somewhere that Terraform 0.12 could find and use it. You can adapt that strategy to Terraform 0.13 by placing the executable at the following location:

/usr/local/share/terraform/plugins/registry.terraform.io/hashicorp/vsphere/1.30.0/linux_amd64/terraform-provider-vsphere_v1.13.0_x4

(Again, the exact filename here isn't important as long as it starts with terraform-provider-vsphere.)

/usr/local/share here is assuming one of the default data directories from the XDG Base Directory specification, but if you have XDG_DATA_HOME/XDG_DATA_DIRS overridden on your system then Terraform should respect that and look in the other locations you've listed.

The presence of such a file, assuming you haven't overridden the default behavior with an explicit provider_installation block, will cause Terraform to behave as if you had written the following in the CLI configuration:

provider_installation {
  filesystem_mirror {
    path    = "/usr/local/share/terraform/plugins"
    include = ["hashicorp/vsphere"]
  }
  direct {
    exclude = ["hashicorp/vsphere"]
  }
}

This form of the configuration forces local installation only for the hashicorp/vsphere provider, thus mimicking what Terraform 0.12 would've done with a local plugin file terraform-provider-vsphere. You can get the more thorough behavior of never contacting remote registries with a configuration like the one I opened this answer with, which doesn't include a direct {} block at all.

Answer 2

I created the file .terraformrc in /root and added the below content and it worked. Now, I am able to spin things without the internet

disable_checkpoint = true
plugin_cache_dir   = "<path to providers in local>"

provider_installation {
  filesystem_mirror {
    path    = "<path to providers in local>"
    include = ["registry.terraform.io/*/*"]
  }
  direct {
    exclude = ["registry.terraform.io/*/*"]
  }
}

Ensure all the required providers are downloaded and available in the path you provide in the file above.