template provider not working with terraform 0.12.31

Question

template provider not working with terraform 0.12.31

bendavies opened this issue 2 years ago · comments

Hi there,

I've installed the template provider as per:

m1-terraform-provider-helper install hashicorp/template -v 2.2.0

Expected Behavior

Terraform commands work:

Actual Behavior

Error installing provider "template": no available version is compatible for the requested platform.

Steps to Reproduce (including precondition)

m1-terraform-provider-helper status
Status: Active
Local providers are us
m1-terraform-provider-helper list
hashicorp/template -> 2.2.0
ben/.terraform.d -> plugins

terraform init (with trace logging):

2022/06/16 11:19:35 [DEBUG] checking for provider in "."
2022/06/16 11:19:35 [DEBUG] checking for provider in "/opt/homebrew/opt/terraform@0.12/bin"
2022/06/16 11:19:35 [DEBUG] checking for provider in ".terraform/plugins/darwin_arm64"
2022/06/16 11:19:35 [DEBUG] found provider "terraform-provider-aws_v3.75.2_x5"
2022/06/16 11:19:35 [DEBUG] checking for provider in "/Users/ben/.terraform.d/plugins"
2022/06/16 11:19:35 [DEBUG] checking for provider in "/Users/ben/.terraform.d/plugins/darwin_arm64"
2022/06/16 11:19:35 [WARN] found legacy provider "terraform-provider-template_2.2.0_x5"
2022/06/16 11:19:35 [DEBUG] found valid plugin: "template_2.2.0_x5", "0.0.0", "/Users/ben/.terraform.d/plugins/darwin_arm64/terraform-provider-template_2.2.0_x5"
2022/06/16 11:19:35 [DEBUG] found valid plugin: "aws", "3.75.2", "/Users/ben/Code/owsy/infrastructure-live/scotam-prod/us-west-2/scotam-prod/data-stores/flowable/postgres/.terragrunt-cache/CoUyt-8Q4K3IIvxGLbTrKVSQw-c/wQX_94XlHXOC511wEnd47cONrjA/data-stores/rds/.terraform/plugins/darwin_arm64/terraform-provider-aws_v3.75.2_x5"

2022/06/16 11:19:35 [DEBUG] plugin requirements: "template"=""
2022/06/16 11:19:35 [DEBUG] plugin requirements: "aws"="~> 3.0"
Initializing provider plugins...
2022/06/16 11:19:35 [DEBUG] Service discovery for registry.terraform.io at https://registry.terraform.io/.well-known/terraform.json
2022/06/16 11:19:35 [TRACE] HTTP client GET request to https://registry.terraform.io/.well-known/terraform.json
- Checking for available provider plugins...
2022/06/16 11:19:35 [DEBUG] fetching provider versions from "https://registry.terraform.io/v1/providers/-/template/versions"
2022/06/16 11:19:35 [DEBUG] GET https://registry.terraform.io/v1/providers/-/template/versions
2022/06/16 11:19:35 [TRACE] HTTP client GET request to https://registry.terraform.io/v1/providers/-/template/versions

Error installing provider "template": no available version is compatible for the requested platform.

Terraform analyses the configuration and state and automatically downloads
plugins for the providers used. However, when attempting to download this
plugin an unexpected error occurred.

This may be caused if for some reason Terraform is unable to reach the
plugin repository. The repository may be unreachable if access is blocked
by a firewall.

If automatic installation is not possible or desirable in your environment,
you may alternatively manually install plugins by downloading a suitable
distribution package and placing the plugin's executable file in the
following directory:
    terraform.d/plugins/darwin_arm64

I don't know if this is a problem, but terrform asks for the plugin to be in ~/.terraform.d/plugins/darwin_arm64, but it's in ~/.terraform.d/plugins/registry.terraform.io/hashicorp/template/2.2.0/darwin_arm64

after copying terraform-provider-template_2.2.0_x5 to ~/.terraform.d/plugins/darwin_arm64, the log is slightly the different:

(note the WARN] found legacy provider "terraform-provider-template_2.2.0_x5")

2022/06/16 11:19:35 [DEBUG] checking for provider in "."
2022/06/16 11:19:35 [DEBUG] checking for provider in "/opt/homebrew/opt/terraform@0.12/bin"
2022/06/16 11:19:35 [DEBUG] checking for provider in ".terraform/plugins/darwin_arm64"
2022/06/16 11:19:35 [DEBUG] found provider "terraform-provider-aws_v3.75.2_x5"
2022/06/16 11:19:35 [DEBUG] checking for provider in "/Users/ben/.terraform.d/plugins"
2022/06/16 11:19:35 [DEBUG] checking for provider in "/Users/ben/.terraform.d/plugins/darwin_arm64"
2022/06/16 11:19:35 [WARN] found legacy provider "terraform-provider-template_2.2.0_x5"
2022/06/16 11:19:35 [DEBUG] found valid plugin: "template_2.2.0_x5", "0.0.0", "/Users/ben/.terraform.d/plugins/darwin_arm64/terraform-provider-template_2.2.0_x5"
2022/06/16 11:19:35 [DEBUG] found valid plugin: "aws", "3.75.2", "/Users/ben/Code/owsy/infrastructure-live/scotam-prod/us-west-2/scotam-prod/data-stores/flowable/postgres/.terragrunt-cache/CoUyt-8Q4K3IIvxGLbTrKVSQw-c/wQX_94XlHXOC511wEnd47cONrjA/data-stores/rds/.terraform/plugins/darwin_arm64/terraform-provider-aws_v3.75.2_x5"

2022/06/16 11:19:35 [DEBUG] plugin requirements: "template"=""
2022/06/16 11:19:35 [DEBUG] plugin requirements: "aws"="~> 3.0"
Initializing provider plugins...
2022/06/16 11:19:35 [DEBUG] Service discovery for registry.terraform.io at https://registry.terraform.io/.well-known/terraform.json
2022/06/16 11:19:35 [TRACE] HTTP client GET request to https://registry.terraform.io/.well-known/terraform.json
- Checking for available provider plugins...
2022/06/16 11:19:35 [DEBUG] fetching provider versions from "https://registry.terraform.io/v1/providers/-/template/versions"
2022/06/16 11:19:35 [DEBUG] GET https://registry.terraform.io/v1/providers/-/template/versions
2022/06/16 11:19:35 [TRACE] HTTP client GET request to https://registry.terraform.io/v1/providers/-/template/versions

Error installing provider "template": no available version is compatible for the requested platform.

Terraform analyses the configuration and state and automatically downloads
plugins for the providers used. However, when attempting to download this
plugin an unexpected error occurred.

This may be caused if for some reason Terraform is unable to reach the
plugin repository. The repository may be unreachable if access is blocked
by a firewall.

If automatic installation is not possible or desirable in your environment,
you may alternatively manually install plugins by downloading a suitable
distribution package and placing the plugin's executable file in the
following directory:
    terraform.d/plugins/darwin_arm64

Screenshot on This Problem (if possible)

Your Environment

OS: 12.4
m1-terraform-provider-helper version: 0.6.0

Emlyn Stokes commented 2 years ago

Thanks!

Martin · Answer 1 · Thu Jun 16 2022 23:33:24 GMT+0800 (China Standard Time)

I just tried it out.
I created a main.tf file with

provider "template" {
  version = "2.1.1"
}

Then tried to run terraform init:

Provider registry.terraform.io/hashicorp/template v2.1.1 does not have a package available for your current platform, darwin_arm64.

After a quick m1-terraform-provider-helper activate && m1-terraform-provider-helper install hashicorp/template -v 2.1.1 I could successfully run terraform init.

What is your terraform version?
Have you considered this part of the error message?

This may be caused if for some reason Terraform is unable to reach the
plugin repository. The repository may be unreachable if access is blocked
by a firewall.

Even in your first tf init the hashicorp/template installation is not working, so that also points into the direction of some network/tf failure 🤔

Ben Davies · Answer 2 · Fri Jun 17 2022 00:19:26 GMT+0800 (China Standard Time)

it's an old version unfortunately, terraform v0.12.31. i cannot upgrade at this time.

main.tf

data "template_file" "init" {
  template = "${file("${path.module}/init.tpl")}"
}

TF_LOG=trace terraform init
2022/06/16 17:18:09 [INFO] Terraform version: 0.12.31
2022/06/16 17:18:09 [INFO] Go runtime version: go1.17.2
2022/06/16 17:18:09 [INFO] CLI args: []string{"/opt/homebrew/opt/terraform@0.12/bin/terraform", "init"}
2022/06/16 17:18:09 [DEBUG] Attempting to open CLI config file: /Users/ben/.terraformrc
2022/06/16 17:18:09 Loading CLI configuration from /Users/ben/.terraformrc
2022/06/16 17:18:09 [DEBUG] checking for credentials in "/Users/ben/.terraform.d/plugins"
2022/06/16 17:18:09 [DEBUG] checking for credentials in "/Users/ben/.terraform.d/plugins/darwin_arm64"
2022/06/16 17:18:09 [INFO] Checkpoint disabled. Not running.
2022/06/16 17:18:09 [INFO] CLI command args: []string{"init"}

Initializing the backend...
2022/06/16 17:18:09 [TRACE] Meta.Backend: no config given or present on disk, so returning nil config
2022/06/16 17:18:09 [TRACE] Meta.Backend: backend has not previously been initialized in this working directory
2022/06/16 17:18:09 [DEBUG] New state was assigned lineage "766211ac-f24d-0a22-e72f-385d7c9142cf"
2022/06/16 17:18:09 [TRACE] Meta.Backend: using default local state only (no backend configuration, and no existing initialized backend)
2022/06/16 17:18:09 [TRACE] Meta.Backend: instantiated backend of type <nil>
2022/06/16 17:18:09 [DEBUG] checking for provider in "."
2022/06/16 17:18:09 [DEBUG] checking for provider in "/opt/homebrew/opt/terraform@0.12/bin"
2022/06/16 17:18:09 [DEBUG] checking for provider in "/Users/ben/.terraform.d/plugins"
2022/06/16 17:18:09 [DEBUG] checking for provider in "/Users/ben/.terraform.d/plugins/darwin_arm64"
2022/06/16 17:18:09 [WARN] found legacy provider "terraform-provider-template_2.2.0_x5"
2022/06/16 17:18:09 [DEBUG] found valid plugin: "template_2.2.0_x5", "0.0.0", "/Users/ben/.terraform.d/plugins/darwin_arm64/terraform-provider-template_2.2.0_x5"
2022/06/16 17:18:09 [DEBUG] checking for provisioner in "."
2022/06/16 17:18:09 [DEBUG] checking for provisioner in "/opt/homebrew/opt/terraform@0.12/bin"
2022/06/16 17:18:09 [DEBUG] checking for provisioner in "/Users/ben/.terraform.d/plugins"
2022/06/16 17:18:09 [DEBUG] checking for provisioner in "/Users/ben/.terraform.d/plugins/darwin_arm64"
2022/06/16 17:18:09 [INFO] Failed to read plugin lock file .terraform/plugins/darwin_arm64/lock.json: open .terraform/plugins/darwin_arm64/lock.json: no such file or directory

2022/06/16 17:18:09 [TRACE] Meta.Backend: backend <nil> does not support operations, so wrapping it in a local backend
2022/06/16 17:18:09 [TRACE] backend/local: state manager for workspace "default" will:
 - read initial snapshot from terraform.tfstate
 - write new snapshots to terraform.tfstate
 - create any backup at terraform.tfstate.backup
2022/06/16 17:18:09 [TRACE] statemgr.Filesystem: reading initial snapshot from terraform.tfstate
2022/06/16 17:18:09 [TRACE] statemgr.Filesystem: snapshot file has nil snapshot, but that's okay
2022/06/16 17:18:09 [TRACE] statemgr.Filesystem: read nil snapshot
2022/06/16 17:18:09 [DEBUG] checking for provider in "."
2022/06/16 17:18:09 [DEBUG] checking for provider in "/opt/homebrew/opt/terraform@0.12/bin"
2022/06/16 17:18:09 [DEBUG] checking for provider in "/Users/ben/.terraform.d/plugins"
2022/06/16 17:18:09 [DEBUG] checking for provider in "/Users/ben/.terraform.d/plugins/darwin_arm64"
2022/06/16 17:18:09 [WARN] found legacy provider "terraform-provider-template_2.2.0_x5"
2022/06/16 17:18:09 [DEBUG] found valid plugin: "template_2.2.0_x5", "0.0.0", "/Users/ben/.terraform.d/plugins/darwin_arm64/terraform-provider-template_2.2.0_x5"
2022/06/16 17:18:09 [DEBUG] plugin requirements: "template"=""
2022/06/16 17:18:09 [DEBUG] Service discovery for registry.terraform.io at https://registry.terraform.io/.well-known/terraform.json
2022/06/16 17:18:09 [TRACE] HTTP client GET request to https://registry.terraform.io/.well-known/terraform.json
Initializing provider plugins...
- Checking for available provider plugins...
2022/06/16 17:18:09 [DEBUG] fetching provider versions from "https://registry.terraform.io/v1/providers/-/template/versions"
2022/06/16 17:18:09 [DEBUG] GET https://registry.terraform.io/v1/providers/-/template/versions
2022/06/16 17:18:09 [TRACE] HTTP client GET request to https://registry.terraform.io/v1/providers/-/template/versions

Error installing provider "template": no available version is compatible for the requested platform.

Terraform analyses the configuration and state and automatically downloads
plugins for the providers used. However, when attempting to download this
plugin an unexpected error occurred.

This may be caused if for some reason Terraform is unable to reach the
plugin repository. The repository may be unreachable if access is blocked
by a firewall.

If automatic installation is not possible or desirable in your environment,
you may alternatively manually install plugins by downloading a suitable
distribution package and placing the plugin's executable file in the
following directory:
    terraform.d/plugins/darwin_arm64


Error: no available version is compatible for the requested platform

Ben Davies · Answer 3 · Fri Jun 17 2022 00:22:02 GMT+0800 (China Standard Time)

What is your terraform version? Have you considered this part of the error message?
This may be caused if for some reason Terraform is unable to reach the
plugin repository. The repository may be unreachable if access is blocked
by a firewall.
Even in your first tf init the hashicorp/template installation is not working, so that also points into the direction of some network/tf failure 🤔

i can hit those urls in the browser just fine, i don't think that's the issue.

Ben Davies · Answer 4 · Fri Jun 17 2022 00:25:52 GMT+0800 (China Standard Time)

seems ok with terraform 1.2.3, so i guess it's an issue with version 0.12.31?

Ben Davies · Answer 5 · Fri Jun 17 2022 00:31:57 GMT+0800 (China Standard Time)

think i fixed it.

i found an old reference here:
https://www.terraform.io/language/configuration-0-11/providers#plugin-names-and-versions (i can't find v0.12 of the docs?) that the filename must be terraform-provider-<NAME>_vX.Y.Z

i renamed terraform-provider-template_2.2.0_x5 to terraform-provider-template_v2.2.0 and it worked.

Ben Davies · Answer 6 · Fri Jun 17 2022 00:37:49 GMT+0800 (China Standard Time)

x5 is apparently the protocol version
https://groups.google.com/g/terraform-tool/c/EolJNDaPqw8

anyway, terraform 0.12.31 didn't like it!

Ben Davies · Answer 7 · Fri Jun 17 2022 00:48:20 GMT+0800 (China Standard Time)

to confirm, my entire fix is:

cp \
    /Users/ben/.terraform.d/plugins/registry.terraform.io/hashicorp/template/2.2.0/darwin_arm64/terraform-provider-template_2.2.0_x5 \
    /Users/ben/.terraform.d/plugins/darwin_arm64/terraform-provider-template_v2.2.0

Martin · Answer 8 · Fri Jun 17 2022 14:58:12 GMT+0800 (China Standard Time)

Oh, that's a really interesting find, thanks for investigating! I will also do some tests on my side to verify whether the x5 suffix is really needed for newer terraform versions. If not, we can omit it and be compatible with all versions :)

Martin · Answer 9 · Fri Jun 17 2022 16:20:09 GMT+0800 (China Standard Time)

@bendavies does it also work for you if you simple remove the _x5 prefix from the binary name?
In your fix you are copying the entire binary to a new path, I want to verify if it is also working when the binary stays in the same location but I get rid of the _x5. Would make fixing a lot easier :D

Ben Davies · Answer 10 · Fri Jun 17 2022 18:53:12 GMT+0800 (China Standard Time)

@Junkern confirmed that it MUST be in /Users/ben/.terraform.d/plugins/darwin_arm64/ for terraform 0.12.31.

i tried this and if failed:

cp \
    /Users/ben/.terraform.d/plugins/registry.terraform.io/hashicorp/template/2.2.0/darwin_arm64/terraform-provider-template_2.2.0_x5 \
    /Users/ben/.terraform.d/plugins/registry.terraform.io/hashicorp/template/2.2.0/darwin_arm64/terraform-provider-template_v2.2.0

Ben Davies · Answer 11 · Fri Jun 17 2022 18:54:39 GMT+0800 (China Standard Time)

btw, it's not simply removing the _x5 postfix.
it's also adding v before the version:
terraform-provider-template_2.2.0_x5 -> terraform-provider-template_v2.2.0

Martin · Answer 12 · Tue Jul 19 2022 21:21:21 GMT+0800 (China Standard Time)

@bendavies released with v0.7.1 :)
Feel free to post any feedback in here

Ben Davies · Answer 13 · Tue Jul 19 2022 23:58:20 GMT+0800 (China Standard Time)

hi @Junkern, upgraded and i get this:

❯ m1-terraform-provider-helper install hashicorp/template -v 2.2.0
Repo: https://github.com/hashicorp/terraform-provider-template
GitRepo: https://github.com/hashicorp/terraform-provider-template
HEAD is now at 477a7b5 Cleanup after v2.2.0 release
2022/07/19 16:57:47 Pulling newest changes from https://github.com/hashicorp/terraform-provider-template
Already on 'master'
Your branch is up to date with 'origin/master'.
Already up to date.
2022/07/19 16:57:48 version: 2.2.0
==> Checking that code complies with gofmt requirements...
go install
error: invalid character 'T' looking for beginning of value

Emlyn Stokes · Answer 14 · Wed Jul 20 2022 00:03:04 GMT+0800 (China Standard Time)

I'm also getting the same error: invalid character 'T' looking for beginning of value error using 0.12.31

Ben Davies · Answer 15 · Wed Jul 20 2022 00:04:52 GMT+0800 (China Standard Time)

confirming this is a problem with 0.7.1. i rolled back to 0.7.0 and it doesn't occur

Martin · Answer 16 · Wed Jul 20 2022 00:20:21 GMT+0800 (China Standard Time)

@bendavies @eastokes you are both on terraform 0.12.31? My assumption is, that the terraform version -json command does not work on 0.12.31. Could you double check that?

Martin · Answer 17 · Wed Jul 20 2022 00:23:07 GMT+0800 (China Standard Time)

According to the source code (https://github.com/hashicorp/terraform/blob/v0.12.31/command/version.go#L43)
It is Terraform v0.12.31

Martin · Answer 18 · Wed Jul 20 2022 00:24:25 GMT+0800 (China Standard Time)

For my currently installed version it is also Terraform v1.1.2. So I will update to not use the -json flag to get the terraform version.

Emlyn Stokes · Answer 19 · Wed Jul 20 2022 00:26:34 GMT+0800 (China Standard Time)

❯ terraform version -json
Terraform v0.12.31

Your version of Terraform is out of date! The latest version
is 1.2.5. You can update by downloading from https://www.terraform.io/downloads.html

Ben Davies · Answer 20 · Wed Jul 20 2022 00:34:39 GMT+0800 (China Standard Time)

same as @eastokes here

❯ terraform version -json
Terraform v0.12.31

Martin · Answer 21 · Wed Jul 20 2022 00:51:36 GMT+0800 (China Standard Time)

Just released 0.7.2, hopefully fixes the error...

Ben Davies · Answer 22 · Wed Jul 20 2022 01:20:09 GMT+0800 (China Standard Time)

looks good to me!