Allow setting HCP Terraform "Working Directory" setting when creating a new workspace

[Robert Colbert]

Terraform Version

Terraform v1.9.0
on windows_amd64

Use Cases

We create terraform modules for consumption by other teams. To do this, we leverage a test folder to hold terratest configs referencing the parent path to run CI/CD pipeline testing. This works great when we manually create the workspaces (via API call) and set the working directory to the test folder but if we wanted to stay purely in the CLI, having the ability to set the working directory when using cloud or remote state configs would simplify our workflow.

Attempted Solutions

Creating the workspaces using TFC REST API works but requires additional management and overhead.

Proposal

Add a flag or argument to the "terraform workspace new" command to add an optional workingdir relative path.

terraform workspace new -workingdir test TestWorkspace
-or to match the actual documentation-
terraform workspace new TestWorkspace test

Note: There is a documentation bug where all of the workspace related commands end with [DIR] implying you could pass in a directory path of some kind.

References

No response

[James Bardin]

Hi @rmcolbert,

Thanks for filing the issue. All terraform commands accept a top-level -chdir flag to indicate the working directory of the command. So your command would be written as:

terraform -chdir=test workspace new TestWorkspace

The CLI help text all appears to be up to date, but it looks like the workspace command in the website docs was missed.

Thanks!

[Robert Colbert]

That doesn't create/update the "Terraform Working Directory" in the tfc workspace config itself so while the cli runs from inside the chdir, the tfc workspace job tries to run from the root and fails.

[James Bardin]

Thanks @rmcolbert, I'll flag this for the cloud team to look at too. The "workspace" concept locally is different from that of TFC, and there isn't necessarily full command parity between the two.

[Martin Atkins]

From reading the above it seems like there's a bit of collision of terminology here, so I just want to try to clarify what this represents to make sure everyone's talking about the same thing. 😀

HCP Terraform workspaces have a setting called "working directory" which contains a relative path from the root of the workspace's main source package (typically a Git repository) to the directory containing the root module of the workspace's Terraform configuration. That is a separate idea from Terraform CLI's own working directory, which is typically just whatever directory you ran terraform from, but can potentially be overridden using the -chdir option as @jbardin described.

HCP Terraform features then use that in two main ways:

• When HCP Terraform is running Terraform CLI inside its remote execution environment, it first retrieves the workspace's main source package into the job's temporary directory and then runs Terraform CLI with its working directory set to the "working directory" specified in the settings.
• When Terraform CLI is uploading source files from your local computer to use as a source package for a remotely-executed plan or apply, it uses the workspace's working directory settings to decide how many parent directories to include in the uploaded source package. For example, if the working directory were given as modules/foo, Terraform CLI would upload everything in ../.. relative to the directory containing the root module.

Since Terraform CLI's terraform workspace subcommands are built around Terraform CLI's limited view of workspaces, rather than HCP Terraform's extended features, there are currently not any options for configuring the HCP Terraform-specific settings of the new workspace, including this "working directory" setting. Instead, you can call the HCP Terraform API Create a Workspace action directly and use data.attributes.working-directory to specify the working directory setting for the new workspace.

Although this issue is focused only on the working directory setting, I think resolving it means deciding whether and how Terraform CLI should deal with HCP Terraform-specific workspace settings in general. Adding new options to terraform workspace new as proposed here is one possibility, though a potentially-confusing one for those not using HCP Terraform. Another option would be to support this through the HCP CLI instead, since that is designed only for working with HashiCorp Cloud Platform services. (It doesn't yet have any Terraform-related subcommands at the time I'm writing this, though.)

As @jbardin said, this will ultimately be driven by the HCP Terraform teams. I just wanted to add some extra context because it seemed like the goals of this issue might be unclear to a hypothetical reader that isn't familiar with HCP Terraform features.

[Robert Colbert]

@apparentlymart Thank you for adding clarity and context around my request.

I would like to point out that the CLI and HCL files already blur the line around this point by having the remote state and cloud configuration being able to leverage the workspace commands.

Perhaps if it was part of the cloud config block instead of a direct CLI command parameter it would fit the bill for not cluttering the CLI command while still providing an avenue for the feature. Especially since 1.6.x added the ability for setting the project assigned to a new workspace in TFC when the "terraform workspace new" command is executed.