State file for multi-region deployment

[Sam S]

Let's say I have the following directory structure. The top most terragrunt.hcl under account-2345 has an S3 bucket parameter associated with the remote state file. So let's say I set it to us-east-1. Now, let's say us-east-1 is down, my environment in us-west-2 is still running and i shift my services over there. I need to update my mysql cluster and therefore need to modify my TF code. How would I do this if I can't get to my state file?

Would this require me to replicate the bucket to us-west-2 and modify my terragrunt.hcl file to point to that bucket?

account-2345/
├── terragrunt.hcl
└── us-east-1/
    └── prod/
        ├── mysql/
        │   └── terragrunt.hcl
        └── webserver-cluster/
            └── terragrunt.hcl
└── us-west-2/
    └── prod/
        ├── mysql/
        │   └── terragrunt.hcl
        └── webserver-cluster/
            └── terragrunt.hcl

[Yevgeniy Brikman]

Would this require me to replicate the bucket to us-west-2 and modify my terragrunt.hcl file to point to that bucket?

Yes and no.

If you store your state in one AWS region, and that region is down, then the only way to keep working with that same state file is if, before the outage, you had been replicating that state file to another region that is still running.

However, this may not help you. For example, if that state file is pointing to EC2 instances in us-east-1, and us-east-1 is down, then when Terraform tries to fetch information about those EC2 instances, it'll fail.

So in practice, if you're looking at disaster recovery, you probably want to share the Terraform code, but have separate state files for different regions.

[Kyle G. Lundstedt]

@brikis98 your response regarding disaster recovery makes sense to me. In a similar vein, however, I've struggled a little to understand the reasoning behind the recommended folder hierarchy in the multi-account case. Doesn't @luckytaxi's question, as well as issues #6 and #19, seem to point toward an approach that has one account for each region-environment combination?

To clarify a bit, the folder structure in this repo nests multiple regions in each account, and one more environments within each region.

account
 └ _global
 └ region (one or more)
    └ terragrunt.hcl (account-level, remote state for all environments in that region)
    └ _global
    └ environment (one or more)
       └ resources (multiple)

An alternative form of account structure/folder organization might look like:

account1 (1 account for each region-environment combo; i.e., "prod-us-east-1")
 └ resource1
 └ resource2
 └ terragrunt.hcl (account-level, remote state for the prod env in us-east-1)

account2 (1 account for each region-environment combo; i.e., "dev-us-east-1")
 └ resource1
   ...

The current structure also seems to require multiple YAML files in various subdirectories which contain account-, region-, and env-level variables.

I'm just wondering if the alternative "1 account per region-env combo" approach might be easier to understand and manage. Is there an obvious drawback to this approach that I'm missing?

BTW, thanks for updating Terraform Up and Running; the 2nd edition really helped get my arms around the new Terraform 0.12 features!

[Yevgeniy Brikman]

A region-environment combo is certainly possible too. I find it helpful to mentally group everything in one account together, as you do auth on a per-account basis in AWS, so the hierarchy account -> region helps with that. But in most ways, it is equivalent to having a flat collection of region-account folders, so feel free to use that if you prefer.

[Kyle G. Lundstedt]

Thanks for the feedback, @brikis98! My biggest concern with the "1 account per regions-environment" combination, given the README for this repo, was the issue of sharing Route 53 hosted zones, and CloudTrail with other regions, and Route 53 A records, SNS topics, and ECR repos with other environments. We use Jumpcloud for federated authentication to keep identity/authentication "dry" across accounts.

[Sheikh Irfan]

How can I create resources in two different region and share these resources in each others in AWS account and state file is one

      main-account/
      ├── terragrunt.hcl
      └── us-east-2/
      │        ├── cloudfront-distribution/
      │        │   └── terragrunt.hcl
      │        └── route53-hosted-zone/
      │              └── terragrunt.hcl
      └── us-west-1/
              └── acm-certificate/
                  └── terragrunt.hcl