Enable support for Azure US Government for provider-azure
aabond opened this issue · comments
How to categorize this issue?
/area control-plane
/area usability
/area robustness
/kind enhancement
/priority 1
/platform azure
What would you like to be added:
Need the ability to deploy shoot clusters on Azure US Government Cloud. It seems the Azure infrastructure secret needs the ability to distinguish between Azure commercial and Azure US Government clouds. This parameter should then be provided to the underlying terraform code to switch to the Azure US Government API endpoint.
Why is this needed:
SAP NS2 needs to deploy SAP FieldGlass on to Azure US Government Cloud.
Current error being displayed while trying to deploy shoot cluster to Azure US Government:
Flow "Shoot cluster reconciliation" encountered task errors: [task "Waiting until shoot infrastructure has been reconciled" failed: Error while waiting for Infrastructure shoot--fglns2can--test-cluster/test-cluster to become ready: error during reconciliation: Error reconciling infrastructure: failed to apply the terraform config: Terraform execution for command 'apply' could not be completed:
* Error building account: Error getting authenticated object ID: Error listing Service Principals: autorest.DetailedError{Original:adal.tokenRefreshError{message:"adal: Refresh request failed. Status Code = '400'. Response body: {\"error\":\"invalid_request\",\"error_description\":\"AADSTS900382: Confidential Client is not supported in Cross Cloud request.\\r\\nTrace ID: <omitted>\\r\\nCorrelation ID: <omitted>\\r\\nTimestamp: 2021-10-06 18:48:08Z\",\"error_codes\":[900382],\"timestamp\":\"2021-10-06 18:48:08Z\",\"trace_id\":\"<omitted>\",\"correlation_id\":\"<omitted>\"} Endpoint https://login.microsoftonline.com/<omitted>/oauth2/token?api-version=1.0", resp:(*http.Response)(0xc0001377a0)}, PackageType:"azure.BearerAuthorizer", Method:"WithAuthorization", StatusCode:400, Message:"Failed to refresh the Token for request to https://graph.windows.net/<omitted>/servicePrincipals?%24filter=appId+eq+%27<omitted>%27&api-version=1.6", ServiceError:[]uint8(nil), Response:(*http.Response)(0xc0001377a0)}
on tf/main.tf line 1, in provider "azurerm":
1: provider "azurerm" {] Operation will be retried.
Further notes:
I came across this post when searching for the error:
Which basically said adding the flag -Environment AzureUSGovernment solved their issue.
Hello @aabond,
thanks for the request.
I guess there will be adaptions on the configuration for multiple components are required.
First of course the obvious things to do:
For the cloud-controller-manager we will need to update the cloud-provider-config by adding the cloud="USGovernmentCloud". Same for the csi/kube-controller-manager cloud-provider-configs.
See here: https://kubernetes-sigs.github.io/cloud-provider-azure/install/configs
We need also to adapt the mcm provider for Azure so that the correct environment is chosen: https://github.com/gardener/machine-controller-manager-provider-azure/blob/master/pkg/spi/azure.go#L36
And last we need also adapt the internal Azure client setup of the Azure extension.
/assign
/unassign as currently there is no active work on this