Az Network Manager Connectivity Configuration Create/Update Not Working for Cross-tenant Hub
jbgorthy opened this issue · comments
Describe the bug
Context
When 'az network manager connect-config create', they have the option to select the property '--connectivity-topology "HubAndSpoke"'.
When hub and spoke topology is used, the customer must provide a resource id for the hub. This looks something like '--hub resource-id="/subscriptions/b7a6ea8b-8aac-46e2-9cd9-0d22ff8778bc/resourceGroups/JaredTestRG/providers/Microsoft.Network/virtualNetworks/testGroupMemberPublish" resource-type="Microsoft.Network/virtualNetworks"'
This resource id has a linked access check on it from ARM, and the resource id can be a resource from a secondary tenant.
Bug
If the resource is in a secondary tenant, it is expected that the CLI extension will recognize this behind the scenes, and fetch a secondary token to authorize for this tenant. We use the 'x-ms-authorization-auxiliary' to accomplish this. The command is currently not doing this, so all create commands with a cross-tenant hub fail from azure cli.
Note, your team has already implemented this functionality for 'az network manager group static-member create', and it is working as expected.
Related command
az network manager connect-config create
Errors
(LinkedAuthorizationFailed) The client has permission to perform action 'Microsoft.Network/virtualNetworks/write' on scope '/subscriptions/f0dc2b34-dfad-40e4-83e0-2309fed8d00b/resourceGroups/jaredgorthy-testResources/providers/Microsoft.Network/networkManagers/jaredgorthy/connectivityConfigurations/testCrossTenantFeature', however the current tenant '72f988bf-86f1-41af-91ab-2d7cd011db47' is not authorized to access linked subscription 'b7a6ea8b-8aac-46e2-9cd9-0d22ff8778bc'.
Code: LinkedAuthorizationFailed
Message: The client has permission to perform action 'Microsoft.Network/virtualNetworks/write' on scope '/subscriptions/f0dc2b34-dfad-40e4-83e0-2309fed8d00b/resourceGroups/jaredgorthy-testResources/providers/Microsoft.Network/networkManagers/jaredgorthy/connectivityConfigurations/testCrossTenantFeature', however the current tenant '72f988bf-86f1-41af-91ab-2d7cd011db47' is not authorized to access linked subscription 'b7a6ea8b-8aac-46e2-9cd9-0d22ff8778bc'.
This linked access check failure is not expected, as the client I'm using has permission on both tenants.
Issue script & Debug output
C:\Users\jaredgorthy\OneDrive - Microsoft\Desktop>az network manager connect-config create --configuration-name "testCrossTenantFeature" --description "hellow world" --applies-to-groups group-connectivity="None" is-global=false network-group-id="/subscriptions/f0dc2b34-dfad-40e4-83e0-2309fed8d00b/resourceGroups/jaredgorthy-testResources/providers/Microsoft.Network/networkManagers/jaredgorthy/networkGroups/test" use-hub-gateway=false --connectivity-topology "HubAndSpoke" --delete-existing-peering false --hub resource-id="/subscriptions/b7a6ea8b-8aac-46e2-9cd9-0d22ff8778bc/resourceGroups/JaredTestRG/providers/Microsoft.Network/virtualNetworks/testGroupMemberPublish" resource-type="Microsoft.Network/virtualNetworks" --is-global true --network-manager-name "jaredgorthy" --resource-group "jaredgorthy-testResources" --debug
cli.knack.cli: Command arguments: ['network', 'manager', 'connect-config', 'create', '--configuration-name', 'testCrossTenantFeature', '--description', 'hellow world', '--applies-to-groups', 'group-connectivity=None', 'is-global=false', 'network-group-id=/subscriptions/f0dc2b34-dfad-40e4-83e0-2309fed8d00b/resourceGroups/jaredgorthy-testResources/providers/Microsoft.Network/networkManagers/jaredgorthy/networkGroups/test', 'use-hub-gateway=false', '--connectivity-topology', 'HubAndSpoke', '--delete-existing-peering', 'false', '--hub', 'resource-id=/subscriptions/b7a6ea8b-8aac-46e2-9cd9-0d22ff8778bc/resourceGroups/JaredTestRG/providers/Microsoft.Network/virtualNetworks/testGroupMemberPublish', 'resource-type=Microsoft.Network/virtualNetworks', '--is-global', 'true', '--network-manager-name', 'jaredgorthy', '--resource-group', 'jaredgorthy-testResources', '--debug']
cli.knack.cli: init debug log:
Enable color in terminal.
Enable VT mode.
cli.knack.cli: Event: Cli.PreExecute []
cli.knack.cli: Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x018DF7F8>, <function OutputProducer.on_global_arguments at 0x01A088E8>, <function CLIQuery.on_global_arguments at 0x01A296B8>]
cli.knack.cli: Event: CommandInvoker.OnPreCommandTableCreate []
cli.azure.cli.core: Modules found from index for 'network': ['azure.cli.command_modules.network', 'azure.cli.command_modules.privatedns', 'azext_network_manager']
cli.azure.cli.core: Loading command modules:
cli.azure.cli.core: Name Load Time Groups Commands
cli.azure.cli.core: network 0.925 115 453
cli.azure.cli.core: privatedns 0.073 14 60
cli.azure.cli.core: Total (2) 0.998 129 513
cli.azure.cli.core: These extensions are not installed and will be skipped: ['azext_ai_examples', 'azext_next']
cli.azure.cli.core: Loading extensions:
cli.azure.cli.core: Name Load Time Groups Commands Directory
cli.azure.cli.core: virtual-network-manager 0.930 12 28 C:\Users\jaredgorthy.azure\cliextensions\virtual-network-manager
cli.azure.cli.core: Total (1) 0.930 12 28
cli.azure.cli.core: Loaded 139 groups, 541 commands.
cli.azure.cli.core: Found a match in the command table.
cli.azure.cli.core: Raw command : network manager connect-config create
cli.azure.cli.core: Command table: network manager connect-config create
cli.knack.cli: Event: CommandInvoker.OnPreCommandTableTruncate [<function AzCliLogging.init_command_file_logging at 0x03D0BF28>]
cli.azure.cli.core.azlogging: metadata file logging enabled - writing logs to 'C:\Users\jaredgorthy.azure\commands\2024-01-03.12-20-54.network_manager_connect-config_create.37440.log'.
az_command_data_logger: command args: network manager connect-config create --configuration-name {} --description {} --applies-to-groups {} {} {} {} --connectivity-topology {} --delete-existing-peering {} --hub {} {} --is-global {} --network-manager-name {} --resource-group {} --debug
cli.knack.cli: Event: CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument..add_subscription_parameter at 0x03D7C3E8>]
cli.knack.cli: Event: CommandInvoker.OnPostArgumentLoad []
cli.knack.cli: Event: CommandInvoker.OnPostCommandTableCreate [<function register_ids_argument..add_ids_arguments at 0x03D7C398>, <function register_cache_arguments..add_cache_arguments at 0x03D7C488>]
cli.knack.cli: Event: CommandInvoker.OnCommandTableLoaded []
cli.knack.cli: Event: CommandInvoker.OnPreParseArgs []
cli.knack.cli: Event: CommandInvoker.OnPostParseArgs [<function OutputProducer.handle_output_argument at 0x01A08938>, <function CLIQuery.handle_query_parameter at 0x01A29708>, <function register_ids_argument..parse_ids_arguments at 0x03D7C438>]
az_command_data_logger: extension name: virtual-network-manager
az_command_data_logger: extension version: 1.0.0
cli.azure.cli.core.auth.persistence: build_persistence: location='C:\Users\jaredgorthy\.azure\msal_token_cache.bin', encrypt=True
cli.azure.cli.core.auth.binary_cache: load: C:\Users\jaredgorthy.azure\msal_http_cache.bin
urllib3.util.retry: Converted retries value: 1 -> Retry(total=1, connect=None, read=None, redirect=None, status=None)
msal.authority: openid_config = {'token_endpoint': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/oauth2/v2.0/token', 'token_endpoint_auth_methods_supported': ['client_secret_post', 'private_key_jwt', 'client_secret_basic'], 'jwks_uri': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/discovery/v2.0/keys', 'response_modes_supported': ['query', 'fragment', 'form_post'], 'subject_types_supported': ['pairwise'], 'id_token_signing_alg_values_supported': ['RS256'], 'response_types_supported': ['code', 'id_token', 'code id_token', 'id_token token'], 'scopes_supported': ['openid', 'profile', 'email', 'offline_access'], 'issuer': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/v2.0', 'request_uri_parameter_supported': False, 'userinfo_endpoint': 'https://graph.microsoft.com/oidc/userinfo', 'authorization_endpoint': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/oauth2/v2.0/authorize', 'device_authorization_endpoint': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/oauth2/v2.0/devicecode', 'http_logout_supported': True, 'frontchannel_logout_supported': True, 'end_session_endpoint': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/oauth2/v2.0/logout', 'claims_supported': ['sub', 'iss', 'cloud_instance_name', 'cloud_instance_host_name', 'cloud_graph_host_name', 'msgraph_host', 'aud', 'exp', 'iat', 'auth_time', 'acr', 'nonce', 'preferred_username', 'name', 'tid', 'ver', 'at_hash', 'c_hash', 'email'], 'kerberos_endpoint': 'https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/kerberos', 'tenant_region_scope': 'WW', 'cloud_instance_name': 'microsoftonline.com', 'cloud_graph_host_name': 'graph.windows.net', 'msgraph_host': 'graph.microsoft.com', 'rbac_url': 'https://pas.windows.net'}
msal.application: Broker enabled? False
cli.azure.cli.core.auth.credential_adaptor: CredentialAdaptor.get_token: scopes=('https://management.core.windows.net//.default',), kwargs={}
cli.azure.cli.core.auth.msal_authentication: UserCredential.get_token: scopes=('https://management.core.windows.net//.default',), claims=None, kwargs={}
msal.application: Cache hit an AT
msal.telemetry: Generate or reuse correlation_id: 63819bd6-fd7a-4c72-85f6-5d89a588e221
cli.azure.cli.core.sdk.policies: Request URL: 'https://management.azure.com/subscriptions/f0dc2b34-dfad-40e4-83e0-2309fed8d00b/resourceGroups/jaredgorthy-testResources/providers/Microsoft.Network/networkManagers/jaredgorthy/connectivityConfigurations/testCrossTenantFeature?api-version=2022-01-01'
cli.azure.cli.core.sdk.policies: Request method: 'PUT'
cli.azure.cli.core.sdk.policies: Request headers:
cli.azure.cli.core.sdk.policies: 'Content-Type': 'application/json'
cli.azure.cli.core.sdk.policies: 'Accept': 'application/json'
cli.azure.cli.core.sdk.policies: 'Content-Length': '656'
cli.azure.cli.core.sdk.policies: 'x-ms-client-request-id': '978604be-aa75-11ee-8474-a4ae1284d41e'
cli.azure.cli.core.sdk.policies: 'CommandName': 'network manager connect-config create'
cli.azure.cli.core.sdk.policies: 'ParameterSetName': '--configuration-name --description --applies-to-groups --connectivity-topology --delete-existing-peering --hub --is-global --network-manager-name --resource-group --debug'
cli.azure.cli.core.sdk.policies: 'User-Agent': 'AZURECLI/2.55.0 (MSI) (AAZ) azsdk-python-core/1.26.0 Python/3.11.5 (Windows-10-10.0.22621-SP0)'
cli.azure.cli.core.sdk.policies: 'Authorization': '*****'
cli.azure.cli.core.sdk.policies: Request body:
cli.azure.cli.core.sdk.policies: {"properties": {"appliesToGroups": [{"groupConnectivity": "None", "isGlobal": "false", "networkGroupId": "/subscriptions/f0dc2b34-dfad-40e4-83e0-2309fed8d00b/resourceGroups/jaredgorthy-testResources/providers/Microsoft.Network/networkManagers/jaredgorthy/networkGroups/test", "useHubGateway": "false"}], "connectivityTopology": "HubAndSpoke", "deleteExistingPeering": "False", "description": "hellow world", "hubs": [{"resourceId": "/subscriptions/b7a6ea8b-8aac-46e2-9cd9-0d22ff8778bc/resourceGroups/JaredTestRG/providers/Microsoft.Network/virtualNetworks/testGroupMemberPublish", "resourceType": "Microsoft.Network/virtualNetworks"}], "isGlobal": "True"}}
urllib3.connectionpool: Starting new HTTPS connection (1): management.azure.com:443
urllib3.connectionpool: https://management.azure.com:443 "PUT /subscriptions/f0dc2b34-dfad-40e4-83e0-2309fed8d00b/resourceGroups/jaredgorthy-testResources/providers/Microsoft.Network/networkManagers/jaredgorthy/connectivityConfigurations/testCrossTenantFeature?api-version=2022-01-01 HTTP/1.1" 403 509
cli.azure.cli.core.sdk.policies: Response status: 403
cli.azure.cli.core.sdk.policies: Response headers:
cli.azure.cli.core.sdk.policies: 'Cache-Control': 'no-cache'
cli.azure.cli.core.sdk.policies: 'Pragma': 'no-cache'
cli.azure.cli.core.sdk.policies: 'Content-Type': 'application/json; charset=utf-8'
cli.azure.cli.core.sdk.policies: 'Expires': '-1'
cli.azure.cli.core.sdk.policies: 'x-ms-failure-cause': 'gateway'
cli.azure.cli.core.sdk.policies: 'x-ms-request-id': '83636791-3520-4ba2-acb1-7cf5d357f4df'
cli.azure.cli.core.sdk.policies: 'x-ms-correlation-request-id': '83636791-3520-4ba2-acb1-7cf5d357f4df'
cli.azure.cli.core.sdk.policies: 'x-ms-routing-request-id': 'WESTUS:20240103T202055Z:83636791-3520-4ba2-acb1-7cf5d357f4df'
cli.azure.cli.core.sdk.policies: 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains'
cli.azure.cli.core.sdk.policies: 'X-Content-Type-Options': 'nosniff'
cli.azure.cli.core.sdk.policies: 'Date': 'Wed, 03 Jan 2024 20:20:55 GMT'
cli.azure.cli.core.sdk.policies: 'Connection': 'close'
cli.azure.cli.core.sdk.policies: 'Content-Length': '509'
cli.azure.cli.core.sdk.policies: Response content:
cli.azure.cli.core.sdk.policies: {"error":{"code":"LinkedAuthorizationFailed","message":"The client has permission to perform action 'Microsoft.Network/virtualNetworks/write' on scope '/subscriptions/f0dc2b34-dfad-40e4-83e0-2309fed8d00b/resourceGroups/jaredgorthy-testResources/providers/Microsoft.Network/networkManagers/jaredgorthy/connectivityConfigurations/testCrossTenantFeature', however the current tenant '72f988bf-86f1-41af-91ab-2d7cd011db47' is not authorized to access linked subscription 'b7a6ea8b-8aac-46e2-9cd9-0d22ff8778bc'."}}
cli.azure.cli.core.azclierror: Traceback (most recent call last):
File "D:\a_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\knack/cli.py", line 233, in invoke
File "D:\a_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/init.py", line 663, in execute
File "D:\a_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/init.py", line 726, in _run_jobs_serially
File "D:\a_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/init.py", line 697, in _run_job
File "D:\a_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/init.py", line 333, in call
File "D:\a_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/command_operation.py", line 121, in handler
File "C:\Users\jaredgorthy.azure\cliextensions\virtual-network-manager\azext_network_manager\custom.py", line 94, in network_manager_connect_config_create
return _ConnectConfigCreate(cli_ctx=cmd.cli_ctx)(command_args=connectivity_configuration)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\a_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/aaz/_command.py", line 155, in call
File "C:\Users\jaredgorthy.azure\cliextensions\virtual-network-manager\azext_network_manager\aaz\latest\network\manager\connect_config_create.py", line 33, in _handler
self._execute_operations()
File "C:\Users\jaredgorthy.azure\cliextensions\virtual-network-manager\azext_network_manager\aaz\latest\network\manager\connect_config_create.py", line 140, in _execute_operations
self.ConnectivityConfigurationsCreateOrUpdate(ctx=self.ctx)()
File "C:\Users\jaredgorthy.azure\cliextensions\virtual-network-manager\azext_network_manager\aaz\latest\network\manager\connect_config_create.py", line 164, in call
return self.on_error(session.http_response)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\a_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/aaz/_operation.py", line 332, in on_error
azure.core.exceptions.HttpResponseError: (LinkedAuthorizationFailed) The client has permission to perform action 'Microsoft.Network/virtualNetworks/write' on scope '/subscriptions/f0dc2b34-dfad-40e4-83e0-2309fed8d00b/resourceGroups/jaredgorthy-testResources/providers/Microsoft.Network/networkManagers/jaredgorthy/connectivityConfigurations/testCrossTenantFeature', however the current tenant '72f988bf-86f1-41af-91ab-2d7cd011db47' is not authorized to access linked subscription 'b7a6ea8b-8aac-46e2-9cd9-0d22ff8778bc'.
Code: LinkedAuthorizationFailed
Message: The client has permission to perform action 'Microsoft.Network/virtualNetworks/write' on scope '/subscriptions/f0dc2b34-dfad-40e4-83e0-2309fed8d00b/resourceGroups/jaredgorthy-testResources/providers/Microsoft.Network/networkManagers/jaredgorthy/connectivityConfigurations/testCrossTenantFeature', however the current tenant '72f988bf-86f1-41af-91ab-2d7cd011db47' is not authorized to access linked subscription 'b7a6ea8b-8aac-46e2-9cd9-0d22ff8778bc'.
cli.azure.cli.core.azclierror: (LinkedAuthorizationFailed) The client has permission to perform action 'Microsoft.Network/virtualNetworks/write' on scope '/subscriptions/f0dc2b34-dfad-40e4-83e0-2309fed8d00b/resourceGroups/jaredgorthy-testResources/providers/Microsoft.Network/networkManagers/jaredgorthy/connectivityConfigurations/testCrossTenantFeature', however the current tenant '72f988bf-86f1-41af-91ab-2d7cd011db47' is not authorized to access linked subscription 'b7a6ea8b-8aac-46e2-9cd9-0d22ff8778bc'.
Code: LinkedAuthorizationFailed
Message: The client has permission to perform action 'Microsoft.Network/virtualNetworks/write' on scope '/subscriptions/f0dc2b34-dfad-40e4-83e0-2309fed8d00b/resourceGroups/jaredgorthy-testResources/providers/Microsoft.Network/networkManagers/jaredgorthy/connectivityConfigurations/testCrossTenantFeature', however the current tenant '72f988bf-86f1-41af-91ab-2d7cd011db47' is not authorized to access linked subscription 'b7a6ea8b-8aac-46e2-9cd9-0d22ff8778bc'.
az_command_data_logger: (LinkedAuthorizationFailed) The client has permission to perform action 'Microsoft.Network/virtualNetworks/write' on scope '/subscriptions/f0dc2b34-dfad-40e4-83e0-2309fed8d00b/resourceGroups/jaredgorthy-testResources/providers/Microsoft.Network/networkManagers/jaredgorthy/connectivityConfigurations/testCrossTenantFeature', however the current tenant '72f988bf-86f1-41af-91ab-2d7cd011db47' is not authorized to access linked subscription 'b7a6ea8b-8aac-46e2-9cd9-0d22ff8778bc'.
Code: LinkedAuthorizationFailed
Message: The client has permission to perform action 'Microsoft.Network/virtualNetworks/write' on scope '/subscriptions/f0dc2b34-dfad-40e4-83e0-2309fed8d00b/resourceGroups/jaredgorthy-testResources/providers/Microsoft.Network/networkManagers/jaredgorthy/connectivityConfigurations/testCrossTenantFeature', however the current tenant '72f988bf-86f1-41af-91ab-2d7cd011db47' is not authorized to access linked subscription 'b7a6ea8b-8aac-46e2-9cd9-0d22ff8778bc'.
cli.knack.cli: Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0x03D0F0C8>]
az_command_data_logger: exit code: 1
cli.main: Command ran in 4.794 seconds (init: 0.987, invoke: 3.807)
telemetry.main: Begin splitting cli events and extra events, total events: 1
telemetry.client: Accumulated 0 events. Flush the clients.
telemetry.main: Finish splitting cli events and extra events, cli events: 1
telemetry.save: Save telemetry record of length 4635 in cache
telemetry.main: Begin creating telemetry upload process.
telemetry.process: Creating upload process: "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\python.exe C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\Lib\site-packages\azure\cli\telemetry_init_.pyc C:\Users\jaredgorthy.azure"
telemetry.process: Return from creating process
telemetry.main: Finish creating telemetry upload process.
Expected behavior
I expect the command to recognize the '--hub resource-id="..."' is a resource from another tenant, and automatically fetch the token for that tenant behind the scenes. The token should then be added via an auxiliary header to ensure the linked access check passes.
Note, your team has already implemented this for 'az network manager group static-member create', since this can also contain references to resources in another tenant.
Environment Summary
C:\Users\jaredgorthy\OneDrive - Microsoft\Desktop>az --version
azure-cli 2.55.0
core 2.55.0
telemetry 1.1.0
Extensions:
account 0.2.5
azure-devops 0.26.0
virtual-network-manager 1.0.0
Dependencies:
msal 1.24.0b2
azure-mgmt-resource 23.1.0b2
Python location 'C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\python.exe'
Extensions directory 'C:\Users\jaredgorthy.azure\cliextensions'
Development extension sources:
C:\CLI_test\azure-cli-extensions
Python (Windows) 3.11.5 (tags/v3.11.5:cce6ba9, Aug 24 2023, 14:21:31) [MSC v.1936 32 bit (Intel)]
Legal docs and information: aka.ms/AzureCliLegal
Your CLI is up-to-date.
C:\Users\jaredgorthy\OneDrive - Microsoft\Desktop>az extension update --name virtual-network-manager
Latest version of 'virtual-network-manager' is already installed.
Additional context
This is blocking an s500 customer, please address asap.
Thank you for opening this issue, we will look into it.
Hi @jbgorthy, for the --hub param there is also the option to select resource-type="Microsoft.Network/virtualNetworks". Will it be changed in the future to allow other resource-types? Now the resource id will be only for vnets right.
@calvinhzy correct, right now we only support virtual networks. In the future we will support more resource types, which is why we have left it as a string. However it will always be a well-formatted ARM ID.
Hi @jbgorthy, please give the new version v1.0.1 a try. Can install with az extension add -n virtual-network-manager --upgrade