Want to authenticate in synapse spark pool using managed identity.
dbeavon opened this issue · comments
I'm looking for a way to authenticate within the synapse spark pool, using the MSI.
Managed Identity Support is already available in a variety of other Azure platforms via the nuget called Azure.Identity. Here is a list of products that support "DefaultAzureCredential":
https://www.nuget.org/packages/Azure.Identity#readme-body-tab
- Azure Virtual Machines
- Azure App Service and Azure Functions
- Azure Kubernetes Service
- etc
Whenever this is supported, it becomes possible to create an MSAL access token for a list of scopes WITHOUT providing an additional service principal or secret. Here is an example of authentication to Azure Analysis Services:
using Azure.Core;
using Azure.Identity;
using Microsoft.Identity.Client;
private static Azure.Core.AccessToken GetAzureAccessTokenFromDefaultCredential() { TokenCredential p_CurrentTokenCredential = null; string[] p_ScopesList = null; // Retrieve the current credentials (for the app service) p_CurrentTokenCredential = new DefaultAzureCredential(includeInteractiveCredentials: true); // Special-purpose scope used for // application permissions p_ScopesList = new string[] { "https://*.asazure.windows.net/.default" }; // The context and the resulting token. var ContextObj = new TokenRequestContext(p_ScopesList); Azure.Core.AccessToken TokenObj = p_CurrentTokenCredential.GetToken(ContextObj, CancellationToken.None); return TokenObj; }
Is there any way to accomplish the same thing in synapse spark pools today? I had found the "TokenLibrary" but it seemed extremely limited and I found no way of creating an access token for an arbitrary list of scopes.
Any tips or pointers would be very much appreciated.
@dbeavon did you have any luck with this?
Are you writing application/sparkjobs/notebooks? I know MSI is fully supported from a synapse perspective but directly on the spark pool I am not sure?
Synapse will allow you to get an MSI token, although 99% of the time it is not needed as it is automatic.
MSSparkUtils.Credentials.GetConnectionStringOrCreds()
Outside of setting the MSI IAM on a Synapse workspace I struggle to think of a way too directly authenticate with a spark pool as it does not represent itself in that manner inside AAD applications