kubernetes / kubernetes

/kind bug

What happened:
The K8S token checker does not support resolving distributed claims for (among others) group claims.

What you expected to happen:
It should be possible for the K8S token checker to resolve distributed claims if the IDP provides them.

How to reproduce it (as minimally and precisely as possible):
One can inspect the token checker source code to the token checker to determine that there currently is no special handling of distributed claims. Distributed claims come in very handy when the claim content is fairly large, as is the case e.g. for groups claims in some existing IDP implementations.

Anything else we need to know?:
I have a proof of concept commit that adds distributed claims resolution. It doesn't support multiple issues which is probably something that needs to be addressed, but I'm willing to take that work on.

/sig auth
@mikedanese

cc @kubernetes/sig-auth-feature-requests

@ericchiang @rithujohn191

Relevant spec link: http://openid.net/specs/openid-connect-core-1_0.html#AggregatedDistributedClaims

This would be incredibly useful for getting group claims out of ID Token without having to implement your own token validator.

What's the API server's obligation to validate the JWT returned by the distributed claim endpoint? There doesn't seem to be anything in the spec about that (other than the JWT not having a sub claim). Do we just ignore the signature?

We definitely need some caching of the endpoint response.

What's the TLS story? Can we assume the distributed endpoint is validated by the same CA bundle as the OpenID Connect provider?

Do any providers actually implement this? Not against adding it if they don't, but it'd help validate the concept.

What's the API server's obligation to validate the JWT returned by the distributed claim endpoint? There doesn't seem to be anything in the spec about that (other than the JWT not having a sub claim). Do we just ignore the signature?

JWTs are validated per https://tools.ietf.org/html/rfc7519#section-7.2. We can lookup validation keys by keyid, or "iss".

What's the TLS story? Can we assume the distributed endpoint is validated by the same CA bundle as the OpenID Connect provider?

That's how I would do it.

Do any providers actually implement this? Not against adding it if they don't, but it'd help validate the concept.

AD supports using distributed claims with groups. I'm not sure if they are totally up to spec.

https://docs.microsoft.com/en-us/azure/active-directory/develop/active-directory-token-and-claims

JWTs are validated per https://tools.ietf.org/html/rfc7519#section-7.2. We can lookup validation keys by keyid, or "iss".

Okay, if we require the JWTs to be issued by a provider that supports OpenID Connect discovery, then that makes it easy.

Hi, folks. I've tried to collect your remarks into a consistent pull request. It's now created for review.

@filmil
@ericchiang

K8S server version:

kubectl version
Client Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.2", GitCommit:"bb9ffb1654d4a729bb4cec18ff088eacc153c239", GitTreeState:"clean", BuildDate:"2018-08-07T23:17:28Z", GoVersion:"go1.10.3", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.2", GitCommit:"bb9ffb1654d4a729bb4cec18ff088eacc153c239", GitTreeState:"clean", BuildDate:"2018-08-07T23:08:19Z", GoVersion:"go1.10.3", Compiler:"gc", Platform:"linux/amd64"}

Getting this error with Azure OIDC token with distributed claims:

authentication.go:62] Unable to authenticate the request due to an error: 
[invalid bearer token, 
[invalid bearer token, oidc: could not expand distributed claims: while getting distributed claim "groups": error while getting distributed claim JWT: 400 Bad Request]]

It seems like Azure Graph REST API returns non-200 HTTP status. I think it is because of invalid “access token”. I’m not sure whether we are passing “access_token” or “id_token”

Azure doc says:
https://docs.microsoft.com/en-us/azure/active-directory/develop/v2-oauth2-on-behalf-of-flow#client-limitations

“Client limitations
If a client uses the implicit flow to get an id_token, and that client also has wildcards in a reply URL, the id_token cannot be used for an OBO flow. However, access tokens acquired via the implicit grant flow can still be redeemed by a confidential client even if the initiating client has a wildcard reply URL registered.
“

Appreciate any help to fix this issue.

It seems like Azure Graph REST API returns non-200 HTTP status. I think it is because of invalid “access token”. I’m not sure whether we are passing “access_token” or “id_token”

The only credential passed when requesting the distributed claim is the access_token embedded in the distributed claim endpoint configuration:

kubernetes/staging/src/k8s.io/apiserver/plugin/pkg/authenticator/token/oidc/oidc.go

Lines 514 to 517 in 527d1c3

    
           func (r *claimResolver) resolve(endpoint endpoint, allClaims claims) error { 
        
           	// TODO: cache resolved claims. 
        
           	jwt, err := getClaimJWT(r.client, endpoint.URL, endpoint.AccessToken) 
        
           	if err != nil {

From https://openid.net/specs/openid-connect-core-1_0.html#AggregatedDistributedClaims:

Distributed Claims
JSON object that contains the following members and values:

endpoint

REQUIRED. OAuth 2.0 resource endpoint from which the associated Claim can be retrieved. The endpoint URL MUST return the Claim as a JWT.

access_token

OPTIONAL. Access Token enabling retrieval of the Claims from the endpoint URL by using the OAuth 2.0 Bearer Token Usage [RFC6750] protocol. Claims SHOULD be requested using the Authorization Request header field and Claims Providers MUST support this method.

thanks @liggitt. It is helpful.

Digging a little deep, it seems like Azure oidc token with '_claim_sources' doesn't work with the current Kubernetes implementation because:

The graph API URL embedded in the oidc token doesn't have a parameter - api-version=1.6, required by the graph server (hence we are getting 400 Bad Request error)
The Azure oidc token doesn't have access_token embedded, but required by the graph server. Kubernetes implementation expects access_token in the given oidc token.

  "_claim_names": {
    "groups": "src1"
  },
  "_claim_sources": {
    "src1": {
      "endpoint": "https://graph.windows.net/xxxxxxxx-xxxxxxx/users/xxxxxxxx-xxxxxxx/getMemberObjects"
    }
  }, ...

I think the Azure OIDC support is broken from a protocol perspective (or are there any hidden doc
that says how to fix it?).

I think the Azure OIDC support is broken from a protocol perspective

It would appear so, yes:

out of spec: the specified URL is invalid (missing query parameter)
within spec, but optional and not supported by kubernetes: no access_token available (allowed by OIDC spec but optional and requires out of band support by RPs, which Kubernetes doesn't support)
question: assuming the token and query parameter issues were resolved, does a GET request to that URL return results structured as a JWT?

question: assuming the token and query parameter issues were resolved, does a GET request to that URL return results structured as a JWT?

Not tested, but based on the doc below, it returns a JSON struct, not JWT.
https://docs.microsoft.com/en-us/graph/api/user-getmemberobjects?view=graph-rest-1.0

based on the doc below, it returns a JSON struct, not JWT.
https://docs.microsoft.com/en-us/graph/api/user-getmemberobjects?view=graph-rest-1.0

If so, that is completely incompatible with the oidc spec. From https://openid.net/specs/openid-connect-core-1_0.html#AggregatedDistributedClaims:

Distributed Claims
JSON object that contains the following members and values:

endpoint

REQUIRED. OAuth 2.0 resource endpoint from which the associated Claim can be retrieved. The endpoint URL MUST return the Claim as a JWT

Kind of stuck now with Azure OIDC.

thank you @liggitt for helping us identifying the issue

The Azure folks are well aware of this non-compliance. The v2 Azure OIDC implementation aims to be fully complaint with the OIDC spec. They should have proper support for distributed claims on the v2 endpoints in the near future.

See https://docs.microsoft.com/en-us/azure/active-directory/develop/azure-ad-endpoint-comparison

This issue is closed - but is there an open issue tracking progress until aad login works? I would also call this more of a bug than a feature request because "aad login" is a shipping feature which can't be used right now.

is there an open issue tracking progress until aad login works?

Not really, since the distributed claim incompatibility is on the azure side. I'm not aware of anything actionable on the kubernetes side to track with an issue.

Hmm... It looks like using the v2 Azure OIDC endpoint means changing to the msal library instead of adal - does that meant there are changes which could be started in kubectl to update --auth-provider=azure behavior?

Or is the theory that v2 Azure OIDC authentication would be configured with --auth-provider=oidc instead of --auth-provider=azure?

Hmm... It looks like using the v2 Azure OIDC endpoint means changing to the msal library instead of adal - does that meant there are changes which could be started in kubectl to update --auth-provider=azure behavior?

I'm not sure. Will defer to @kubernetes/sig-azure for that question. I didn't realize use of a new endpoint in azure was required to get the oidc-compatible behavior.

	func (r *claimResolver) resolve(endpoint endpoint, allClaims claims) error {
	// TODO: cache resolved claims.
	jwt, err := getClaimJWT(r.client, endpoint.URL, endpoint.AccessToken)
	if err != nil {

Support OIDC distributed claims for group resolution in the K8S apiserver OIDC token checker