Skip to content

fix: Support Entra ID (Azure AD) token claims in OIDC auth - #6631

Merged
ntkathole merged 7 commits into
feast-dev:masterfrom
larrysingleton007:fix/6630-entra-id-oidc-claims
Jul 23, 2026
Merged

fix: Support Entra ID (Azure AD) token claims in OIDC auth#6631
ntkathole merged 7 commits into
feast-dev:masterfrom
larrysingleton007:fix/6630-entra-id-oidc-claims

Conversation

@larrysingleton007

Copy link
Copy Markdown
Contributor

What this PR does / why we need it:

The OIDC token parser is written against Keycloak's token shape, and that breaks Microsoft Entra ID (Azure AD) in two independent places.

Username extraction accepted only preferred_username or upn. Entra client-credentials (app-only) tokens carry neither, so every machine-to-machine caller was rejected at authentication. This falls back to the calling application's own identity when no human claim is present: azp on v2 tokens, appid on v1, then sub, which every issuer sets. A claim that is present but null or non-string is skipped rather than returned, and a token providing none of the five claims as a string still raises AuthenticationError.

Roles were read only from Keycloak's nested resource_access.<client_id>.roles. Entra puts app roles in the top-level roles claim, so the extracted list was always empty and RoleBasedPolicy never matched. This merges the top-level claim into the existing extraction, preserving order and dropping duplicates.

Keycloak's default behavior is unchanged: the username fallbacks only fire when preferred_username and upn are both absent, the merge only adds roles, and token validation (signature and expiry) is untouched. One behavior change worth calling out for existing deployments: role resolution now also reads the top-level roles claim for every issuer, not only Entra. An installation whose IdP already emits a top-level roles claim (for example a Keycloak realm with a custom mapper) will see those roles merged into the user's Feast roles. The Keycloak resource_access.<client_id>.roles path itself is unchanged.

We run this exact change against an Entra tenant in production as a build-time patch on 0.64.

Which issue(s) this PR fixes:

Fixes #6630. Supersedes #4934, which requested the roles half and was closed stale.

Checks

  • I've made sure the tests are passing.
  • My commits are signed off (git commit -s)
  • My PR title follows conventional commits format

Testing Strategy

  • Unit tests
  • Integration tests
  • Manual tests
  • Testing is not required for this change

New unit tests in sdk/python/tests/unit/permissions/auth/test_token_parser.py cover: preferred_username and upn precedence over the application claims (including directly against appid/sub); azp-only, appid-only, and sub-only tokens each authenticating with that value; tokens whose only identity claims are null or non-string still raising; top-level roles extracted on their own; Keycloak resource_access roles still working; the de-duplicated merge when both shapes are present, including duplicates within the top-level claim; and that identity and roles are read from the signature-verified decode rather than the unverified routing decode.

Misc

Release note: OIDC authentication now accepts Microsoft Entra ID (Azure AD) tokens. The username falls back to azp, appid, or sub for client-credentials (app-only) tokens, and app roles in the top-level roles claim are merged with Keycloak's resource_access.<client_id>.roles.

The PR template here does not have an auto-generated release-note field, so the note is inline above. I can't apply labels as an outside contributor, so a maintainer will need to add kind/bug (and ok-to-test for CI).

The last two test: commits are self-contained (a fidelity test plus a fixture extraction) and can be dropped without affecting the fix if a smaller diff is preferred.

@larrysingleton007
larrysingleton007 requested a review from a team as a code owner July 22, 2026 19:31
@larrysingleton007

Copy link
Copy Markdown
Contributor Author

I validated this against real Microsoft Entra ID tokens from a production tenant, running them through the parser and a RoleBasedPolicy check.

Confirmed:

  • v1.0 client-credentials (app-only) tokens carry appid, with no azp and no preferred_username/upn, put app roles in a top-level roles array, and have no resource_access. This is the shape the unpatched parser rejects at username extraction. With this PR the username resolves to appid and the top-level roles merge in.
  • v2.0 app-only tokens are the same except the caller identity is in azp rather than appid, which exercises that fallback.
  • Delegated user tokens carry preferred_username and keep using the existing path unchanged.
  • RoleBasedPolicy matches on the top-level role string, and a policy requiring a role the token lacks correctly denies.

One interop detail others may hit: it is valid to set auth_discovery_url to the v2.0 endpoint while the tenant still issues v1.0 tokens (iss: https://sts.windows.net/...). It works because the parser uses discovery only to fetch JWKS and Entra shares signing keys across v1 and v2, and because it runs with verify_aud=False and no issuer check. If stricter iss/aud validation is added later, it would need to allow v1.0 tokens against a v2.0 discovery document.

@larrysingleton007

Copy link
Copy Markdown
Contributor Author

/kind bug
/cc @ntkathole @franciscojavierarceo

Small, self-contained fix for a filed issue (#6630): the OIDC parser now accepts Microsoft Entra ID (Azure AD) tokens, with a username fallback for app-only tokens and support for the top-level roles claim. The diff is the two parser changes plus tests and a docs note, and unit tests pass.

As an outside contributor I can't self-label or start CI. If the /kind bug above doesn't take, could a maintainer add the label and run ok-to-test when convenient?

@codecov-commenter

codecov-commenter commented Jul 23, 2026

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 45.88%. Comparing base (0399380) to head (4b4c564).
⚠️ Report is 1 commits behind head on master.
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #6631      +/-   ##
==========================================
+ Coverage   45.86%   45.88%   +0.01%     
==========================================
  Files         414      414              
  Lines       49948    49950       +2     
  Branches     7140     7140              
==========================================
+ Hits        22911    22918       +7     
+ Misses      25435    25431       -4     
+ Partials     1602     1601       -1     
Flag Coverage Δ
go-feature-server 30.58% <ø> (ø)
python-unit 47.14% <100.00%> (+0.01%) ⬆️
Files with missing lines Coverage Δ
...python/feast/permissions/auth/oidc_token_parser.py 65.89% <100.00%> (+4.47%) ⬆️

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0399380...4b4c564. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@ntkathole

Copy link
Copy Markdown
Member

@aniketpalu Can you please take a look ?

@larrysingleton007

Copy link
Copy Markdown
Contributor Author

Heads up on the red unit-test-python (3.12, ubuntu-latest) check here: it is a pre-existing flaky collection error in test_mcp_server.py, a KeyError importing feast.infra.mcp_servers.mcp_server under parallel collection. It is unrelated to the changes in this PR. I addressed it separately at its root in #6634 (issue #6633) to clear the blocker. The tests in this PR pass on the other CI matrices and locally.

Comment thread sdk/python/tests/unit/permissions/auth/conftest.py Outdated
Comment thread docs/getting-started/components/authz_manager.md Outdated
Comment thread sdk/python/feast/permissions/auth/oidc_token_parser.py

@aniketpalu aniketpalu left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR looks good. Few nitpicks. Thanks for the important contribution @larrysingleton007

@larrysingleton007

Copy link
Copy Markdown
Contributor Author

Thanks for the review @aniketpalu. All three suggestions are applied in 25c364f: typed the discovery_data fixture as Dict[str, str], split the Entra group-claim caveat into its own bullet, and added the debug log of the token's claim keys before the missing-username raise. Threads resolved, ready for another look when you have a moment.

@aniketpalu

Copy link
Copy Markdown
Contributor

Thanks @larrysingleton007, LGTM. Please rebase.

@ntkathole
ntkathole force-pushed the fix/6630-entra-id-oidc-claims branch from 25c364f to e185cf0 Compare July 23, 2026 15:01
@larrysingleton007
larrysingleton007 force-pushed the fix/6630-entra-id-oidc-claims branch from e185cf0 to 1141060 Compare July 23, 2026 16:02
@larrysingleton007

Copy link
Copy Markdown
Contributor Author

Rebased onto the latest master (clean, no conflicts). Thanks for the review @aniketpalu.

The OIDC token parser was written against Keycloak's token shape, which
rejected Microsoft Entra ID tokens in two independent ways.

Username extraction accepted only preferred_username or upn. Entra
client-credentials (app-only) tokens carry neither, so every
machine-to-machine caller failed authentication. Fall back to the calling
application's own identity when no human claim is present: azp on v2
tokens, appid on v1, then sub, which every issuer sets. A token with none
of the five claims still raises AuthenticationError.

Roles were read only from Keycloak's nested
resource_access.<client_id>.roles. Entra emits app roles in the top-level
roles claim, so the extracted list was always empty and RoleBasedPolicy
could never match. Merge the top-level claim into the existing
extraction, preserving order and dropping duplicates.

Both changes are additive, so Keycloak behavior is unchanged: the
username fallbacks only fire when preferred_username and upn are both
absent, and the merge only adds roles. Token validation is untouched.

Signed-off-by: Larry Singleton <166439969+larrysingleton007@users.noreply.github.com>
The OIDC assumptions listed for the auth manager described only
Keycloak's token shape. Record that roles are also read from the
top-level roles claim and merged, and that the username falls back
through upn, azp, appid and sub when preferred_username is absent.

Signed-off-by: Larry Singleton <166439969+larrysingleton007@users.noreply.github.com>
Refine the initial Entra ID support in response to review:

- Merge roles with list(dict.fromkeys(...)), the codebase's order-
  preserving dedup idiom (feast/utils.py), so duplicates within the
  top-level roles claim are also collapsed, not only cross-claim ones.
- Skip username claims whose value is null or non-string instead of
  returning them, so a present-but-null early claim no longer shadows a
  usable later claim; the -> str contract now holds.
- Correct the _extract_username_or_raise_error docstring: the raise
  fires only when a token provides none of the five claims as a string.

Extend the tests: intra-claim role de-duplication, human-claim
precedence over appid/sub, and rejection of tokens whose only identity
claims are null or non-string.

Signed-off-by: Larry Singleton <166439969+larrysingleton007@users.noreply.github.com>
Add an app-only (client-credentials) Entra ID token example alongside
the Keycloak one, and note that Entra emits group object IDs (GUIDs)
rather than names and omits the groups claim under the overage limit,
so GroupBasedPolicy on Entra must reference those IDs.

Signed-off-by: Larry Singleton <166439969+larrysingleton007@users.noreply.github.com>
Add a test that gives the unverified routing decode and the verified
decode different payloads and asserts identity and roles come from the
verified one, so a regression that read claims from the unverified
decode would be caught.

Drop the has_matching_role assertions in the roles-merge test: the exact
roles-list equality already pins the result, and has_matching_role has
its own coverage in test_user.py.

Signed-off-by: Larry Singleton <166439969+larrysingleton007@users.noreply.github.com>
The OIDC token-parser tests each re-declared the same mock discovery
document and JWKS signing key. Move both into conftest.py fixtures and
have the tests consume them, removing the repeated setup blocks.

Signed-off-by: Larry Singleton <166439969+larrysingleton007@users.noreply.github.com>
- Type the discovery_data test fixture as Dict[str, str].
- Split the Entra group-claim caveat into its own bullet in authz_manager.md.
- Log the token's claim keys at debug level before raising on a missing
  username claim, to aid diagnosis (keys only, no values).

Signed-off-by: Larry Singleton <166439969+larrysingleton007@users.noreply.github.com>
@ntkathole
ntkathole force-pushed the fix/6630-entra-id-oidc-claims branch from 1141060 to 4b4c564 Compare July 23, 2026 16:25
@ntkathole
ntkathole merged commit f843c63 into feast-dev:master Jul 23, 2026
11 of 15 checks passed
@larrysingleton007
larrysingleton007 deleted the fix/6630-entra-id-oidc-claims branch July 23, 2026 16:35
aniketpalu pushed a commit to aniketpalu/feast that referenced this pull request Jul 28, 2026
…#6631)

* fix: Support Entra ID (Azure AD) token claims in OIDC auth

The OIDC token parser was written against Keycloak's token shape, which
rejected Microsoft Entra ID tokens in two independent ways.

Username extraction accepted only preferred_username or upn. Entra
client-credentials (app-only) tokens carry neither, so every
machine-to-machine caller failed authentication. Fall back to the calling
application's own identity when no human claim is present: azp on v2
tokens, appid on v1, then sub, which every issuer sets. A token with none
of the five claims still raises AuthenticationError.

Roles were read only from Keycloak's nested
resource_access.<client_id>.roles. Entra emits app roles in the top-level
roles claim, so the extracted list was always empty and RoleBasedPolicy
could never match. Merge the top-level claim into the existing
extraction, preserving order and dropping duplicates.

Both changes are additive, so Keycloak behavior is unchanged: the
username fallbacks only fire when preferred_username and upn are both
absent, and the merge only adds roles. Token validation is untouched.

Signed-off-by: Larry Singleton <166439969+larrysingleton007@users.noreply.github.com>

* docs: Document Entra ID claim support in OIDC authorization

The OIDC assumptions listed for the auth manager described only
Keycloak's token shape. Record that roles are also read from the
top-level roles claim and merged, and that the username falls back
through upn, azp, appid and sub when preferred_username is absent.

Signed-off-by: Larry Singleton <166439969+larrysingleton007@users.noreply.github.com>

* fix: Harden Entra ID OIDC claim handling from review

Refine the initial Entra ID support in response to review:

- Merge roles with list(dict.fromkeys(...)), the codebase's order-
  preserving dedup idiom (feast/utils.py), so duplicates within the
  top-level roles claim are also collapsed, not only cross-claim ones.
- Skip username claims whose value is null or non-string instead of
  returning them, so a present-but-null early claim no longer shadows a
  usable later claim; the -> str contract now holds.
- Correct the _extract_username_or_raise_error docstring: the raise
  fires only when a token provides none of the five claims as a string.

Extend the tests: intra-claim role de-duplication, human-claim
precedence over appid/sub, and rejection of tokens whose only identity
claims are null or non-string.

Signed-off-by: Larry Singleton <166439969+larrysingleton007@users.noreply.github.com>

* docs: Add Entra ID token example and group-claim caveat

Add an app-only (client-credentials) Entra ID token example alongside
the Keycloak one, and note that Entra emits group object IDs (GUIDs)
rather than names and omits the groups claim under the overage limit,
so GroupBasedPolicy on Entra must reference those IDs.

Signed-off-by: Larry Singleton <166439969+larrysingleton007@users.noreply.github.com>

* test: Cover OIDC verified-decode sourcing and trim redundant asserts

Add a test that gives the unverified routing decode and the verified
decode different payloads and asserts identity and roles come from the
verified one, so a regression that read claims from the unverified
decode would be caught.

Drop the has_matching_role assertions in the roles-merge test: the exact
roles-list equality already pins the result, and has_matching_role has
its own coverage in test_user.py.

Signed-off-by: Larry Singleton <166439969+larrysingleton007@users.noreply.github.com>

* test: Extract shared OIDC discovery and signing-key fixtures

The OIDC token-parser tests each re-declared the same mock discovery
document and JWKS signing key. Move both into conftest.py fixtures and
have the tests consume them, removing the repeated setup blocks.

Signed-off-by: Larry Singleton <166439969+larrysingleton007@users.noreply.github.com>

* fix: Address review nitpicks on Entra OIDC PR

- Type the discovery_data test fixture as Dict[str, str].
- Split the Entra group-claim caveat into its own bullet in authz_manager.md.
- Log the token's claim keys at debug level before raising on a missing
  username claim, to aid diagnosis (keys only, no values).

Signed-off-by: Larry Singleton <166439969+larrysingleton007@users.noreply.github.com>

---------

Signed-off-by: Larry Singleton <166439969+larrysingleton007@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

OIDC auth rejects Entra ID (Azure AD) tokens: username claims and top-level app roles unsupported

4 participants