Skip to content

Fix #60269: Replace non-POSIX 'source' with '.' in EKS hook#61441

Open
idrisakorede wants to merge 3 commits intoapache:mainfrom
idrisakorede:fix-60269-eks-posix-shell
Open

Fix #60269: Replace non-POSIX 'source' with '.' in EKS hook#61441
idrisakorede wants to merge 3 commits intoapache:mainfrom
idrisakorede:fix-60269-eks-posix-shell

Conversation

@idrisakorede
Copy link

Fix #60269: Replace non-POSIX 'source' with '.' in EKS hook

Problem

EksPodOperator fails with 401 Unauthorized errors when running on Debian/Ubuntu-based containers (Astronomer Runtime, official Airflow Docker images, MWAA, etc.).

Root Cause

The issue occurs in airflow/providers/amazon/aws/hooks/eks.py line 83, where the COMMAND template uses source:

source {credentials_file}

The problem: source is a bash-specific builtin command, not a POSIX standard command. On Debian/Ubuntu systems, /bin/sh is symlinked to dash (not bash), which doesn't recognize source:

$ sh -c 'source /dev/null'
sh: 1: source: not found

This causes the credential loading to fail silently, resulting in 401 Unauthorized errors when the EKS token generation falls back to an empty credential chain.

Why This Is Hard to Detect

The bug is masked during local development when developers have ~/.aws/credentials mounted in containers:

  1. source {credentials_file} fails silently (stderr not checked)
  2. eks_get_token.py falls back to boto3's default credential chain
  3. Finds credentials in ~/.aws/credentials → token generation succeeds ✅

In production/cloud environments without ~/.aws/ directory, credentials are only available via the temp file that failed to source, causing 401 errors ❌

Solution

This PR implements two fixes:

1. Use POSIX-Compliant Dot Operator

Replace source with . (dot operator), which is POSIX-compliant and works in all shells (bash, dash, sh):

# Before (bash-specific)
source {credentials_file}

# After (POSIX-compliant)
. {credentials_file}

2. Update Deprecated Kubernetes API Version

Update the authentication API version from deprecated v1alpha1 to v1beta1:

# Before
AUTHENTICATION_API_VERSION = "client.authentication.k8s.io/v1alpha1"

# After
AUTHENTICATION_API_VERSION = "client.authentication.k8s.io/v1beta1"

Note: v1alpha1 was deprecated in Kubernetes 1.24 and removed in 1.28.

Changes Made

  • Replace 'source' with POSIX-compliant '.' operator in COMMAND template
  • Update deprecated v1alpha1 to v1beta1 Kubernetes API version
  • Add shell compatibility tests for dash/POSIX shells
  • Update comment to reflect POSIX compliance

Testing

Added comprehensive test coverage in test_eks.py:

New Test Classes

  1. TestEksHookShellCompatibility

    • test_command_template_is_posix_compliant: Verifies the template uses . not source
    • test_credential_loading_works_with_dash: Confirms credentials load correctly with dash shell
    • test_source_command_fails_with_dash: Documents the original bug
  2. TestEksHookKubernetesVersion

    • test_uses_stable_kubernetes_api_version: Ensures we're not using deprecated v1alpha1

Manual Testing

Verified in Breeze (Debian-based container):

$ ls -la /bin/sh
/bin/sh -> dash

$ sh -c '. /tmp/test.sh && echo $AWS_ACCESS_KEY_ID'
test_key  # ✅ Works

$ sh -c 'source /tmp/test.sh && echo $AWS_ACCESS_KEY_ID'
sh: 1: source: not found  # ❌ Fails

Impact

This fix resolves 401 Unauthorized errors for all Debian/Ubuntu-based Airflow deployments:

  • ✅ Astronomer Runtime
  • ✅ Official Apache Airflow Docker images
  • ✅ Amazon MWAA (Managed Workflows for Apache Airflow)
  • ✅ Any deployment where /bin/sh is dash

The change is backward-compatible as the . operator works in both bash and dash shells.


Fixes #60269


Was generative AI tooling used to co-author this PR?
  • Yes (Claude AI was used for documentation)

@boring-cyborg
Copy link

boring-cyborg bot commented Feb 4, 2026

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:

  • Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
  • Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
  • Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: dev@airflow.apache.org
    Slack: https://s.apache.org/airflow-slack

@boring-cyborg boring-cyborg bot added area:providers provider:amazon AWS/Amazon - related issues labels Feb 4, 2026
@idrisakorede idrisakorede force-pushed the fix-60269-eks-posix-shell branch from f982627 to 3adc5a2 Compare February 4, 2026 10:00
@vincbeck
Copy link
Contributor

vincbeck commented Feb 4, 2026

I dont think I talk to a human but the code looks good

@idrisakorede
Copy link
Author

I dont think I talk to a human but the code looks good

Thanks. But i don't get the human reference

- Replace 'source' with POSIX-compliant '.' operator in COMMAND template
- Update deprecated v1alpha1 to v1beta1 Kubernetes API version
- Add shell compatibility tests for dash/POSIX shells
- Update comment to reflect POSIX compliance

Fixes apache#60269
Reduced to single assertion test as suggested by @vincbeck
@idrisakorede idrisakorede force-pushed the fix-60269-eks-posix-shell branch from f89341c to cbc5481 Compare February 4, 2026 20:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers provider:amazon AWS/Amazon - related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

EksPodOperator fails on POSIX shells (dash) due to non-portable 'source' command

2 participants