Skip to content

chore: Support "offline" source builds of the feature-server image #5193

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
franciscojavierarceo merged 2 commits into from
Apr 3, 2025
Merged

chore: Support "offline" source builds of the feature-server image #5193

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
fdb664e
support offline source builds for feature-server image
tchughesiv Mar 11, 2025
3e33393
offline build review changes
tchughesiv Apr 2, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .dockerignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,4 @@ docs
charts
env
**/.terraform
offline_build/
16 changes: 5 additions & 11 deletions .github/fork_workflows/fork_pr_integration_tests_aws.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,18 +51,10 @@ jobs:
aws-region: us-west-2
- name: Use AWS CLI
run: aws sts get-caller-identity
- name: Install uv
run: |
curl -LsSf https://astral.sh/uv/install.sh | sh
- name: Get uv cache dir
id: uv-cache
run: |
echo "::set-output name=dir::$(uv cache dir)"
- name: uv cache
uses: actions/cache@v4
- name: Install the latest version of uv
uses: astral-sh/setup-uv@v5
with:
path: ${{ steps.uv-cache.outputs.dir }}
key: ${{ runner.os }}-${{ steps.setup-python.outputs.python-version }}-uv-${{ hashFiles(format('**/py{0}-ci-requirements.txt', env.PYTHON)) }}
enable-cache: true
- name: Install apache-arrow on ubuntu
if: matrix.os == 'ubuntu-latest'
run: |
Expand All @@ -85,3 +77,5 @@ jobs:
pytest -n 8 --cov=./ --cov-report=xml --color=yes sdk/python/tests --integration --durations=5 --timeout=1200 --timeout_method=thread -k "File and not Snowflake and not BigQuery and not minio_registry"
pytest -n 8 --cov=./ --cov-report=xml --color=yes sdk/python/tests --integration --durations=5 --timeout=1200 --timeout_method=thread -k "dynamo and not Snowflake and not BigQuery and not minio_registry"
pytest -n 8 --cov=./ --cov-report=xml --color=yes sdk/python/tests --integration --durations=5 --timeout=1200 --timeout_method=thread -k "Redshift and not Snowflake and not BigQuery and not minio_registry"
- name: Minimize uv cache
run: uv cache prune --ci
16 changes: 5 additions & 11 deletions .github/fork_workflows/fork_pr_integration_tests_gcp.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,18 +53,10 @@ jobs:
project_id: ${{ secrets.GCP_PROJECT_ID }}
- name: Use gcloud CLI
run: gcloud info
- name: Install uv
run: |
curl -LsSf https://astral.sh/uv/install.sh | sh
- name: Get uv cache dir
id: uv-cache
run: |
echo "::set-output name=dir::$(uv cache dir)"
- name: uv cache
uses: actions/cache@v4
- name: Install the latest version of uv
uses: astral-sh/setup-uv@v5
with:
path: ${{ steps.uv-cache.outputs.dir }}
key: ${{ runner.os }}-${{ steps.setup-python.outputs.python-version }}-uv-${{ hashFiles(format('**/py{0}-ci-requirements.txt', env.PYTHON)) }}
enable-cache: true
- name: Install apache-arrow on ubuntu
if: matrix.os == 'ubuntu-latest'
run: |
Expand All @@ -86,3 +78,5 @@ jobs:
run: |
pytest -n 8 --cov=./ --cov-report=xml --color=yes sdk/python/tests --integration --durations=5 --timeout=1200 --timeout_method=thread -k "BigQuery and not dynamo and not Redshift and not Snowflake and not minio_registry"
pytest -n 8 --cov=./ --cov-report=xml --color=yes sdk/python/tests --integration --durations=5 --timeout=1200 --timeout_method=thread -k "File and not dynamo and not Redshift and not Snowflake and not minio_registry"
- name: Minimize uv cache
run: uv cache prune --ci
16 changes: 5 additions & 11 deletions .github/fork_workflows/fork_pr_integration_tests_snowflake.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,18 +43,10 @@ jobs:
uses: actions/setup-go@v2
with:
go-version: 1.18.0
- name: Install uv
run: |
curl -LsSf https://astral.sh/uv/install.sh | sh
- name: Get uv cache dir
id: uv-cache
run: |
echo "::set-output name=dir::$(uv cache dir)"
- name: uv cache
uses: actions/cache@v4
- name: Install the latest version of uv
uses: astral-sh/setup-uv@v5
with:
path: ${{ steps.uv-cache.outputs.dir }}
key: ${{ runner.os }}-${{ steps.setup-python.outputs.python-version }}-uv-${{ hashFiles(format('**/py{0}-ci-requirements.txt', env.PYTHON)) }}
enable-cache: true
- name: Install apache-arrow on ubuntu
if: matrix.os == 'ubuntu-latest'
run: |
Expand Down Expand Up @@ -82,3 +74,5 @@ jobs:
run: |
pytest -n 8 --cov=./ --cov-report=xml --color=yes sdk/python/tests --integration --durations=5 --timeout=1200 --timeout_method=thread -k "Snowflake and not dynamo and not Redshift and not Bigquery and not gcp and not minio_registry"
pytest -n 8 --cov=./ --cov-report=xml --color=yes sdk/python/tests --integration --durations=5 --timeout=1200 --timeout_method=thread -k "File and not dynamo and not Redshift and not Bigquery and not gcp and not minio_registry"
- name: Minimize uv cache
run: uv cache prune --ci
17 changes: 5 additions & 12 deletions .github/workflows/java_master_only.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -118,19 +118,10 @@ jobs:
with:
python-version: 3.11
architecture: x64
- name: Install uv
run: |
curl -LsSf https://astral.sh/uv/install.sh | sh
- name: Get uv cache dir
id: uv-cache
run: |
echo "::set-output name=dir::$(uv cache dir)"
- name: uv cache
uses: actions/cache@v4
- name: Install the latest version of uv
uses: astral-sh/setup-uv@v5
with:
path: ${{ steps.uv-cache.outputs.dir }}
key: ${{ runner.os }}-${{ steps.setup-python.outputs.python-version }}-uv-${{ hashFiles(format('**/py{0}-ci-requirements.txt', env.PYTHON)) }}

enable-cache: true
- name: Install Python dependencies
run: make install-python-dependencies-ci
- uses: actions/cache@v4
Expand Down Expand Up @@ -159,3 +150,5 @@ jobs:
run: aws sts get-caller-identity
- name: Run integration tests
run: make test-java-integration
- name: Minimize uv cache
run: uv cache prune --ci
18 changes: 6 additions & 12 deletions .github/workflows/java_pr.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -167,18 +167,10 @@ jobs:
with:
python-version: 3.11
architecture: x64
- name: Install uv
run: |
curl -LsSf https://astral.sh/uv/install.sh | sh
- name: Get uv cache dir
id: uv-cache
run: |
echo "::set-output name=dir::$(uv cache dir)"
- name: uv cache
uses: actions/cache@v4
with:
path: ${{ steps.uv-cache.outputs.dir }}
key: ${{ runner.os }}-${{ steps.setup-python.outputs.python-version }}-uv-${{ hashFiles(format('**/py{0}-ci-requirements.txt', env.PYTHON)) }}
- name: Install the latest version of uv
uses: astral-sh/setup-uv@v5
with:
enable-cache: true
- name: Install dependencies
run: make install-python-dependencies-ci
- name: Run integration tests
Expand All @@ -190,3 +182,5 @@ jobs:
name: it-report
path: spark/ingestion/target/test-reports/TestSuite.txt
retention-days: 5
- name: Minimize uv cache
run: uv cache prune --ci
20 changes: 15 additions & 5 deletions .github/workflows/linter.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,21 +4,31 @@ on: [push, pull_request]

jobs:
lint-python:
runs-on: [ubuntu-latest]
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
python-version: [ "3.9", "3.10", "3.11"]
os: [ ubuntu-latest ]
env:
PYTHON: 3.11
OS: ${{ matrix.os }}
Copy link
Member

@franciscojavierarceo franciscojavierarceo Mar 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this change?

Copy link
Contributor Author

@tchughesiv tchughesiv Apr 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

because i've enabled this job to run against multiple python versions

PYTHON: ${{ matrix.python-version }}
steps:
- uses: actions/checkout@v4
- name: Setup Python
id: setup-python
uses: actions/setup-python@v5
with:
python-version: "3.11"
python-version: ${{ matrix.python-version }}
architecture: x64
- name: Install uv
run: curl -LsSf https://astral.sh/uv/install.sh | sh
- name: Install the latest version of uv
uses: astral-sh/setup-uv@v5
with:
enable-cache: true
- name: Install dependencies
run: |
make install-python-dependencies-ci
- name: Lint python
run: make lint-python
- name: Minimize uv cache
run: uv cache prune --ci
15 changes: 5 additions & 10 deletions .github/workflows/master_only.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,17 +53,10 @@ jobs:
aws-region: us-west-2
- name: Use AWS CLI
run: aws sts get-caller-identity
- name: Install uv
run: curl -LsSf https://astral.sh/uv/install.sh | sh
- name: Get uv cache dir
id: uv-cache
run: |
echo "::set-output name=dir::$(uv cache dir)"
- name: uv cache
uses: actions/cache@v4
- name: Install the latest version of uv
uses: astral-sh/setup-uv@v5
with:
path: ${{ steps.uv-cache.outputs.dir }}
key: ${{ runner.os }}-${{ steps.setup-python.outputs.python-version }}-uv-${{ hashFiles(format('**/py{0}-ci-requirements.txt', env.PYTHON)) }}
enable-cache: true
- name: Install dependencies
run: make install-python-dependencies-ci
- name: Setup Redis Cluster
Expand All @@ -88,6 +81,8 @@ jobs:
run: pytest --verbose --color=yes sdk/python/tests --integration --benchmark --benchmark-autosave --benchmark-save-data --durations=5
- name: Upload Benchmark Artifact to S3
run: aws s3 cp --recursive .benchmarks s3://feast-ci-pytest-benchmark
- name: Minimize uv cache
run: uv cache prune --ci

build-all-docker-images:
if: github.repository == 'feast-dev/feast'
Expand Down
15 changes: 5 additions & 10 deletions .github/workflows/nightly-ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -117,17 +117,10 @@ jobs:
aws-region: us-west-2
- name: Use AWS CLI
run: aws sts get-caller-identity
- name: Install uv
run: curl -LsSf https://astral.sh/uv/install.sh | sh
- name: Get uv cache dir
id: uv-cache
run: |
echo "::set-output name=dir::$(uv cache dir)"
- name: uv cache
uses: actions/cache@v4
- name: Install the latest version of uv
uses: astral-sh/setup-uv@v5
with:
path: ${{ steps.uv-cache.outputs.dir }}
key: ${{ runner.os }}-${{ steps.setup-python.outputs.python-version }}-uv-${{ hashFiles(format('**/py{0}-ci-requirements.txt', env.PYTHON)) }}
enable-cache: true
- name: Install apache-arrow on ubuntu
if: matrix.os == 'ubuntu-latest'
run: |
Expand Down Expand Up @@ -155,3 +148,5 @@ jobs:
SNOWFLAKE_CI_ROLE: ${{ secrets.SNOWFLAKE_CI_ROLE }}
SNOWFLAKE_CI_WAREHOUSE: ${{ secrets.SNOWFLAKE_CI_WAREHOUSE }}
run: make test-python-integration
- name: Minimize uv cache
run: uv cache prune --ci
15 changes: 5 additions & 10 deletions .github/workflows/pr_integration_tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,17 +78,10 @@ jobs:
aws-region: us-west-2
- name: Use AWS CLI
run: aws sts get-caller-identity
- name: Install uv
run: curl -LsSf https://astral.sh/uv/install.sh | sh
- name: Get uv cache dir
id: uv-cache
run: |
echo "::set-output name=dir::$(uv cache dir)"
- name: uv cache
uses: actions/cache@v4
- name: Install the latest version of uv
uses: astral-sh/setup-uv@v5
with:
path: ${{ steps.uv-cache.outputs.dir }}
key: ${{ runner.os }}-${{ steps.setup-python.outputs.python-version }}-uv-${{ hashFiles(format('**/py{0}-ci-requirements.txt', env.PYTHON)) }}
enable-cache: true
- name: Install dependencies
run: make install-python-dependencies-ci
- name: Setup Redis Cluster
Expand All @@ -105,3 +98,5 @@ jobs:
SNOWFLAKE_CI_ROLE: ${{ secrets.SNOWFLAKE_CI_ROLE }}
SNOWFLAKE_CI_WAREHOUSE: ${{ secrets.SNOWFLAKE_CI_WAREHOUSE }}
run: make test-python-integration
- name: Minimize uv cache
run: uv cache prune --ci
15 changes: 5 additions & 10 deletions .github/workflows/pr_local_integration_tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,17 +40,10 @@ jobs:
with:
python-version: ${{ matrix.python-version }}
architecture: x64
- name: Install uv
run: curl -LsSf https://astral.sh/uv/install.sh | sh
- name: Get uv cache dir
id: uv-cache
run: |
echo "dir=$(uv cache dir)" >> $GITHUB_OUTPUT
- name: uv cache
uses: actions/cache@v4
- name: Install the latest version of uv
uses: astral-sh/setup-uv@v5
with:
path: ${{ steps.uv-cache.outputs.dir }}
key: ${{ runner.os }}-${{ matrix.python-version }}-uv-${{ hashFiles(format('**/py{0}-ci-requirements.txt', matrix.python-version)) }}
enable-cache: true
- name: Install dependencies
run: make install-python-dependencies-ci
- name: Test local integration tests
Expand All @@ -62,3 +55,5 @@ jobs:
go-version: 1.22.9
- name: Operator Data Source types test
run: make -C infra/feast-operator test-datasources
- name: Minimize uv cache
run: uv cache prune --ci
15 changes: 5 additions & 10 deletions .github/workflows/pr_remote_rbac_integration_tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,19 +40,14 @@ jobs:
with:
python-version: ${{ matrix.python-version }}
architecture: x64
- name: Install uv
run: curl -LsSf https://astral.sh/uv/install.sh | sh
- name: Get uv cache dir
id: uv-cache
run: |
echo "dir=$(uv cache dir)" >> $GITHUB_OUTPUT
- name: uv cache
uses: actions/cache@v4
- name: Install the latest version of uv
uses: astral-sh/setup-uv@v5
with:
path: ${{ steps.uv-cache.outputs.dir }}
key: ${{ runner.os }}-${{ matrix.python-version }}-uv-${{ hashFiles(format('**/py{0}-ci-requirements.txt', matrix.python-version)) }}
enable-cache: true
- name: Install dependencies
run: make install-python-dependencies-ci
- name: Test rbac and remote feature integration tests
if: ${{ always() }} # this will guarantee that step won't be canceled and resources won't leak
run: make test-python-integration-rbac-remote
- name: Minimize uv cache
run: uv cache prune --ci
20 changes: 6 additions & 14 deletions .github/workflows/smoke_tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,21 +25,13 @@ jobs:
with:
python-version: ${{ matrix.python-version }}
architecture: x64
- name: Install uv
run: |
curl -LsSf https://astral.sh/uv/install.sh | sh
- name: Get uv cache dir
id: uv-cache
run: |
echo "dir=$(uv cache dir)" >> $GITHUB_OUTPUT
- name: uv cache
uses: actions/cache@v4
- name: Install the latest version of uv
uses: astral-sh/setup-uv@v5
with:
path: ${{ steps.uv-cache.outputs.dir }}
key: ${{ runner.os }}-${{ steps.setup-python.outputs.python-version }}-uv-${{ hashFiles(format('**/py{0}-ci-requirements.txt', env.PYTHON)) }}
enable-cache: true
- name: Install dependencies
run: |
uv pip sync --system sdk/python/requirements/py${{ matrix.python-version }}-requirements.txt
uv pip install --system --no-deps .
run: make install-python-dependencies-ci
- name: Test Imports
run: python -c "from feast.cli import cli"
- name: Minimize uv cache
run: uv cache prune --ci
17 changes: 5 additions & 12 deletions .github/workflows/unit_tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,23 +33,16 @@ jobs:
with:
python-version: ${{ matrix.python-version }}
architecture: x64
- name: Install uv
run: |
curl -LsSf https://astral.sh/uv/install.sh | sh
- name: Get uv cache dir
id: uv-cache
run: |
echo "dir=$(uv cache dir)" >> $GITHUB_OUTPUT
- name: uv cache
uses: actions/cache@v4
- name: Install the latest version of uv
uses: astral-sh/setup-uv@v5
with:
path: ${{ steps.uv-cache.outputs.dir }}
key: ${{ runner.os }}-${{ steps.setup-python.outputs.python-version }}-uv-${{ hashFiles(format('**/py{0}-ci-requirements.txt', env.PYTHON)) }}
enable-cache: true
- name: Install dependencies
run: make install-python-dependencies-ci
- name: Test Python
run: make test-python-unit

- name: Minimize uv cache
run: uv cache prune --ci

unit-test-ui:
runs-on: ubuntu-latest
Expand Down
Loading
Loading