Skip to content

chore: Auto-detect python version in Makefile #4419

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
franciscojavierarceo merged 1 commit into from
Aug 17, 2024
Merged

chore: Auto-detect python version in Makefile #4419

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 7 additions & 6 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ ifeq ($(shell uname -s), Darwin)
OS = osx
endif
TRINO_VERSION ?= 376
PYTHON_VERSION = ${shell python --version | grep -Eo '[0-9]\.[0-9]+'}

# General

Expand All @@ -37,22 +38,22 @@ build: protos build-java build-docker
# Python SDK

install-python-ci-dependencies:
python -m piptools sync sdk/python/requirements/py$(PYTHON)-ci-requirements.txt
python -m piptools sync sdk/python/requirements/py$(PYTHON_VERSION)-ci-requirements.txt
pip install --no-deps -e .
python setup.py build_python_protos --inplace

install-python-ci-dependencies-uv:
uv pip sync --system sdk/python/requirements/py$(PYTHON)-ci-requirements.txt
uv pip sync --system sdk/python/requirements/py$(PYTHON_VERSION)-ci-requirements.txt
uv pip install --system --no-deps -e .
python setup.py build_python_protos --inplace

install-python-ci-dependencies-uv-venv:
uv pip sync sdk/python/requirements/py$(PYTHON)-ci-requirements.txt
uv pip sync sdk/python/requirements/py$(PYTHON_VERSION)-ci-requirements.txt
uv pip install --no-deps -e .
python setup.py build_python_protos --inplace

lock-python-ci-dependencies:
uv pip compile --system --no-strip-extras setup.py --extra ci --output-file sdk/python/requirements/py$(PYTHON)-ci-requirements.txt
uv pip compile --system --no-strip-extras setup.py --extra ci --output-file sdk/python/requirements/py$(PYTHON_VERSION)-ci-requirements.txt

package-protos:
cp -r ${ROOT_DIR}/protos ${ROOT_DIR}/sdk/python/feast/protos
Expand All @@ -61,11 +62,11 @@ compile-protos-python:
python setup.py build_python_protos --inplace

install-python:
python -m piptools sync sdk/python/requirements/py$(PYTHON)-requirements.txt
python -m piptools sync sdk/python/requirements/py$(PYTHON_VERSION)-requirements.txt
python setup.py develop

lock-python-dependencies:
uv pip compile --system --no-strip-extras setup.py --output-file sdk/python/requirements/py$(PYTHON)-requirements.txt
uv pip compile --system --no-strip-extras setup.py --output-file sdk/python/requirements/py$(PYTHON_VERSION)-requirements.txt

lock-python-dependencies-all:
pixi run --environment py39 --manifest-path infra/scripts/pixi/pixi.toml "uv pip compile --system --no-strip-extras setup.py --output-file sdk/python/requirements/py3.9-requirements.txt"
Expand Down
20 changes: 12 additions & 8 deletions docs/project/development-guide.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -132,8 +132,7 @@ Setting up your development environment for Feast Python SDK / CLI:
source venv/bin/activate
```
4. (M1 Mac only): Follow the [dev guide](https://github.com/feast-dev/feast/issues/2105)
5. Install uv
It is recommended to use uv for managing python dependencies.
5. Install uv. It is recommended to use uv for managing python dependencies.
```sh
curl -LsSf https://astral.sh/uv/install.sh | sh
```
Expand All @@ -145,21 +144,26 @@ pip install uv
```
make build-ui
```
7. (Optional) install pixi
pixi is necessary to run step 8 for all python versions at once.
7. (Optional) install pixi. pixi is necessary to run step 8 for all python versions at once.
```sh
curl -fsSL https://pixi.sh/install.sh | bash
```
8. (Optional): Recompile python lock files
If you make changes to requirements or simply want to update python lock files to reflect latest versioons.
8. (Optional): Recompile python lock files. Only when you make changes to requirements or simply want to update python lock files to reflect latest versioons.
```sh
make lock-python-dependencies-all
```
9. Install development dependencies for Feast Python SDK / CLI
This will install package versions from the lock file, install editable version of feast and compile protobufs.
9. Install development dependencies for Feast Python SDK / CLI. This will install package versions from the lock file, install editable version of feast and compile protobufs.

If running inside a virtual environment:
```sh
make install-python-ci-dependencies-uv-venv
```

Otherwise:
```sh
make install-python-ci-dependencies-uv
```

10. Spin up Docker Image
```sh
docker build -t docker-whale -f ./sdk/python/feast/infra/feature_servers/multicloud/Dockerfile .
Expand Down