Skip to content

cosmos-xenna: Add version 0.5.11 - #4771

Open
luhenry wants to merge 7 commits into
mainfrom
cosmos-xenna
Open

luhenry wants to merge 7 commits into
mainfrom
cosmos-xenna

Conversation

@luhenry

@luhenry luhenry commented Nov 9, 2026 •

Copy link
Copy Markdown
Member

Compiles a single PyO3/maturin extension (cosmos_xenna._cosmos_xenna, Ray pipeline scheduling, file distribution over object_store) via a plain crates.io Cargo tree. Upstream publishes no riscv64 wheel.

Mirrors the build-wheels job of upstream's release-from-tag.yml.

Differs from upstream

  • Rust toolchain installed in-container via rustup - upstream's own job sets up a toolchain through actions/setup-python + maturin-action, neither of which supports riscv64.
  • musllinux dropped - rustup.rs ships no riscv64 musl toolchain.

Testing

  • --pyargs cosmos_xenna -m "not slow" - the test files already ship inside the wheel, so this runs them straight against the installed, compiled package; ray[default], obstore<0.9.0 and the rest of the hard runtime dependencies resolve from the registry.
  • Drops upstream's separate pytest -m 'slow' CI step as disproportionate for a shared riscv64 runner.

License: OK

Built on cp312-abi3 (also tested on cp313/cp314); CI pending.

luhenry added a commit that referenced this pull request Nov 9, 2026
ray[default]'s test-install resolves py-spy>=0.4.0 directly and
cryptography>=38.0.3 transitively (opencensus -> google-api-core ->
google-auth), both unbounded. PIP_EXTRA_INDEX_URL alone lets pip pick
PyPI's newest release of each over our registry's riscv64 wheel, and
both fail to source-build: py-spy 0.4.3 hits a rustc error in its own
source, cryptography 50.0.2's openssl-sys can't find openssl.pc.
PIP_ONLY_BINARY in CIBW_TEST_ENVIRONMENT (test-scoped only) makes pip
pick the registry's highest wheel for each, which satisfies both specs.
cosmos_xenna/file_distribution/_file_distribution.py does a top-level
import tqdm, but tqdm isn't declared anywhere in pyproject.toml
(dependencies or any optional-dependencies group). Upstream's own
uv.lock only carries tqdm in transitively, through huggingface-hub
and transformers in the examples extra, which uv.lock pulls in by
default. A plain pip install of the wheel never gets tqdm, so
collection blows up as soon as pytest imports file_distribution
(via pipelines.v1 and pipelines.private.pipelines), aborting all
21 test files that import it.

This is an upstream packaging gap, but there's nothing to patch in
our own install_requires here, so list tqdm explicitly for the
test step. It's a pure Python (py3-none-any) wheel, so it installs
fine from either index with no riscv64 build involved.
_publish-wheel.yml now requires a matrix input (main's "publish:
Validate effective wheel targets"), but this PR branch forked before
that landed and cosmos-xenna has never been merged to main, so
build-cosmos-xenna.yml never picked up the matching one-line update
every already-published package's workflow got. The merge ref
combines the new required-input _publish-wheel.yml with this branch's
old build-cosmos-xenna.yml, so every run has failed startup with
"Input matrix is required, but not provided".

Pass it the same way the mass-update commit did for every other
package's publish job.
5 non-slow tests (test_batch_size_hang, test_batching, test_empty_return,
test_serving, test_setup_on_node) failed with 'Not enough CPU resources
to run pipeline' or a stalled queue read. cosmos_xenna's own tests scale
their Ray resource requirements down via is_running_in_cicd(), which
upstream's ci.yml enables by setting IS_RUNNING_IN_CICD=1 - but
cibuildwheel's podman container doesn't forward that from the runner,
so the tests used their full-size (non-CI) requirements against a
3-CPU cluster and asked for more than was available. Setting the same
env var in CIBW_TEST_ENVIRONMENT restores upstream's own CI sizing.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant