[deps] bump ray to 2.56.0, add TE wheel to pyproject.toml - #1872
Merged
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
SumanthRH
reviewed
Jul 7, 2026
SumanthRH
left a comment
SumanthRH
left a comment
Member
There was a problem hiding this comment.
update image URIs in the docs as well?
erictang000
added
run_train_gpu_ci
run_train_megatron_gpu_ci_models
run_h100_gpu_ci
erictang000
added
run_train_gpu_ci
run_train_megatron_gpu_ci
run_h100_gpu_ci
erictang000
commented
Jul 8, 2026
|
|
||
| ENV CMAKE_ARGS=-DPython_EXECUTABLE=/home/ray/anaconda3/bin/python3 | ||
|
|
||
| RUN sudo mkdir /opt/wheels && sudo chown ray: /opt/wheels && \ |
Collaborator
Author
There was a problem hiding this comment.
this was not working previously:
the cleaner solution is just to directly save the wheel source in the pyproject.toml for TE-torch.
Member
There was a problem hiding this comment.
Hmm this looks like an issue with the path in our docker builds;
Installed 344 packages in 967ms
× Failed to download `transformer-engine-torch==2.11.0`
├─▶ Failed to read from the distribution cache
╰─▶ failed to query metadata of file
`/tmp/ray/opt/wheels/transformer_engine_torch-2.11.0-cp312-cp312-linux_x86_64.whl`:
No such file or directory (os error 2)
help: `transformer-engine-torch` (v2.11.0) was included because
`skyrl[megatron]` depends on `transformer-engine[pytorch]` (v2.11.0)
which depends on `transformer-engine-torch`Interesting that the path used is /tmp/ray/opt/wheels rather than /opt/wheels.
erictang000
changed the title
SumanthRH
approved these changes
Jul 9, 2026
SumanthRH
reviewed
Jul 9, 2026
SumanthRH
left a comment
SumanthRH
left a comment
Member
There was a problem hiding this comment.
Can you also update examples/train/thunder_agent/scripts/r2egym_32b/setup_env.sh:
This was referenced Jul 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bump Ray to 2.56.0 (and Transformer Engine to 2.11.0)
Summary
Upgrades the pinned Ray version from
2.51.1→2.56.0across Docker images, CI configs, docs, and the lockfile, and moves the Megatron backend's Transformer Engine install from a build-from-source step to prebuilt wheels(
2.10.0→2.11.0).Changes
Ray 2.56.0 bump
docker/Dockerfileanddocker/Dockerfile.megatrontoanyscale/ray:2.56.0-slim-py312-cu128.image_uri/ray_versionin allci/anyscale_*.yamlCI configs to2.56.0.override-dependenciesinskyrl-agent/pyproject.tomland refresheduv.lock.installation.mdx,.claude/docs/ci.md, READMEs) to reference the new2.56.0image tags and version.PlacementGroupSchedulingStrategyis now imported fromray.util.scheduling_strategiesinstead ofray.util.placement_group(inworker.pyandutils/utils.py).Transformer Engine 2.11.0 via prebuilt wheels
pip wheel/ build step fortransformer-engine-torchfromDockerfile.megatron; bumpedtorch2.10.0→2.11.0in the base layer.transformer-engine-torch==2.11.0directly in themegatronextra so the[tool.uv.sources]URL (a prebuilt torch-2.11 / cu128 / cp312 wheel) is honored — uv ignores sources for transitive-only deps.transformer-engine-cu12==2.11.0explicitly: the metatransformer-enginepackage hardcodestransformer-engine-cu13as its core dep (overridden tonever), so the matching cu12 core lib must be pulled indirectly — otherwise the meta package's sanity check fails on this CUDA 12.8 image.