feat(scenarios): added scenario class to sdk #701
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sid-rl
force-pushed
the
siddarth/scenario-sdk
branch
from
b4e2996 to
10592f3
Compare
stainless-app
bot
force-pushed
the
next
branch
from
23670ef to
5fe7d00
Compare
… and add metadata tag instead of creating new ones every time (workaround for lack of cleanup option)
sid-rl
force-pushed
the
siddarth/scenario-sdk
branch
from
10592f3 to
dc8a68e
Compare
james-rl
requested changes
Dec 4, 2025
Contributor
james-rl
left a comment
james-rl
left a comment
There was a problem hiding this comment.
double-check you committed everything. Other than that i just have some minor doc suggestions
Comment on lines
14
to
17
| """Async wrapper around a scenario resource. | ||
|
|
||
| Provides async methods for retrieving scenario details, updating the scenario, | ||
| and starting scenario runs. |
Contributor
There was a problem hiding this comment.
get rid of the Async wrapper comment. It doesn't add value and it leaks system internals
| """ | ||
|
|
||
| def __init__(self, client: AsyncRunloop, scenario_id: str) -> None: | ||
| """Initialize the wrapper. |
| the underlying devbox, and retrieving scoring results. | ||
|
|
||
| Example: | ||
| >>> scenario = await sdk.scenario.from_id("scn-xxx") |
Contributor
There was a problem hiding this comment.
you have await here but the method isn't async
| The underlying devbox may still be starting; call await_env_ready() | ||
| on the returned ScenarioRun to wait for it to be ready. | ||
|
|
||
| :param params: See SDKScenarioRunParams for available parameters |
Contributor
There was a problem hiding this comment.
should be SDKScenarioRunAsyncParams; this is referencing the sync version
| pass | ||
|
|
||
|
|
||
| class TestAsyncScenarioBuilder: |
Contributor
There was a problem hiding this comment.
I think you might have missed a file. There's a test version of the builders, but I don't see the non-test version of the builder.
…d update docstrings to not leak internals
james-rl
approved these changes
Dec 5, 2025
Comment on lines
109
to
110
| self._client.devboxes.await_running(self._devbox_id, polling_config=options.get("polling_config")) | ||
| return self.get_info(**filter_params(options, PollingRequestOptions)) |
Contributor
There was a problem hiding this comment.
await_env_ready forwards the polling config, but get_info doesn't support it, so the arg will get dropped
Merged
stainless-app bot
pushed a commit
that referenced
this pull request
Dec 6, 2025
* uv lock upgrade * scenario wrapper * modified ScenarioStartRunParams to follow our previous pattern for use in the SDK * fixed redundancy and formatting in scenario and scenario run unit tests * formatting fixes * changed scenario and scenario smoketests to update existing scenarios and add metadata tag instead of creating new ones every time (workaround for lack of cleanup option) * changed scenario run methods to `run` and `run_async`, updated parameter TypedDicts appropriately * exposed all scenario update parameters appropriately using typeddict * formatting fixes * remove scenario builder smoketests * add score_and_complete and download_logs methods to scenario runs, and update docstrings to not leak internals * add unit tests for ScenarioOps and AsyncScenarioOps * update tests for scenario.update and change test name for scenario.run and run_async * add unit tests for download_logs and score_and_complete * don't pass polling_config to get_info within await_env_ready
stainless-app bot
added a commit
that referenced
this pull request
Dec 6, 2025
* chore(smoketests): fixed smoketests and cleaned up typing/formatting (#699) * increase timeout for flaky smoketest * don't convert params from TypedDict to dict (unnecessary and undoes type hints) * docstring formatting fixes * removed shell exec tests with 'working_dir' parameter testing * feat(scenarios): added scenario class to sdk (#701) * uv lock upgrade * scenario wrapper * modified ScenarioStartRunParams to follow our previous pattern for use in the SDK * fixed redundancy and formatting in scenario and scenario run unit tests * formatting fixes * changed scenario and scenario smoketests to update existing scenarios and add metadata tag instead of creating new ones every time (workaround for lack of cleanup option) * changed scenario run methods to `run` and `run_async`, updated parameter TypedDicts appropriately * exposed all scenario update parameters appropriately using typeddict * formatting fixes * remove scenario builder smoketests * add score_and_complete and download_logs methods to scenario runs, and update docstrings to not leak internals * add unit tests for ScenarioOps and AsyncScenarioOps * update tests for scenario.update and change test name for scenario.run and run_async * add unit tests for download_logs and score_and_complete * don't pass polling_config to get_info within await_env_ready * release: 1.1.0 --------- Co-authored-by: sid-rl <siddarth@runloop.ai> Co-authored-by: stainless-app[bot] <142633134+stainless-app[bot]@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Thin scenario class wrapper for sdk. Creation will be through a builder pattern (coming soon)