Conversation
shuyangli
force-pushed
the
sl/update-gepa-config-for-new-evaluators
branch
2 times, most recently
from
c577fc4 to
cb57aa5
Compare
shuyangli
changed the base branch from
main
to
sl/support-evaluator-autopilot-tool
shuyangli
force-pushed
the
sl/update-gepa-config-for-new-evaluators
branch
2 times, most recently
from
5e67cbe to
a12e781
Compare
shuyangli
force-pushed
the
sl/support-evaluator-autopilot-tool
branch
2 times, most recently
from
55d9bdf to
d898c24
Compare
shuyangli
force-pushed
the
sl/update-gepa-config-for-new-evaluators
branch
2 times, most recently
from
030e036 to
9dbd5ee
Compare
shuyangli
force-pushed
the
sl/support-evaluator-autopilot-tool
branch
from
d898c24 to
53ba089
Compare
shuyangli
force-pushed
the
sl/update-gepa-config-for-new-evaluators
branch
from
9dbd5ee to
05c0d25
Compare
github-actions
bot
added
has-merge-conflicts
and removed
stacked-pr-blocked-on-base-pr
labels
shuyangli
force-pushed
the
sl/update-gepa-config-for-new-evaluators
branch
from
05c0d25 to
dff3cae
Compare
shuyangli
force-pushed
the
sl/update-gepa-config-for-new-evaluators
branch
3 times, most recently
from
5bb8023 to
cc8ea0b
Compare
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: cc8ea0b6ee
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
shuyangli
force-pushed
the
sl/update-gepa-config-for-new-evaluators
branch
from
cc8ea0b to
b2aaf54
Compare
shuyangli
force-pushed
the
sl/update-gepa-config-for-new-evaluators
branch
from
b2aaf54 to
eccb558
Compare
shuyangli
force-pushed
the
sl/update-gepa-config-for-new-evaluators
branch
from
eccb558 to
758f52b
Compare
|
@BugBot review |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
| .get(evaluation_name) | ||
| .ok_or_else(|| { | ||
| Error::new(ErrorDetails::Config { | ||
| message: format!("Evaluation '{evaluation_name}' not found in config"), |
There was a problem hiding this comment.
Error message uses single quotes instead of backticks
Low Severity
The newly added error message wraps evaluation_name in single quotes (') instead of backticks. This is inconsistent with the same error message at line 659 in validate_gepa_config_uninitialized, which correctly uses backticks. This violates the rule requiring backticks to wrap technical terms in error messages.
Triggered by team rule: Prefer backticks (`) instead of ticks (') to wrap technical terms
3 participants
Step towards #7011.
This creates new Stored optimizer config types so we can continue deserializing old Gepa configs with evaluation names, and creates a new gepa config option that takes function_name and evaluator_names.
Note
Medium Risk
Touches GEPA optimizer configuration, validation, and evaluation execution paths; mistakes could break optimizer launches or change which evaluators are run. Backward-compat snapshot handling reduces rollout risk but adds new config-state permutations to test.
Overview
GEPA is updated to accept either a deprecated top-level
evaluation_nameor new function-scopedevaluator_names, enabling GEPA to run directly against evaluators defined under[functions.<fn>.evaluators].This threads the new fields through durable GEPA step params/checkpoints and runtime evaluation calls, builds inline evaluation configs from function evaluators when
evaluator_namesis used, and adjusts dataset labeling accordingly. Core config loading adds stored optimizer snapshot types to preserve historical GEPA configs and introduces validation + deprecation warnings (with new tests) to enforce exactly one ofevaluation_name/evaluator_names.Written by Cursor Bugbot for commit 758f52b. This will update automatically on new commits. Configure here.