test(steering): Fix failing integ tests#1580
Merged
mkmeral merged 1 commit intostrands-agents:mainfrom Jan 28, 2026
Merged
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
zastrowm
previously approved these changes
Jan 28, 2026
afarntrog
previously approved these changes
Jan 28, 2026
This fix addresses the flaky test_agent_with_tool_steering_e2e test.
Root causes identified and fixed:
1. LLMSteeringHandler used 'or' operator for context_providers which
treated empty list as falsy, causing unintended default to LedgerProvider
2. The steering handler's guidance message was too soft ("Consider this
approach") instead of directive ("You MUST follow this guidance")
3. The test's system prompt was ambiguous, allowing the steering LLM to
analyze context data instead of following simple rules
Changes:
- Fixed LLMSteeringHandler.__init__ to properly distinguish between None
and empty list for context_providers parameter
- Updated guidance message format in SteeringHandler._handle_tool_steering_action
to be more directive when guiding agents
- Updated test system prompt to be explicit and rule-based, disabling
context providers to prevent confusing ledger data from influencing
the steering decision
The test now reliably:
1. Intercepts send_email calls with 'guide' decision
2. Provides clear directive to use send_notification instead
3. Main agent follows guidance and calls send_notification successfully
4c85033 to
12a2f17
Compare
lizradway
approved these changes
Jan 28, 2026
afarntrog
approved these changes
Jan 28, 2026
gaurav71531
pushed a commit
to gaurav71531/sdk-python
that referenced
this pull request
Feb 4, 2026
manoj-selvakumar5
pushed a commit
to manoj-selvakumar5/strands-sdk-python
that referenced
this pull request
Feb 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR fixes the flaky
test_agent_with_tool_steering_e2eintegration test intests_integ/steering/test_tool_steering.py.Problem
The test was failing intermittently because:
context_providers or [LedgerProvider()]pattern treated an empty list as falsy, defaulting to[LedgerProvider()]even when explicitly passed an empty listRoot Cause Analysis
When the test ran:
send_emailsend_notificationSolution
1. Fix LLMSteeringHandler context_providers handling
2. Make guidance message more directive
3. Improve test system prompt
Made the system prompt explicit, rule-based, and disabled context providers to prevent ledger data from influencing the steering decision.
Testing
Files Changed
src/strands/experimental/steering/core/handler.py- More directive guidance messagesrc/strands/experimental/steering/handlers/llm/llm_handler.py- Fix context_providers handlingtests/strands/experimental/steering/core/test_handler.py- Update unit test expectationtests_integ/steering/test_tool_steering.py- Improved test configuration