Skip to content

add AWS AgentCore zero code example#131

Open
mesutoezdil wants to merge 3 commits into
agentevals-dev:mainfrom
mesutoezdil:feature/agentcore-zero-code-example
Open

add AWS AgentCore zero code example#131
mesutoezdil wants to merge 3 commits into
agentevals-dev:mainfrom
mesutoezdil:feature/agentcore-zero-code-example

Conversation

@mesutoezdil
Copy link
Copy Markdown
Contributor

@mesutoezdil mesutoezdil commented May 1, 2026 •
edited
Loading

Closes #93

Adds an AWS AgentCore zero-code OTLP example.

Uses BedrockAgentCoreApp from bedrock-agentcore and StrandsTelemetry for OTLP export.

pip install -r examples/zero-code-examples/agentcore/requirements.txt
export AWS_DEFAULT_REGION=us-east-1
agentevals serve --dev
python examples/zero-code-examples/agentcore/run.py
curl http://localhost:8080/invocations -d '{"prompt": "Roll a 20-sided die"}'

@mesutoezdil
Copy link
Copy Markdown
Contributor Author

mesutoezdil commented May 1, 2026 •
edited
Loading

Imports work. OTel setup follows the same pattern as the existing Strands example. roll_die and check_prime tool logic is correct.

@mesutoezdil
Copy link
Copy Markdown
Contributor Author

mesutoezdil commented May 1, 2026 •
edited
Loading

Tested the OTel pipeline locally (agentevals serve --dev) using a separate script with Strands and a Nebius Cloud endpoint in place of BedrockModel. The actual run.py was not executed end to end.
I do not have an AWS account..

What was verified:

  • StrandsTelemetry sets up the tracer provider correctly
  • OTLPSpanExporter sends spans to agentevals on port 4318
  • roll_die and check_prime are called and captured in traces
  • Sessions showed up in agentevals

What still needs AWS to verify:

  • BedrockModel connecting to Amazon Bedrock
  • e2e tests in tests/integration/test_live_agents.py (skipped without AWS_DEFAULT_REGION)

I think, it will work.

@mesutoezdil mesutoezdil force-pushed the feature/agentcore-zero-code-example branch from 7790773 to e79db6f Compare May 1, 2026 18:03
@mesutoezdil mesutoezdil mentioned this pull request May 1, 2026
Open
@mesutoezdil mesutoezdil force-pushed the feature/agentcore-zero-code-example branch 2 times, most recently from 6f8a84d to 18b69d7 Compare May 1, 2026 18:48
@krisztianfekete
Copy link
Copy Markdown
Contributor

krisztianfekete commented May 4, 2026 •
edited
Loading

Hi, thanks for the PR!

Apologies for not being clearer in #93, but this PR seems to be adding another Strands example via Bedrock models, but not really an AgentCore example. We are looking for an example running on/using AgentCore runtime/SDK, e.g. via https://github.com/aws/bedrock-agentcore-sdk-python.

If I understand correctly, you can use agentcore dev for local testing, so having an AWS account might not be a hard requirement to test traces.

@mesutoezdil
Copy link
Copy Markdown
Contributor Author

mesutoezdil commented May 4, 2026 •
edited
Loading

@krisztianfekete thanks.

Rewrote the example to use BedrockAgentCoreApp from bedrock-agentcore-sdk-python.

The handler is now an async generator decorated with @app.entrypoint and the server starts with app.run(), which is the AgentCore runtime pattern.

You were right about AWS not being a hard requirement for tracing.

Strands still exports OTLP spans even when BedrockModel raises NoCredentialsError, so all three e2e tests pass without AWS credentials.

@mesutoezdil mesutoezdil force-pushed the feature/agentcore-zero-code-example branch from 466d35a to acb22d9 Compare May 4, 2026 10:16
@mesutoezdil mesutoezdil force-pushed the feature/agentcore-zero-code-example branch 6 times, most recently from 3139267 to 3831ec7 Compare May 12, 2026 18:03
@mesutoezdil
Copy link
Copy Markdown
Contributor Author

mesutoezdil commented May 12, 2026

@krisztianfekete rebased on main and addressed all your feedback. ready for another look when you have time.

krisztianfekete
krisztianfekete requested changes May 13, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

@krisztianfekete krisztianfekete left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you share a screenshot or log excerpt from agentevals serve --dev showing the session/invocation produced by this script when Bedrock is actually reachable (or substitute a non-Bedrock model via the CLI flow you used originally)? I want to confirm that gen_ai.request.model, gen_ai.input.messages, and AgentCore runtime spans all land at the receiver, not just an empty error span.

Comment thread examples/zero-code-examples/agentcore/run.py Outdated
@@ -0,0 +1,49 @@
"""AWS AgentCore zero-code OTLP example."""
Copy link
Copy Markdown
Contributor

@krisztianfekete krisztianfekete May 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please provide more details on the approach like other examples.

Comment thread tests/integration/test_live_agents.py Outdated
main_port, otlp_http_port, mgr = live_servers
session_name = "e2e-agentcore"
with _agentcore_server(otlp_http_port, session_name, extra_env=_AGENTCORE_ENV):
httpx.post("http://127.0.0.1:8080/invocations", json={"prompt": "Roll a 20-sided die"}, timeout=60)
Copy link
Copy Markdown
Contributor

@krisztianfekete krisztianfekete May 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please assert 200 response codes here and below.

Comment thread tests/integration/test_live_agents.py Outdated
main_port, otlp_http_port, mgr = live_servers
session_name = "e2e-agentcore"
with _agentcore_server(otlp_http_port, session_name, extra_env=_AGENTCORE_ENV):
httpx.post("http://127.0.0.1:8080/invocations", json={"prompt": "Roll a 20-sided die"}, timeout=60)
Copy link
Copy Markdown
Contributor

@krisztianfekete krisztianfekete May 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please do not hardcode the port.

@krisztianfekete krisztianfekete added the Changes requested Waiting for submitter to make changes to their PR label May 13, 2026
@mesutoezdil
Copy link
Copy Markdown
Contributor Author

mesutoezdil commented May 13, 2026 •
edited
Loading

@krisztianfekete Tested locally now again with meta-llama/Llama-3.3-70B-Instruct via a Nebius endpoint (OpenAI-compatible) in place of Bedrock, like I said I use just Nebius for everything, using the same BedrockAgentCoreApp structure. gen_ai.request.model, gen_ai.input.messages, tool calls, and AgentCore runtime spans all landed at the receiver. Screenshots attached.

Screenshot 2026-05-13 at 13 16 23 Screenshot 2026-05-13 at 13 17 01 
❯ curl -s "http://localhost:8001/api/streaming/sessions" | python3 -m json.tool 2>/dev/null | tail -60
                    "modelInfo": {
                        "models": [
                            "Qwen/Qwen3-235B-A22B"
                        ],
                        "provider": "strands-agents"
                    }
                }
            ]
        },
        {
            "sessionId": "agentcore-zero-code-2",
            "traceId": "dae9fc127e92a1ed44023f38067d0d07",
            "evalSetId": "agentcore_eval",
            "spanCount": 6,
            "isComplete": true,
            "startedAt": "2026-05-13T11:12:59.174844+00:00",
            "metadata": {
                "telemetry.sdk.language": "python",
                "telemetry.sdk.name": "opentelemetry",
                "telemetry.sdk.version": "1.41.1",
                "service.name": "strands-agents",
                "service.version": "1.39.0"
            },
            "invocations": [
                {
                    "invocationId": "genai-92d4385228be3a70",
                    "userText": "Roll a 20-sided die for me",
                    "agentText": "The function call {\"name\": \"roll_die\", \"parameters\": {\"sides\": \"20\"}} will roll a 20-sided die.",
                    "toolCalls": [
                        {
                            "name": "roll_die",
                            "args": {
                                "sides": "20"
                            },
                            "id": "chatcmpl-tool-098f6b19fd4347d7814e8dbe0b23adb2"
                        }
                    ],
                    "toolResponses": [
                        {
                            "name": "roll_die",
                            "response": {
                                "result": "7"
                            },
                            "id": "chatcmpl-tool-098f6b19fd4347d7814e8dbe0b23adb2"
                        }
                    ],
                    "modelInfo": {
                        "models": [
                            "meta-llama/Llama-3.3-70B-Instruct"
                        ],
                        "inputTokens": 1270,
                        "outputTokens": 106,
                        "provider": "strands-agents"
                    }
                }
            ]
        }
    ],
    "error": null
}

@mesutoezdil mesutoezdil force-pushed the feature/agentcore-zero-code-example branch from fb26b2a to 62bd782 Compare May 13, 2026 11:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@krisztianfekete krisztianfekete Awaiting requested review from krisztianfekete

Requested changes must be addressed to merge this pull request.

Assignees

No one assigned

Labels

Changes requested Waiting for submitter to make changes to their PR

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Add AWS AgentCore zero code example

2 participants

@mesutoezdil @krisztianfekete