Repository navigation
feat(aws-lambda): Link SQS and SNS invocations to their producers - #8257
Conversation
The boto3 integration propagates sentry-trace in SQS and SNS message attributes (#8145). Read it back in the Lambda integration and link the invocation span to each distinct producer span, like the boto3 receive span does. Messages never parent the invocation. Co-Authored-By: Alexander Webb <alexander.webb@sentry.io>
|
@sentry-junior unstack this from major/3.0, they are independent |
|
Done. I dissolved the stack that GitHub made from #5005 and this PR, using |
Coverage Results 📊✅ 58806 passed | ⏭️ 3071 skipped | Total: 61877 | Pass Rate: 95.04% | Execution Time: 160m 54s All tests are passing successfully. ❌ Patch coverage is 11.11% (4 of 36 changed executable lines covered; target 80%). Changed files with executable lines (1)
Coverage diff@@ Coverage Diff @@
## master #8257 +/-##
==========================================
+ Coverage 90.22% 90.27% +0.05%
==========================================
Files 203 199 -4
Tracked lines 27335 21826 -5509
Branches 10246 7604 -2642
==========================================
- Hits 24660 19702 -4958
+ Misses 2675 2124 -551
+ Partials 1559 1254 -305Generated by Coverage Action |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit ceb919c. Configure here.
A body that mentions sentry-trace but isn't a JSON object raised in json.loads, and the whole link list was dropped. Skip that record instead, like boto3's _message_link does.
|
@sentry-junior update with latest major/3.0 and solve the merge conflicts please |
…a-producer-links # Conflicts: # sentry_sdk/integrations/aws_lambda/_handler.py # tests/integrations/aws_lambda/test_aws_lambda.py
The span links spec says not to add a message_origin link when the consumer continues the producer's trace. Build the links after continue_trace and skip any whose trace_id matches the invocation's. Also test a sentry-trace without a sampling flag, which links without sampled. Co-Authored-By: Ivana Kellyer <131587164+sentrivana@users.noreply.github.com>
#8257 links SQS/SNS invocations to their producers on the invocation span. Remove this PR's second copy of that code from _trigger.py, keep the queue.process span without links, and reuse the test helpers and records already on major/3.0. Co-Authored-By: Ivana Kellyer <131587164+sentrivana@users.noreply.github.com>
Following the OTel messaging conventions, the process span carries the links to each message's creation context. Move the links that #8257 put on the invocation span to the queue.process span, reusing the same _get_producer_links. Co-Authored-By: Ivana Kellyer <131587164+sentrivana@users.noreply.github.com>
### Description For an SQS or SNS delivery, the handler now runs inside a `queue.process` child span of kind `consumer`. The span has: - `messaging.system` - `messaging.operation.type` and `messaging.operation.name`, both `process` - the batch message count - the destination name (queue or topic, from the first record) This fills in the processing side of the Queues insight. **Producer links move to this span.** The OTel messaging conventions put a consumer's links to each message's creation context on its process span. #8257 (merged) put them on the invocation span. This PR moves them: the invocation span has no links anymore, and `queue.process` gets them. The links are built by the same `_get_producer_links` from #8257 (dedupe, the cap of 128, skipping the continued trace), with no second copy of that parsing. The tests reuse the `SQS_RECORD` / `SNS_RECORD` fixtures and the `test_producer_links` cases already on `major/3.0`. Those tests now assert the links on the process span and that the invocation span has none. For comparison, the JS SDK's Lambda integration adds no links. It only continues the trace from `ClientContext` or headers. Its links are on the SQS `ReceiveMessage` span only. This is Pablo's code from `pablo/aws-lambda-messaging`, minus his link code. On his branch it sat on top of HTTP and invocation attribute work (PY-3017, PY-3019, PY-3021), which isn't part of the November plan. In `_handler.py` I only wrapped the handler call in the process span. `test_process_span_error` covers a handler that raises. Open points for review: - `messaging.message.id` on single-message process spans isn't added yet (proposed in PY-3020). - The destination name comes from the first record. The alternative is to set it only when all records share it. #### Issues * resolves: PY-3020 * Part of #8083 <!-- junior-request-attribution:start --> via @dingsdax, @sentrivana. <!-- junior-request-attribution:end --> <!-- junior-session-footer:start --> <!-- junior-conversation-id:slack%3AC09M26BC6A1%3A1794211222.463239 --> -- [View Session](https://junior-prod.sentry.dev/conversations/slack%3AC09M26BC6A1%3A1794211222.463239) [[Sentry]](https://sentry.sentry.io/explore/conversations/slack%3AC09M26BC6A1%3A1794211222.463239/?project=4510944073809921) <!-- junior-session-footer:end --> --------- Co-authored-by: Pablo Deputter <pablo.deputter@sentry.io> Co-authored-by: Pablo Deputter <71842639+pabloDeputter@users.noreply.github.com> Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com> Co-authored-by: Johannes Daxböck <114897+dingsdax@users.noreply.github.com> Co-authored-by: Ivana Kellyer <131587164+sentrivana@users.noreply.github.com>

Description
Since #8137, #8142, and #8145, boto3 puts
sentry-tracein SQS and SNS message attributes, but nothing reads it on the Lambda side. This PR is the smallest change that closes the loop for SQS- and SNS-triggered functions. The Lambda integration now readssentry-tracefrom each record and adds amessage_originlink from the invocation span to each distinct producer span. This matches what the boto3ReceiveMessagespan already does. Messages never become the invocation's parent.It reads:
messageAttributes["sentry-trace"].stringValueSns.MessageAttributes["sentry-trace"].Valuebody, the same as boto3 doesLinks are deduplicated, because a batch send gives many messages the same producer span, and capped at 128, OTel's default, because Lambda can deliver up to 10,000 SQS messages at once.
Scope. This is intentionally kept separate from the stretch stack. It doesn't add
faas.trigger(#8222) orqueue.processspans (#8223), and it doesn't touchmessaging.batch.message_count(#8229). The invocation span is otherwise unchanged. SynchronousInvokepropagation throughClientContextis #8221. That PR changes thecontinue_tracecall and this one only addslinks=, so the two are independent.The link-building code repeats boto3's
_message_linkin a Lambda-event shape: Lambda records usemessageAttributes/stringValue/body, while the boto3 response usesMessageAttributes/StringValue/Body. Importing from the boto3 integration would makeaws_lambdadepend on botocore.Testing.
test_producer_linkssends SQS, SNS, and SNS-to-SQS events toBasicOkSpanStreaming. It asserts the segment span'slinksand that the segment span has no parent. It passes in CI's Cloud (Lambda) jobs, andruffandmypy sentry_sdkpass.codecov/patchis red because the Lambda tests run insam localcontainers, so their coverage isn't counted.Issues
via webb, @sentrivana.
--
View Session [Sentry]