Repository navigation
feat(boto3): Add SQS extension - #8137
Conversation
Coverage Results 📊✅ 58193 passed | ❌ 1 failed | ⏭️ 3058 skipped | Total: 61252 | Pass Rate: 95.01% | Execution Time: 162m 50s ✅ Patch coverage is 100.00% (70 of 70 changed executable lines covered; target 80%). Changed files with executable lines (5)
Coverage diff@@ Coverage Diff @@
## master #8137 +/-##
==========================================
+ Coverage 90.24% 90.37% +0.13%
==========================================
Files 203 199 -4
Tracked lines 27114 21660 -5454
Branches 10136 7518 -2618
==========================================
- Hits 24468 19575 -4893
+ Misses 2646 2085 -561
+ Partials 1540 1254 -286Generated by Coverage Action |
5f4f1d0 to
0f90423
Compare
### Description Removes `AwsCallContext.service_id_hyphenized` from `sentry_sdk/integrations/boto3/_context.py`. It's still computed on every call (`service_id.hyphenize()`), but on `major/3.0` nothing reads it. It had two readers. One was the `aws.service_id` span tag, which #7261 ("Drop tags auto-set by the SDK") removed. The other was the span name: #7800 ("align span names with OTel conventions") switched that to `ctx.service_id`, so the hyphenized value lost its last reader. This removes the slot, the attribute and the `hyphenize()` call. **Evidence** - On `major/3.0`, `git grep -w service_id_hyphenized` matches only the three lines in `_context.py` that set it up. - The open boto3 extension PRs against `major/3.0` (#8117 SQS, #8137 Lambda) don't reference it. On `master`, `service_id_hyphenized` is still used for the span name and the tag, so this only applies to `major/3.0`. #### Issues n/a <!-- junior-request-attribution:start --> via **Ivana Kellyer**. <!-- junior-request-attribution:end --> <!-- junior-session-footer:start --> <!-- junior-conversation-id:slack%3AC0BTREZTS2Z%3A1793270559.842119 --> -- [View Junior Session](https://junior-prod.sentry.dev/conversations/slack%3AC0BTREZTS2Z%3A1793270559.842119) [[Sentry]](https://sentry.sentry.io/explore/conversations/slack%3AC0BTREZTS2Z%3A1793270559.842119/?project=4510944073809921) <!-- junior-session-footer:end --> Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com> Co-authored-by: Ivana Kellyer <ivana.kellyer@sentry.io>
| attributes[SPANDATA.AWS_SQS_QUEUE_URL] = queue_url | ||
|
|
||
| # `https://sqs.eu-north-1.amazonaws.com/123456789012/orders` => ['123456789012', 'orders'] | ||
| path = unquote(urlsplit(queue_url).path).strip("/").split("/") |
There was a problem hiding this comment.
Is queue_url validated by the library before it's handed to us through ctx.params?
There was a problem hiding this comment.
No, all parameters used in get_request_attributes() are yet to be validated. QueueUrl is also only checked to be a string, e.g. a non-string 1234 will raise an error at urlsplit and is captured at boto3/_instrumentation.py=180, so all SQS request attributes are dropped. Then get_span_name() at boto3/_instrumentation.py:183 will raise an error (if it's one of the API operations in _MESSAGING_OPERATIONS), which is captured again at boto3/_client.py:92, so no span is sent. Other operations still send the span, just without any SQS attributes.
There was a problem hiding this comment.
I did fix this partly in #8142, by adding another guard to get_span_name() at boto3/_instrumentation.py=183, so the span is sent, but still has no request attributes.
So I added some more precautions (c640799) in get_span_name() so that it doesn't take the operation type from attributes anymore, and the queue_url parsing is also guarded.
There was a problem hiding this comment.
Summary for whoever resolves this thread. botocore only checks that QueueUrl is a string, so this PR no longer relies on its shape:
- c640799 (this PR):
get_span_name()takes the operation type from_MESSAGING_OPERATIONSinstead of the request attributes, and the URL parsing inget_request_attributes()sits insidecapture_internal_exceptions(). A malformed URL now only loses the destination name and account ID, and the span is still sent. - c73d9ba (feat(boto3): Add SNS extension #8142): a second
capture_internal_exceptions()guard aroundget_span_name()in_instrumentation.py.
I merged major/3.0 into this branch today, and the boto3 tests still pass.
- stop reading operation type from attributes in `get_span_name()` so that failure in `get_request_attributes()` can't make this fail as well - wrap URL parsing in `get_request_attributes()` so failure just loses the attributes that come from url
…sqs-extension Co-Authored-By: Johannes Daxböck <114897+dingsdax@users.noreply.github.com>
Botocore validates params after our hook, so a bad Entries or ARN value now only loses that attribute, not every SQS request attribute. Move the batch error.type code to _utils so SNS can share it, and link the SQS docs for BatchResultErrorEntry. Co-Authored-By: Johannes Daxböck <114897+dingsdax@users.noreply.github.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 438ded1. Configure here.
on_success_response() ran outside capture_internal_exceptions(), so a SendMessage or ReceiveMessage response without MessageId (e.g. from a botocore Stubber or an SQS-compatible service) raised a KeyError into the caller after a successful call and left the span open. Co-Authored-By: Ivana Kellyer <131587164+sentrivana@users.noreply.github.com>
SQS messaging calls no longer follow the <ServiceID>.<Operation> name and rpc op the guide describes for boto3 spans. Co-Authored-By: Ivana Kellyer <131587164+sentrivana@users.noreply.github.com>
### Description Propagates the trace through SQS and SNS messages, so the consumer can be linked back to the request that sent the message. **Send side** (`SendMessage`, `SendMessageBatch`, `Publish`, `PublishBatch`): - Adds `sentry-trace` and `baggage` as `String` message attributes, on the message or on each batch entry. - Only when `trace_propagation_targets` match the endpoint. - Skips a message if the caller already set these attributes, if it would go over the 10-attribute limit, or if it would go over 256 KiB. AWS would reject the message otherwise. - Works on a copy of the params. The caller's dict isn't changed. **Receive side** (`ReceiveMessage`): - Adds `sentry-trace` and `baggage` to `MessageAttributeNames`, so SQS returns them. Only when the span is recorded, and not when the caller already asks for `All` or `.*`. - Adds a span link to each distinct producer context in the received messages, with `sentry.link.type: message_origin`. - Reads the context from SQS message attributes, and from the JSON envelope of an SNS notification without raw message delivery. **Tests**: injection and extraction for single and batch calls, the limits above, and a round-trip test: send a message, receive what was injected, and check that the link points at the send span. Stacked on #8142 (SNS), which is stacked on #8137 (SQS). #### Issues * resolves: PY-3008 <!-- junior-request-attribution:start --> via @dingsdax. <!-- junior-request-attribution:end --> <!-- junior-session-footer:start --> <!-- junior-conversation-id:slack%3AC0C9X9V5NMT%3A1793904671.980379 --> -- [View Session](https://junior-prod.sentry.dev/conversations/slack%3AC0C9X9V5NMT%3A1793904671.980379) [[Sentry]](https://sentry.sentry.io/explore/conversations/slack%3AC0C9X9V5NMT%3A1793904671.980379/?project=4510944073809921) <!-- junior-session-footer:end --> --------- Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com> Co-authored-by: Johannes Daxböck <114897+dingsdax@users.noreply.github.com> Co-authored-by: Ivana Kellyer <ivana.kellyer@sentry.io> Co-authored-by: Ivana Kellyer <131587164+sentrivana@users.noreply.github.com>
) ### Description Since #8137, #8142, and #8145, boto3 puts `sentry-trace` in SQS and SNS message attributes, but nothing reads it on the Lambda side. This PR is the smallest change that closes the loop for SQS- and SNS-triggered functions. The Lambda integration now reads `sentry-trace` from each record and adds a `message_origin` link from the invocation span to each distinct producer span. This matches what the boto3 `ReceiveMessage` span already does. Messages never become the invocation's parent. It reads: - SQS records: `messageAttributes["sentry-trace"].stringValue` - SNS records: `Sns.MessageAttributes["sentry-trace"].Value` - SNS → SQS without raw message delivery: the SNS envelope in the SQS `body`, the same as boto3 does Links are deduplicated, because a batch send gives many messages the same producer span, and capped at 128, OTel's default, because Lambda can deliver up to 10,000 SQS messages at once. **Scope.** This is intentionally kept separate from the stretch stack. It doesn't add `faas.trigger` (#8222) or `queue.process` spans (#8223), and it doesn't touch `messaging.batch.message_count` (#8229). The invocation span is otherwise unchanged. Synchronous `Invoke` propagation through `ClientContext` is #8221. That PR changes the `continue_trace` call and this one only adds `links=`, so the two are independent. The link-building code repeats boto3's `_message_link` in a Lambda-event shape: Lambda records use `messageAttributes` / `stringValue` / `body`, while the boto3 response uses `MessageAttributes` / `StringValue` / `Body`. Importing from the boto3 integration would make `aws_lambda` depend on botocore. **Testing.** `test_producer_links` sends SQS, SNS, and SNS-to-SQS events to `BasicOkSpanStreaming`. It asserts the segment span's `links` and that the segment span has no parent. It passes in CI's Cloud (Lambda) jobs, and `ruff` and `mypy sentry_sdk` pass. `codecov/patch` is red because the Lambda tests run in `sam local` containers, so their coverage isn't counted. #### Issues * Part of #8081 <!-- junior-request-attribution:start --> via **webb**, @sentrivana. <!-- junior-request-attribution:end --> <!-- junior-session-footer:start --> <!-- junior-conversation-id:slack%3AC09M26BC6A1%3A1794211222.463239 --> -- [View Session](https://junior-prod.sentry.dev/conversations/slack%3AC09M26BC6A1%3A1794211222.463239) [[Sentry]](https://sentry.sentry.io/explore/conversations/slack%3AC09M26BC6A1%3A1794211222.463239/?project=4510944073809921) <!-- junior-session-footer:end --> --------- Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com> Co-authored-by: Alexander Webb <alexander.webb@sentry.io> Co-authored-by: Ivana Kellyer <131587164+sentrivana@users.noreply.github.com>
… span (#8229) ### Description The Lambda span set `messaging.batch.message_count` to `1` for every event that isn't a list, and to `len(event)` for a top-level list. An SQS or SNS delivery is a dict with `Records`, so a batch of 10 messages reported 1. HTTP calls and direct invocations reported a batch they don't have. This PR removes the attribute from the Lambda span. The count belongs on messaging spans, as in the OTel messaging conventions: - boto3 `ReceiveMessage` and batch-send spans already set it (#8137, #8142). - For SQS/SNS-triggered Lambdas, the `queue.process` span in #8223 sets it. The JS SDK doesn't set it on the Lambda span either. The attribute was added in 3.0 (#7941) and never shipped in 2.x, so no released behavior changes. Replaces #8219 (same commits). GitHub never ran CI on #8219. **Note for review:** this is narrower than PY-3015, which asks to keep the attribute for SQS/SNS deliveries. @alexander-alderman-webb, please confirm that's OK. #### Issues * resolves: PY-3015 <!-- junior-request-attribution:start --> via @dingsdax. <!-- junior-request-attribution:end --> <!-- junior-session-footer:start --> <!-- junior-conversation-id:slack%3AC0C9X9V5NMT%3A1793904671.980379 --> -- [View Session](https://junior-prod.sentry.dev/conversations/slack%3AC0C9X9V5NMT%3A1793904671.980379) [[Sentry]](https://sentry.sentry.io/explore/conversations/slack%3AC0C9X9V5NMT%3A1793904671.980379/?project=4510944073809921) <!-- junior-session-footer:end --> --------- Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com> Co-authored-by: Johannes Daxböck <114897+dingsdax@users.noreply.github.com>

Description
Adds the SQS extension, now SQS MESSAGING operations become
queue.*spans so they show up in the Queues insight; all other SQS APIs stayrpc:SendMessage,SendMessageBatch:queue.publish, kindproducer(default isclient), namedsend <queue>ReceiveMessage:queue.receive, namedreceive <queue>DeleteMessage,DeleteMessageBatch:queue.settle, namedsettle <queue>Adds the following attributes:
messaging.system(aws_sqs),messaging.operation.typeandmessaging.operation.nameon the messaging APIs (all from OTel)messaging.batch.message_counton batch APIs: submitted entries for send/delete, returned messages for receivemessaging.message.idonSendMessage, and onReceiveMessagewhen exactly one message is returnedaws.sqs.queue.urlandmessaging.destination.name(queue name) on every API withQueueUrl; destination name also fromQueueNameandSourceArnIssues
Resolves #7572 & PY-2803