Skip to content

feat(boto3): Add SQS extension - #8137

Merged
sentrivana merged 12 commits into
major/3.0from
pablo/boto3-add-sqs-extension
Nov 11, 2026
Merged

sentrivana merged 12 commits into
major/3.0from
pablo/boto3-add-sqs-extension

Conversation

@pabloDeputter

Copy link
Copy Markdown
Contributor

Description

Adds the SQS extension, now SQS MESSAGING operations become queue.* spans so they show up in the Queues insight; all other SQS APIs stay rpc:

  • SendMessage, SendMessageBatch: queue.publish, kind producer (default is client), named send <queue>
  • ReceiveMessage: queue.receive, named receive <queue>
  • DeleteMessage, DeleteMessageBatch: queue.settle, named settle <queue>

Adds the following attributes:

  • messaging.system (aws_sqs), messaging.operation.type and messaging.operation.name on the messaging APIs (all from OTel)
  • messaging.batch.message_count on batch APIs: submitted entries for send/delete, returned messages for receive
  • messaging.message.id on SendMessage, and on ReceiveMessage when exactly one message is returned
  • aws.sqs.queue.url and messaging.destination.name (queue name) on every API with QueueUrl; destination name also from QueueName and SourceArn

Issues

Resolves #7572 & PY-2803

@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Coverage Results 📊

✅ 58193 passed | ❌ 1 failed | ⏭️ 3058 skipped | Total: 61252 | Pass Rate: 95.01% | Execution Time: 162m 50s

✅ Patch coverage is 100.00% (70 of 70 changed executable lines covered; target 80%).
Project statement coverage is 90.37% (up 0.13 percentage points from base (86b2e98) to head (4d4d66c)).

Changed files with executable lines (5)
File Patch coverage Changed executable lines
sentry_sdk/integrations/boto3/_services/sqs.py 100.00% 49/49 covered; partial branches: 71, 81
sentry_sdk/consts.py 100.00% 8/8 covered
sentry_sdk/integrations/boto3/_instrumentation.py 100.00% 4/4 covered
sentry_sdk/integrations/boto3/_services/base.py 100.00% 2/2 covered
sentry_sdk/integrations/boto3/_utils.py 100.00% 7/7 covered
Coverage diff
@@            Coverage Diff             @@
##        master     #8137       +/-##
==========================================
+ Coverage    90.24%    90.37%    +0.13%
==========================================
  Files          203       199        -4
  Tracked lines     27114     21660     -5454
  Branches     10136      7518     -2618
==========================================
- Hits         24468     19575     -4893
+ Misses        2646      2085      -561
+ Partials      1540      1254      -286

Generated by Coverage Action

@pabloDeputter
pabloDeputter force-pushed the pablo/boto3-add-sqs-extension branch from 5f4f1d0 to 0f90423 Compare October 27, 2026 16:59
@pabloDeputter
pabloDeputter marked this pull request as ready for review October 29, 2026 09:03
@pabloDeputter
pabloDeputter requested a review from a team as a code owner October 29, 2026 09:03
sentrivana added a commit that referenced this pull request Oct 29, 2026
### Description

Removes `AwsCallContext.service_id_hyphenized` from
`sentry_sdk/integrations/boto3/_context.py`. It's still computed on
every call (`service_id.hyphenize()`), but on `major/3.0` nothing reads
it.

It had two readers. One was the `aws.service_id` span tag, which #7261
("Drop tags auto-set by the SDK") removed. The other was the span name:
#7800 ("align span names with OTel conventions") switched that to
`ctx.service_id`, so the hyphenized value lost its last reader. This
removes the slot, the attribute and the `hyphenize()` call.

**Evidence**
- On `major/3.0`, `git grep -w service_id_hyphenized` matches only the
three lines in `_context.py` that set it up.
- The open boto3 extension PRs against `major/3.0` (#8117 SQS, #8137
Lambda) don't reference it.

On `master`, `service_id_hyphenized` is still used for the span name and
the tag, so this only applies to `major/3.0`.

#### Issues
n/a

<!-- junior-request-attribution:start -->
via **Ivana Kellyer**.
<!-- junior-request-attribution:end -->

<!-- junior-session-footer:start -->
<!-- junior-conversation-id:slack%3AC0BTREZTS2Z%3A1793270559.842119 -->

--

[View Junior
Session](https://junior-prod.sentry.dev/conversations/slack%3AC0BTREZTS2Z%3A1793270559.842119)
[[Sentry]](https://sentry.sentry.io/explore/conversations/slack%3AC0BTREZTS2Z%3A1793270559.842119/?project=4510944073809921)

<!-- junior-session-footer:end -->

Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com>
Co-authored-by: Ivana Kellyer <ivana.kellyer@sentry.io>
attributes[SPANDATA.AWS_SQS_QUEUE_URL] = queue_url

# `https://sqs.eu-north-1.amazonaws.com/123456789012/orders` => ['123456789012', 'orders']
path = unquote(urlsplit(queue_url).path).strip("/").split("/")

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is queue_url validated by the library before it's handed to us through ctx.params?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, all parameters used in get_request_attributes() are yet to be validated. QueueUrl is also only checked to be a string, e.g. a non-string 1234 will raise an error at urlsplit and is captured at boto3/_instrumentation.py=180, so all SQS request attributes are dropped. Then get_span_name() at boto3/_instrumentation.py:183 will raise an error (if it's one of the API operations in _MESSAGING_OPERATIONS), which is captured again at boto3/_client.py:92, so no span is sent. Other operations still send the span, just without any SQS attributes.

@pabloDeputter pabloDeputter Oct 30, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did fix this partly in #8142, by adding another guard to get_span_name() at boto3/_instrumentation.py=183, so the span is sent, but still has no request attributes.

So I added some more precautions (c640799) in get_span_name() so that it doesn't take the operation type from attributes anymore, and the queue_url parsing is also guarded.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary for whoever resolves this thread. botocore only checks that QueueUrl is a string, so this PR no longer relies on its shape:

  • c640799 (this PR): get_span_name() takes the operation type from _MESSAGING_OPERATIONS instead of the request attributes, and the URL parsing in get_request_attributes() sits inside capture_internal_exceptions(). A malformed URL now only loses the destination name and account ID, and the span is still sent.
  • c73d9ba (feat(boto3): Add SNS extension #8142): a second capture_internal_exceptions() guard around get_span_name() in _instrumentation.py.

I merged major/3.0 into this branch today, and the boto3 tests still pass.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correction to my summary above: c73d9ba (the second capture_internal_exceptions() guard around get_span_name() in _instrumentation.py) is in this PR, not in #8142.

pabloDeputter and others added 2 commits October 30, 2026 13:55
- stop reading operation type from attributes in `get_span_name()` so that failure in `get_request_attributes()` can't make this fail as well
- wrap URL parsing in `get_request_attributes()` so failure just loses the attributes that come from url
…sqs-extension

Co-Authored-By: Johannes Daxböck <114897+dingsdax@users.noreply.github.com>

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Botocore validates params after our hook, so a bad Entries or ARN value
now only loses that attribute, not every SQS request attribute. Move the
batch error.type code to _utils so SNS can share it, and link the SQS
docs for BatchResultErrorEntry.

Co-Authored-By: Johannes Daxböck <114897+dingsdax@users.noreply.github.com>
@sentry-junior sentry-junior Bot changed the title feat(boto3): Add SQS extension feat(boto3): Add SQS extension Nov 9, 2026

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 438ded1. Configure here.

sentrivana and others added 3 commits November 11, 2026 10:56
on_success_response() ran outside capture_internal_exceptions(), so a
SendMessage or ReceiveMessage response without MessageId (e.g. from a
botocore Stubber or an SQS-compatible service) raised a KeyError into
the caller after a successful call and left the span open.

Co-Authored-By: Ivana Kellyer <131587164+sentrivana@users.noreply.github.com>
SQS messaging calls no longer follow the <ServiceID>.<Operation> name
and rpc op the guide describes for boto3 spans.

Co-Authored-By: Ivana Kellyer <131587164+sentrivana@users.noreply.github.com>
@sentrivana
sentrivana merged commit 42cd7a7 into major/3.0 Nov 11, 2026
153 of 155 checks passed
@sentrivana
sentrivana deleted the pablo/boto3-add-sqs-extension branch November 11, 2026 12:10
sentrivana added a commit that referenced this pull request Nov 11, 2026
### Description

Propagates the trace through SQS and SNS messages, so the consumer can
be linked back to the request that sent the message.

**Send side** (`SendMessage`, `SendMessageBatch`, `Publish`,
`PublishBatch`):
- Adds `sentry-trace` and `baggage` as `String` message attributes, on
the message or on each batch entry.
- Only when `trace_propagation_targets` match the endpoint.
- Skips a message if the caller already set these attributes, if it
would go over the 10-attribute limit, or if it would go over 256 KiB.
AWS would reject the message otherwise.
- Works on a copy of the params. The caller's dict isn't changed.

**Receive side** (`ReceiveMessage`):
- Adds `sentry-trace` and `baggage` to `MessageAttributeNames`, so SQS
returns them. Only when the span is recorded, and not when the caller
already asks for `All` or `.*`.
- Adds a span link to each distinct producer context in the received
messages, with `sentry.link.type: message_origin`.
- Reads the context from SQS message attributes, and from the JSON
envelope of an SNS notification without raw message delivery.

**Tests**: injection and extraction for single and batch calls, the
limits above, and a round-trip test: send a message, receive what was
injected, and check that the link points at the send span.

Stacked on #8142 (SNS), which is stacked on #8137 (SQS).

#### Issues
* resolves: PY-3008

<!-- junior-request-attribution:start -->
via @dingsdax.
<!-- junior-request-attribution:end -->

<!-- junior-session-footer:start -->
<!-- junior-conversation-id:slack%3AC0C9X9V5NMT%3A1793904671.980379 -->

--

[View
Session](https://junior-prod.sentry.dev/conversations/slack%3AC0C9X9V5NMT%3A1793904671.980379)
[[Sentry]](https://sentry.sentry.io/explore/conversations/slack%3AC0C9X9V5NMT%3A1793904671.980379/?project=4510944073809921)

<!-- junior-session-footer:end -->

---------

Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com>
Co-authored-by: Johannes Daxböck <114897+dingsdax@users.noreply.github.com>
Co-authored-by: Ivana Kellyer <ivana.kellyer@sentry.io>
Co-authored-by: Ivana Kellyer <131587164+sentrivana@users.noreply.github.com>
sentrivana added a commit that referenced this pull request Nov 12, 2026
)

### Description

Since #8137, #8142, and #8145, boto3 puts `sentry-trace` in SQS and SNS
message attributes, but nothing reads it on the Lambda side. This PR is
the smallest change that closes the loop for SQS- and SNS-triggered
functions. The Lambda integration now reads `sentry-trace` from each
record and adds a `message_origin` link from the invocation span to each
distinct producer span. This matches what the boto3 `ReceiveMessage`
span already does. Messages never become the invocation's parent.

It reads:
- SQS records: `messageAttributes["sentry-trace"].stringValue`
- SNS records: `Sns.MessageAttributes["sentry-trace"].Value`
- SNS → SQS without raw message delivery: the SNS envelope in the SQS
`body`, the same as boto3 does

Links are deduplicated, because a batch send gives many messages the
same producer span, and capped at 128, OTel's default, because Lambda
can deliver up to 10,000 SQS messages at once.

**Scope.** This is intentionally kept separate from the stretch stack.
It doesn't add `faas.trigger` (#8222) or `queue.process` spans (#8223),
and it doesn't touch `messaging.batch.message_count` (#8229). The
invocation span is otherwise unchanged. Synchronous `Invoke` propagation
through `ClientContext` is #8221. That PR changes the `continue_trace`
call and this one only adds `links=`, so the two are independent.

The link-building code repeats boto3's `_message_link` in a Lambda-event
shape: Lambda records use `messageAttributes` / `stringValue` / `body`,
while the boto3 response uses `MessageAttributes` / `StringValue` /
`Body`. Importing from the boto3 integration would make `aws_lambda`
depend on botocore.

**Testing.** `test_producer_links` sends SQS, SNS, and SNS-to-SQS events
to `BasicOkSpanStreaming`. It asserts the segment span's `links` and
that the segment span has no parent. It passes in CI's Cloud (Lambda)
jobs, and `ruff` and `mypy sentry_sdk` pass. `codecov/patch` is red
because the Lambda tests run in `sam local` containers, so their
coverage isn't counted.

#### Issues
* Part of #8081

<!-- junior-request-attribution:start -->
via **webb**, @sentrivana.
<!-- junior-request-attribution:end -->

<!-- junior-session-footer:start -->
<!-- junior-conversation-id:slack%3AC09M26BC6A1%3A1794211222.463239 -->

--

[View
Session](https://junior-prod.sentry.dev/conversations/slack%3AC09M26BC6A1%3A1794211222.463239)
[[Sentry]](https://sentry.sentry.io/explore/conversations/slack%3AC09M26BC6A1%3A1794211222.463239/?project=4510944073809921)

<!-- junior-session-footer:end -->

---------

Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com>
Co-authored-by: Alexander Webb <alexander.webb@sentry.io>
Co-authored-by: Ivana Kellyer <131587164+sentrivana@users.noreply.github.com>
sentrivana pushed a commit that referenced this pull request Nov 12, 2026
… span (#8229)

### Description

The Lambda span set `messaging.batch.message_count` to `1` for every
event that isn't a list, and to `len(event)` for a top-level list. An
SQS or SNS delivery is a dict with `Records`, so a batch of 10 messages
reported 1. HTTP calls and direct invocations reported a batch they
don't have.

This PR removes the attribute from the Lambda span. The count belongs on
messaging spans, as in the OTel messaging conventions:
- boto3 `ReceiveMessage` and batch-send spans already set it (#8137,
#8142).
- For SQS/SNS-triggered Lambdas, the `queue.process` span in #8223 sets
it.

The JS SDK doesn't set it on the Lambda span either. The attribute was
added in 3.0 (#7941) and never shipped in 2.x, so no released behavior
changes.

Replaces #8219 (same commits). GitHub never ran CI on #8219.

**Note for review:** this is narrower than PY-3015, which asks to keep
the attribute for SQS/SNS deliveries. @alexander-alderman-webb, please
confirm that's OK.

#### Issues
* resolves: PY-3015

<!-- junior-request-attribution:start -->
via @dingsdax.
<!-- junior-request-attribution:end -->

<!-- junior-session-footer:start -->
<!-- junior-conversation-id:slack%3AC0C9X9V5NMT%3A1793904671.980379 -->

--

[View
Session](https://junior-prod.sentry.dev/conversations/slack%3AC0C9X9V5NMT%3A1793904671.980379)
[[Sentry]](https://sentry.sentry.io/explore/conversations/slack%3AC0C9X9V5NMT%3A1793904671.980379/?project=4510944073809921)

<!-- junior-session-footer:end -->

---------

Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com>
Co-authored-by: Johannes Daxböck <114897+dingsdax@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants