Skip to content

feat(tracing): Record lost spans in client reports#3244

Merged
szokeasaurusrex merged 3 commits into
masterfrom
szokeasuaursrex/lost-spans
Jul 9, 2024
Merged

feat(tracing): Record lost spans in client reports#3244
szokeasaurusrex merged 3 commits into
masterfrom
szokeasuaursrex/lost-spans

Conversation

@szokeasaurusrex

@szokeasaurusrex szokeasaurusrex commented Jul 3, 2024

Copy link
Copy Markdown
Member

Resolves #3229
Resolves #3248

Note to reviewers: I have attempted to split the commits up into logical units, so you may wish to review the changes commit-by-commit.

@szokeasaurusrex
szokeasaurusrex changed the base branch from master to szokeasaurusrex/improve-transport-data-category-typing July 3, 2024 16:07
@szokeasaurusrex szokeasaurusrex changed the title szokeasuaursrex/lost spans feat(tracing): Record lost spans in client reports Jul 3, 2024
@szokeasaurusrex
szokeasaurusrex marked this pull request as draft July 3, 2024 16:08
@szokeasaurusrex

Copy link
Copy Markdown
Member Author

Marked as draft, still need to test this

@cleptric

cleptric commented Jul 3, 2024

Copy link
Copy Markdown
Member

We need to keep track of the amount of spans prior before_send_transaction and any event processors and calculate the delta afterwards, so we can report eventually dropped spans as well.

@codecov

codecov Bot commented Jul 3, 2024

Copy link
Copy Markdown

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 79.38%. Comparing base (ee84c81) to head (f33be4a).

Additional details and impacted files
@@                                       Coverage Diff                                       @@
##           szokeasaurusrex/use-capture_record_lost_event_calls-fixture    #3244      +/-   ##
===============================================================================================
+ Coverage                                                        79.35%   79.38%   +0.03%     
===============================================================================================
  Files                                                              132      132              
  Lines                                                            14227    14243      +16     
  Branches                                                          2987     2991       +4     
===============================================================================================
+ Hits                                                             11290    11307      +17     
+ Misses                                                            2092     2091       -1     
  Partials                                                           845      845              
Files Coverage Δ
sentry_sdk/client.py 79.85% <100.00%> (+0.53%) ⬆️
sentry_sdk/tracing.py 73.60% <100.00%> (+0.05%) ⬆️
sentry_sdk/transport.py 81.31% <100.00%> (+0.58%) ⬆️

@szokeasaurusrex szokeasaurusrex linked an issue Jul 4, 2024 that may be closed by this pull request
@szokeasaurusrex

Copy link
Copy Markdown
Member Author

We need to keep track of the amount of spans prior before_send_transaction and any event processors and calculate the delta afterwards, so we can report eventually dropped spans as well.

@cleptric, does this mean that we also need to report dropped spans in the before_send_transaction even if the transaction gets sent to Sentry?

For example, in the situation below, do we report any dropped spans to Sentry?

Transaction diagram

And for clarity, in the following situation where the transaction gets dropped after before_send/before_send_transaction, e.g. in the Transport, how many spans do we report as dropped, 4 or 6?

Transaction diagram-2

@cleptric

cleptric commented Jul 4, 2024

Copy link
Copy Markdown
Member

The first example reports two dropped spans, the second example reports six dropped spans and one dropped transaction.

@szokeasaurusrex
szokeasaurusrex force-pushed the szokeasaurusrex/improve-transport-data-category-typing branch 2 times, most recently from f098e6a to 854b46c Compare July 4, 2024 11:40
@szokeasaurusrex
szokeasaurusrex force-pushed the szokeasuaursrex/lost-spans branch from 1cc0144 to 0ce0523 Compare July 4, 2024 11:41
@szokeasaurusrex szokeasaurusrex linked an issue Jul 4, 2024 that may be closed by this pull request
@szokeasaurusrex
szokeasaurusrex force-pushed the szokeasuaursrex/lost-spans branch 2 times, most recently from cc4291c to 3c62d91 Compare July 5, 2024 12:06
@szokeasaurusrex
szokeasaurusrex marked this pull request as ready for review July 5, 2024 12:07
Comment thread sentry_sdk/tracing.py Outdated

client.transport.record_lost_event(reason, data_category="transaction")
# No spans are discarded here because they were never recorded in the first place
client.transport.record_lost_transaction(reason, span_count=0)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just realized I still need to test this codepath

@szokeasaurusrex
szokeasaurusrex force-pushed the szokeasuaursrex/lost-spans branch from 3c62d91 to 7a142ef Compare July 8, 2024 08:52
@szokeasaurusrex
szokeasaurusrex force-pushed the szokeasaurusrex/improve-transport-data-category-typing branch from 854b46c to 9b6a718 Compare July 8, 2024 08:52
Base automatically changed from szokeasaurusrex/improve-transport-data-category-typing to master July 8, 2024 11:59
szokeasaurusrex added a commit that referenced this pull request Jul 8, 2024
`capture_record_lost_event_calls` replaces the `capture_client_reports` fixture. The fixture records calls to `Transport.record_lost_event` by noting the arguments passed to each call.

This change is being introduced in preparation for #3244, which changes `Transport.record_lost_event`'s signature and behavior.
@szokeasaurusrex
szokeasaurusrex marked this pull request as draft July 8, 2024 14:16
@szokeasaurusrex
szokeasaurusrex force-pushed the szokeasuaursrex/lost-spans branch from 7a142ef to 4c8dfff Compare July 8, 2024 15:02
szokeasaurusrex added a commit that referenced this pull request Jul 8, 2024
Make the `report["discarded_events"]` assertion logic (in `test_data_category_limits_reporting`) not rely on the ordering of events or any sorting logic. Done in preparation of #3244, where the sorting logic cannot be relied on anymore, since the same number of spans will be discarded as transactions.
@szokeasaurusrex
szokeasaurusrex force-pushed the szokeasuaursrex/lost-spans branch from 4c8dfff to fb4822a Compare July 8, 2024 15:47
szokeasaurusrex added a commit that referenced this pull request Jul 9, 2024
…calls`

Replace custom `record_lost_event` call capturing logic in `test_sampling.py` with the `capture_record_lost_event_calls` Pytest fixture. This change will simplify implementation of #3244.
@szokeasaurusrex
szokeasaurusrex force-pushed the szokeasuaursrex/lost-spans branch from 07d9da7 to f33be4a Compare July 9, 2024 09:09
@szokeasaurusrex
szokeasaurusrex changed the base branch from master to szokeasaurusrex/use-capture_record_lost_event_calls-fixture July 9, 2024 09:09
@szokeasaurusrex
szokeasaurusrex force-pushed the szokeasuaursrex/lost-spans branch 2 times, most recently from b98f150 to f8d866f Compare July 9, 2024 10:34
Base automatically changed from szokeasaurusrex/use-capture_record_lost_event_calls-fixture to master July 9, 2024 10:41
@szokeasaurusrex
szokeasaurusrex marked this pull request as ready for review July 9, 2024 10:41
Also, update existing transport tests so they pass against the changes introduced in this commit.

Resolves #3229
  - Add test for `record_lost_event` method's new `quantity` parameter
  - Add test for `record_lost_event` when passed a transaction item
@szokeasaurusrex
szokeasaurusrex force-pushed the szokeasuaursrex/lost-spans branch from f8d866f to 79e8970 Compare July 9, 2024 10:43

@antonpirker antonpirker left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice!

@szokeasaurusrex
szokeasaurusrex merged commit 79e8970 into master Jul 9, 2024
@szokeasaurusrex
szokeasaurusrex deleted the szokeasuaursrex/lost-spans branch July 9, 2024 14:23
arjennienhuis pushed a commit to arjennienhuis/sentry-python that referenced this pull request Sep 30, 2024
`capture_record_lost_event_calls` replaces the `capture_client_reports` fixture. The fixture records calls to `Transport.record_lost_event` by noting the arguments passed to each call.

This change is being introduced in preparation for getsentry#3244, which changes `Transport.record_lost_event`'s signature and behavior.
arjennienhuis pushed a commit to arjennienhuis/sentry-python that referenced this pull request Sep 30, 2024
Make the `report["discarded_events"]` assertion logic (in `test_data_category_limits_reporting`) not rely on the ordering of events or any sorting logic. Done in preparation of getsentry#3244, where the sorting logic cannot be relied on anymore, since the same number of spans will be discarded as transactions.
arjennienhuis pushed a commit to arjennienhuis/sentry-python that referenced this pull request Sep 30, 2024
…calls`

Replace custom `record_lost_event` call capturing logic in `test_sampling.py` with the `capture_record_lost_event_calls` Pytest fixture. This change will simplify implementation of getsentry#3244.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improve record_lost_event parameter documentation Report dropped spans in client reports

3 participants