Skip to content

Add an ability to filter and enrich tracing activities#5853

Merged
vonzshik merged 9 commits into
mainfrom
tracind-add-filter-and-enrich
Nov 9, 2024
Merged

Add an ability to filter and enrich tracing activities#5853
vonzshik merged 9 commits into
mainfrom
tracind-add-filter-and-enrich

Conversation

@vonzshik

@vonzshik vonzshik commented Sep 26, 2024

Copy link
Copy Markdown
Contributor

Closes #4192
Closes #4250
Closes #4228
Closes #4245
Closes #5595

@vonzshik vonzshik requested a review from roji as a code owner September 26, 2024 10:11
@vonzshik vonzshik force-pushed the tracind-add-filter-and-enrich branch from 8f3a209 to 43a47b9 Compare October 9, 2024 09:52
Comment thread src/Npgsql/NpgsqlTracingOptions.cs Outdated
/// <summary>
/// Gets or sets a function that provides a span's name on a per <see cref="NpgsqlCommand"/> basis.
/// </summary>
public Func<NpgsqlCommand, string?>? ProvideSpanNameForNpgsqlCommand { get; set; }

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unless NpgsqlCommand also exposes a property where I could store a custom span name (in which case this function wouldn't be needed in the first place) at the moment of command creation, this isn't going to be particularly helpful in my opinion. Yes, deriving the span name by using the first N characters of the command text would be trivial, but what if I want to use some label - Select customers with payments due, Verify user account, etc.? I'd have to maintain some global dictionary for mapping command texts to these labels, which would be extremely annoying.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really, you can also use Baggage.SetBaggage/Baggage.GetBaggage or even a custom AsyncLocal to pass required info. I'm personally a bit hesitant to add a new property because it's not going to be visible if you're using ADO.NET interfaces +NpgsqlCommand is probably the most allocated object we have and another property there will just make it even more heavy.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the callback isn't executed completely out of band with respect to the created activity, then that could work.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the callback isn't executed completely out of band with respect to the created activity, then that could work.

Yep, it's called just before we start doing any possible async actions (the same is true for multiplexing) so it definitely should work.

NpgsqlEventSource.Log.CommandStart(CommandText);
startTimestamp = connector.DataSource.MetricsReporter.ReportCommandStart();
TraceCommandStart(connector.Settings);
TraceCommandEnrich(connector);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re adding a property to NpgsqlCommand, see related conversation in SqlClient (dotnet/SqlClient#2937, dotnet/SqlClient#2925).

I agree that we should at least start with the callback without adding anything to NpgsqlCommand (and people can e.g. use AsyncLocal) - we can always do that later if we want.

@vonzshik vonzshik force-pushed the tracind-add-filter-and-enrich branch from 43a47b9 to 62749c8 Compare October 21, 2024 10:16
@bunkrur

bunkrur commented Nov 6, 2024

Copy link
Copy Markdown

I will give someone a pretty giftcard for this PR

@vonzshik

vonzshik commented Nov 6, 2024

Copy link
Copy Markdown
Contributor Author

@bunkrur I'll aggressively ping other team members offline so we merge this just before the release)

@bunkrur

bunkrur commented Nov 6, 2024

Copy link
Copy Markdown

This could also fix #4245 potentially

I added a PR for this branch to include two options to disable the generation of expensive events on spans for time-to-first-read and time-to-last-read, as its extremely expensive if youre using an observability vendor.

#5912

@vonzshik

vonzshik commented Nov 7, 2024

Copy link
Copy Markdown
Contributor Author

This could also fix #4245 potentially

It sure could. It could also fix other issues we have with tracing/metrics/logging. The reason why I went with a relatively small pr is:

  1. It would be better if we (npgsql team) first agree on the way we're going to support passing settings. In this pr it's done through a static field, which of course doesn't allow having different settings for multiple data sources.
  2. It's much easier to review a small pr fixing a few things rather than a big one fixing many (on this depends whether we'll be able to merge this for 9.0 or not).

@roji roji left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vonzshik finally got around to reviewing those - let's try to get it in for 9 :)

Comment thread src/Npgsql.OpenTelemetry/TracerProviderBuilderExtensions.cs Outdated
Comment thread src/Npgsql/NpgsqlActivitySource.cs Outdated
Comment thread src/Npgsql/NpgsqlTracingOptions.cs Outdated
Comment thread src/Npgsql/NpgsqlTracingOptions.cs Outdated
/// <summary>
/// Gets or sets a function that provides a span's name on a per <see cref="NpgsqlCommand"/> basis.
/// </summary>
public Func<NpgsqlCommand, string?>? ProvideSpanNameForNpgsqlCommand { get; set; }

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re adding a property to NpgsqlCommand, see related conversation in SqlClient (dotnet/SqlClient#2937, dotnet/SqlClient#2925).

I agree that we should at least start with the callback without adding anything to NpgsqlCommand (and people can e.g. use AsyncLocal) - we can always do that later if we want.

Comment thread src/Npgsql/NpgsqlTracingOptions.cs Outdated
Comment thread src/Npgsql/NpgsqlCommand.cs Outdated
Comment thread src/Npgsql/NpgsqlCommand.cs Outdated
Comment thread src/Npgsql/NpgsqlTracingOptions.cs Outdated

@roji roji left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @vonzshik!

Comment thread src/Npgsql/NpgsqlTracingOptions.cs Outdated
@vonzshik vonzshik enabled auto-merge (squash) November 9, 2024 06:17
@vonzshik vonzshik merged commit 7949c45 into main Nov 9, 2024
@vonzshik vonzshik deleted the tracind-add-filter-and-enrich branch November 9, 2024 06:23
@bunkrur

bunkrur commented Nov 9, 2024

Copy link
Copy Markdown

Stoked on this thank you

roji added a commit to roji/Npgsql that referenced this pull request Nov 12, 2024
roji added a commit that referenced this pull request Nov 16, 2024
Continues #5853

Co-authored-by: Nikita Kazmin <vonzshik@gmail.com>
Co-authored-by: Nino Floris <mail@ninofloris.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

4 participants