test(e2e): Add a Mistral E2E app covering both instrumentation paths - #24378
Draft
RulaKhaled wants to merge 1 commit into
Draft
RulaKhaled wants to merge 1 commit into
RulaKhaled wants to merge 1 commit into
Conversation
RulaKhaled
added this pull request to stack #24379
September 14, 2026 18:53
Contributor
size-limit report 📦
|
RulaKhaled
force-pushed
the
test/mistral-e2e-app
branch
from
September 14, 2026 19:18
4920054 to
c90fff4
Compare
Follows `node-eve` and `node-mastra`: the Mistral SDK talks to a real endpoint
through `E2E_OPENROUTER_API_KEY` rather than a mock, and the app is marked
`sentryTest.optional` so it lands on the job where that secret is set.
The suite runs twice the way `node-mastra` splits dev and prod, so both
instrumentation paths are covered by the same assertions:
production - `dist/app.cjs`, whose Mistral, dataloader and express copies were
transformed at build time by `sentryEsbuildPlugin`
development - unbundled ESM behind the runtime `--import` hook
What it covers:
- gen_ai spans for streaming and non-streaming calls, and the shapes of
`gen_ai.response.text` and `gen_ai.output.messages`
- a rejected model id reaches Sentry as an error, the gen_ai span is marked
errored and records nothing from a response, and the error shares its trace
- a manual span nests under the request span, and the gen_ai span directly under
the manual one, for both streaming and not
- streams drained through `tee()` and `pipeThrough()`, which take their reader
from internal slots and so produced no span before the accompanying fix
- dataloader spans in the same trace as gen_ai spans, which also covers a
CommonJS and an ESM-only module through the same transform in one process
Assertions avoid anything a live model decides. Token counts are checked as
positive numbers and response text for shape, while origin, provider, operation
name, the `chat {model}` naming rule, the stream flags and every parent/child
relationship are exact.
`Sentry.init` registers the runtime hook unless `enableRuntimeChannelInjection`
is false, which the bundled mode sets, so the production run also asserts the
injected channel names are present in the built file. Without that a passing
production run would not distinguish build-time instrumentation from a silent
runtime fallback.
OpenRouter serves an OpenAI-compatible `/v1/chat/completions`, which is what
`chat.complete` and `chat.stream` post to, and the SDK's response schemas accept
it: `usage` carries a `catchall` and `finish_reason` is an open enum.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
RulaKhaled
force-pushed
the
test/mistral-e2e-app
branch
from
September 14, 2026 19:52
c90fff4 to
2fc5bc8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #24243. Adds an E2E test application for the Mistral AI integration.
The app runs one Express server twice, side by side, and asserts the same behaviour against each:
devserves unbundled ESM through the runtime--importhook,prodserves an esbuild bundle whose Mistral,dataloaderandexpresscopies were transformed at build time bysentryEsbuildPlugin. Every assertion therefore runs against both orchestrion paths, which is the surface most likely to diverge for a provider that ships ESM-only.What it covers:
gen_aispans for streaming and non-streaming calls, including request, response and usage attributes and thegen_ai.response.text/gen_ai.output.messagesshapes.gen_aispan is marked errored and records nothing from a response, and the error shares the span's trace.gen_aispan nests directly under the manual one.dataloaderspans land in the same trace asgen_aispans.Decisions worth knowing about:
Two event proxies, one per mode. Envelopes from the two servers would otherwise interleave and a test could satisfy its assertion with the other mode's spans.
The prod bundle is checked for its injected channel names.
Sentry.initregisters the runtime injection hook unlessenableRuntimeChannelInjectionisfalse, which the prod mode sets. Without that flag and that check, a passing[prod]result would not distinguish build-time instrumentation from a silent runtime fallback.Requests to
/chat-errorcarry a per-request model.collectStreamedSpansresolves with the first trace matching its predicate, and several tests exercise that route, so each test names its own trace through the model rather than the route.Ancestry, not direct parentage, for the manual-span-under-request-span check. Express contributes its own middleware and request-handler spans in between. The direct-parent assertion is kept where it is meaningful: the
gen_aispan directly under the manual span.dataloaderis CommonJS and@mistralai/mistralaiis ESM-only, so the co-instrumentation test also covers both module formats going through the same transform in one process. The Mistral API is mocked in-process, so the app needs no key and no network. Startup avoids top-level await so the same source bundles to CommonJS for the prod build.Retarget to
developonce #24243 merges.🤖 Generated with Claude Code