Skip to content

Prepare for a release with telemetry on by default#717

Merged
samikshya-db merged 10 commits intomainfrom
samikshya-chand_data/release
Dec 4, 2025
Merged

Prepare for a release with telemetry on by default#717
samikshya-db merged 10 commits intomainfrom
samikshya-chand_data/release

Conversation

@samikshya-db
Copy link
Contributor

@samikshya-db samikshya-db commented Dec 1, 2025 •
edited
Loading

What type of PR is this?

  • Refactor
  • Feature
  • Bug Fix
  • Other : Release

Description

  • Make telemetry is ON by default to track connection stats. This strictly excludes PII, query text, and results.
  • Prepare for release 4.2.2
  • Also updated the documentation to state that 3.8 is not supported.
  • Updated E2E tests for telemetry to fix the checks (as the telemetry is ON by default now)
  • The tests are now grouped by host ID, so there were additional fixes that were made to make them serial runs rather than parallelise them - to ensure testing is done thoroughly and correctly.

How is this tested?

  • Unit tests
  • E2E Tests
  • Manually
  • N/A

Related Tickets & Documents

Manually verified Telemetry use-cases by running it locally and checking on the dashboard too

Screenshot 2025-12-04 at 4 18 50 PM

@github-actions
Copy link

github-actions bot commented Dec 1, 2025

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

@github-actions
Copy link

github-actions bot commented Dec 1, 2025

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

@samikshya-db samikshya-db marked this pull request as ready for review December 1, 2025 12:52
@github-actions
Copy link

github-actions bot commented Dec 1, 2025

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

jprakash-db
jprakash-db approved these changes Dec 1, 2025
View reviewed changes
Copy link
Contributor

@jprakash-db jprakash-db left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, 🚀 🚀

vikrantpuppala
vikrantpuppala approved these changes Dec 1, 2025
View reviewed changes
Copy link
Contributor

@vikrantpuppala vikrantpuppala left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ty! please fix changelog style

CHANGELOG.md Outdated
- Circuit breaker changes using pybreaker by @nikhilsuri-db in https://github.com/databricks/databricks-sql-python/pull/705
- perf: Optimize telemetry latency logging to reduce overhead by @samikshya-db in https://github.com/databricks/databricks-sql-python/pull/715
- basic e2e test for force telemetry verification by @nikhilsuri-db in https://github.com/databricks/databricks-sql-python/pull/708
- Telemetry is ON by default to track connection stats. (Note : This strictly excludes PII, query text, and results) by @samikshya-db in https://github.com/databricks/databricks-sql-python/pull/717
Copy link
Contributor

@vikrantpuppala vikrantpuppala Dec 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you please format inline with previous changelogs?

@samikshya-db samikshya-db force-pushed the samikshya-chand_data/release branch from 72486d9 to 70022b5 Compare December 3, 2025 15:25
@samikshya-db
Prepare for a release with telemetry on by default
cd49dab 
Signed-off-by: samikshya-chand_data <samikshya.chand@databricks.com>
@samikshya-db
Make edits
a04d6da 
Signed-off-by: samikshya-chand_data <samikshya.chand@databricks.com>
@samikshya-db
Update version
0eda9f0 
Signed-off-by: samikshya-chand_data <samikshya.chand@databricks.com>
@samikshya-db
Fix CHANGELOG formatting to match previous style
ad01fc2 
Signed-off-by: samikshya-chand_data <samikshya.chand@databricks.com>
@samikshya-db samikshya-db force-pushed the samikshya-chand_data/release branch from 70022b5 to ad01fc2 Compare December 3, 2025 15:28
@samikshya-db
Fix telemetry e2e tests for default-enabled behavior
38230c6 
- Update test expectations to reflect telemetry being enabled by default
- Add feature flags cache cleanup in teardown to prevent state leakage between tests
- This ensures each test runs with fresh feature flag state
@github-actions
Copy link

github-actions bot commented Dec 3, 2025

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

@github-actions
Copy link

github-actions bot commented Dec 3, 2025

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

@github-actions
Copy link

github-actions bot commented Dec 4, 2025

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

nikhilsuri-db
nikhilsuri-db approved these changes Dec 4, 2025
View reviewed changes
Copy link
Contributor

@nikhilsuri-db nikhilsuri-db left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@samikshya-db
Mark telemetry e2e tests as serial - must not run in parallel
e898be6 
Root cause: Telemetry tests share host-level client across pytest-xdist workers,
causing test isolation issues with patches. Tests pass serially but fail with -n auto.

Solution: Add @pytest.mark.serial marker. CI needs to run these separately without -n auto.
@github-actions
Copy link

github-actions bot commented Dec 4, 2025

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

@samikshya-db
Split test execution to run serial tests separately
98aa382 
Telemetry e2e tests must run serially due to shared host-level
telemetry client across pytest-xdist workers. Running with -n auto
causes test isolation issues where futures aren't properly captured.

Changes:
- Run parallel tests with -m 'not serial' -n auto
- Run serial tests with -m 'serial' without parallelization
- Use --cov-append for serial tests to combine coverage
- Mark telemetry e2e tests with @pytest.mark.serial
- Update test expectations for default telemetry behavior
- Add feature flags cache cleanup in test teardown
@github-actions
Copy link

github-actions bot commented Dec 4, 2025

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

@samikshya-db
Mark telemetry e2e tests as serial - must not run in parallel
5921b08 
The concurrent telemetry e2e test globally patches telemetry methods
to capture events. When run in parallel with other tests via pytest-xdist,
it captures telemetry events from other concurrent tests, causing
assertion failures (expected 60 events, got 88).

All telemetry e2e tests must run serially to avoid cross-test
interference with the shared host-level telemetry client.
@github-actions
Copy link

github-actions bot commented Dec 4, 2025

Thanks for your contribution! To satisfy the DCO policy in our contributing guide every commit message must include a sign-off message. One or more of your commits is missing this message. You can reword previous commit messages with an interactive rebase (git rebase -i main).

@samikshya-db samikshya-db merged commit d2ae1e8 into main Dec 4, 2025
29 of 30 checks passed
@samikshya-db samikshya-db deleted the samikshya-chand_data/release branch December 4, 2025 10:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vikrantpuppala vikrantpuppala vikrantpuppala approved these changes

@jprakash-db jprakash-db jprakash-db approved these changes

@nikhilsuri-db nikhilsuri-db nikhilsuri-db approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@samikshya-db @vikrantpuppala @jprakash-db @nikhilsuri-db