Add @pytest.mark.xdist_group to telemetry test classes to ensure they
run sequentially on the same worker when using pytest-xdist (-n auto).

Root cause: Tests marked @pytest.mark.serial were still being
parallelized in CI because pytest-xdist doesn't respect custom markers
by default. With host-level telemetry batching (PR #718), tests
running in parallel would share the same TelemetryClient and interfere
with each other's event counting, causing test_concurrent_queries_sends_telemetry
to see 88 events instead of the expected 60.

The xdist_group marker ensures all tests in the "serial_telemetry"
group run on the same worker sequentially, preventing state interference.

Signed-off-by: Claude Sonnet 4.5 <noreply@anthropic.com>

* Fix #729 and #731: Telemetry lifecycle management

Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>

* Address review comments: revert timeout and telemetry_enabled changes

Per reviewer feedback on PR #734:

1. Revert timeout from 30s back to 900s (line 299)
   - Reviewer noted that with wait=False, timeout is not critical
   - The async nature and wait=False handle the exit speed

2. Revert telemetry_enabled parameter back to True (line 734)
   - Reviewer noted this is redundant given the early return
   - If enable_telemetry=False, we return early (line 729)
   - Line 734 only executes when enable_telemetry=True
   - Therefore using the parameter here is unnecessary

These changes address the reviewer's valid technical concerns while
keeping the core fixes intact:
- wait=False for non-blocking shutdown (critical for Issue #729)
- Early return when enable_telemetry=False (critical for Issue #729)
- All Issue #731 fixes (null-safety, __del__, documentation)

Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>

* Fix Black formatting violations

Apply Black formatting to files modified in previous commits:
- src/databricks/sql/common/unified_http_client.py
- src/databricks/sql/telemetry/telemetry_client.py

Changes are purely cosmetic (quote style consistency).

Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>

* Fix CI test failure: Prevent parallel execution of telemetry tests

Add @pytest.mark.xdist_group to telemetry test classes to ensure they
run sequentially on the same worker when using pytest-xdist (-n auto).

Root cause: Tests marked @pytest.mark.serial were still being
parallelized in CI because pytest-xdist doesn't respect custom markers
by default. With host-level telemetry batching (PR #718), tests
running in parallel would share the same TelemetryClient and interfere
with each other's event counting, causing test_concurrent_queries_sends_telemetry
to see 88 events instead of the expected 60.

The xdist_group marker ensures all tests in the "serial_telemetry"
group run on the same worker sequentially, preventing state interference.

Signed-off-by: Claude Sonnet 4.5 <noreply@anthropic.com>

* Fix telemetry test fixtures: Clean up state before AND after tests

Modified telemetry_setup_teardown fixtures to clean up
TelemetryClientFactory state both BEFORE and AFTER each test, not just
after. This prevents leftover state from previous tests (pending events,
active executors) from interfering with the current test.

Root cause: In CI with sequential execution on the same worker, if a
previous test left pending telemetry events in the executor, those
events could be captured by the next test's mock, causing inflated
event counts (88 instead of 60).

Now ensures complete isolation between tests by resetting all shared
state before each test starts.

Signed-off-by: Claude Sonnet 4.5 <noreply@anthropic.com>

* Fix CI test failure: Clear _flush_event between tests

The _flush_event threading.Event was never cleared after stopping the
flush thread, remaining in "set" state. This caused timing issues in
subsequent tests where the Event was already signaled, triggering
unexpected flush behavior and causing extra telemetry events to be
captured (88 instead of 60).

Now explicitly clear the _flush_event flag in both setup (before test)
and teardown (after test) to ensure clean state isolation between tests.

This explains why CI consistently got 88 events - the flush_event from
previous tests triggered additional flushes during test execution.

Signed-off-by: Claude Sonnet 4.5 <noreply@anthropic.com>

* Add debug workflow and output to diagnose CI test failure

1. Created new workflow 'test-telemetry-only.yml' that runs only the
   failing telemetry test with -n auto, mimicking real CI but much faster

2. Added debug output to test showing:
   - Client-side captured events
   - Number of futures/batches
   - Number of server responses
   - Server-reported successful events

This will help identify why CI gets 88 events vs local 60 events.

Signed-off-by: Claude Sonnet 4.5 <noreply@anthropic.com>

* Fix workflow: Add krb5 system dependency

The workflow was failing during poetry install due to missing krb5
system libraries needed for kerberos dependencies.

Signed-off-by: Claude Sonnet 4.5 <noreply@anthropic.com>

* Fix xdist_group: Add --dist=loadgroup to pytest commands

The @pytest.mark.xdist_group markers were being ignored because
pytest-xdist uses --dist=load by default, which doesn't respect groups.

With --dist=loadgroup, tests in the same xdist_group run sequentially
on the same worker, preventing telemetry state interference between
tests.

This is the ROOT CAUSE of the 88 vs 60 events issue - tests were
running in parallel across workers instead of sequentially on one
worker as intended.

Signed-off-by: Claude Sonnet 4.5 <noreply@anthropic.com>

* Add aggressive flush before test to prevent event interference

CI shows 72 events instead of 60. Debug output reveals:
- Client captured: 60 events (correct)
- Server received: 72 events across 2 batches

The 12 extra events accumulate in the timing window between fixture
cleanup and mock setup. Other tests (like circuit breaker tests not in
our xdist_group) may be sending telemetry concurrently.

Solution: Add an explicit flush+shutdown RIGHT BEFORE setting up the
mock to ensure a completely clean slate with zero buffered events.

Signed-off-by: Claude Sonnet 4.5 <noreply@anthropic.com>

* Split workflow: Isolate telemetry tests in separate job

To prevent interference from other e2e tests, split into two jobs:

Job 1 (run-non-telemetry-tests):
- Runs all e2e tests EXCEPT telemetry tests
- Uses -n auto for parallel execution

Job 2 (run-telemetry-tests):
- Runs ONLY telemetry tests
- Depends on Job 1 completing (needs: run-non-telemetry-tests)
- Fresh Python process = complete isolation
- No ambient telemetry from other tests

This eliminates the 68 vs 60 event discrepancy by ensuring
telemetry tests run in a clean environment with zero interference.

Signed-off-by: Claude Sonnet 4.5 <noreply@anthropic.com>

* Fix workflows: Add krb5 deps and cleanup debug code

Changes across multiple workflows:

1. integration.yml:
   - Add krb5 system dependency to telemetry job
   - Fixes: krb5-config command not found error during poetry install

2. code-coverage.yml:
   - Add krb5 system dependency
   - Split telemetry tests into separate step for isolation
   - Maintains coverage accumulation with --cov-append

3. publish-test.yml:
   - Add krb5 system dependency for consistent builds

4. test_concurrent_telemetry.py:
   - Remove debug print statements

5. Delete test-telemetry-only.yml:
   - Remove temporary debug workflow

All workflows now have proper telemetry test isolation and
required system dependencies for kerberos packages.

Signed-off-by: Claude Sonnet 4.5 <noreply@anthropic.com>

* Fix publish-test.yml: Update Python 3.9 -> 3.10

Poetry 2.3.2 installation fails with Python 3.9:
  Installing Poetry (2.3.2): An error occurred.

Other workflows use Python 3.10 and work fine. Updating to match
ensures consistency and avoids Poetry installation issues.

Signed-off-by: Claude Sonnet 4.5 <noreply@anthropic.com>

* Fix integration workflow: Remove --dist=loadgroup from non-telemetry tests

- Remove --dist=loadgroup from non-telemetry job (only needed for telemetry)
- Remove test_telemetry_e2e.py from telemetry job (was skipped before)
- This should fix test_uc_volume_life_cycle failure caused by changed test distribution

* Fix code-coverage workflow: Remove test_telemetry_e2e.py from coverage tests

- Only run test_concurrent_telemetry.py in isolated telemetry step
- test_telemetry_e2e.py was excluded in original workflow, keep it excluded

* Fix publish-test workflow: Remove cache conditional

- Always run poetry install (not just on cache miss)
- Ensures fresh install with system dependencies (krb5)
- Matches pattern used in integration.yml

* Fix publish-test.yml: Remove duplicate krb5 install, restore cache conditional

- Remove duplicate system dependencies step
- Restore cache conditional to match main branch
- Keep Python 3.10 (our change from 3.9)

* Fix code-coverage: Remove serial tests step

- All serial tests are telemetry tests (test_concurrent_telemetry.py and test_telemetry_e2e.py)
- They're already run in the isolated telemetry step
- Running -m serial with --ignore on both files results in 0 tests (exit code 5)

---------

Signed-off-by: Madhavendra Rathore <madhavendra.rathore@databricks.com>
Signed-off-by: Claude Sonnet 4.5 <noreply@anthropic.com>