Commits on Aug 13, 2026

1. Lazy-load Apache Thrift so SEA/kernel paths never import it …

   The connector declared `thrift` as a hard dependency and imported it
   eagerly the moment `connect()` loaded `databricks.sql.client` -- 16
   thrift modules, including the top-level `thrift` package -- regardless
   of whether the caller selected `use_sea=True` or `use_kernel=True`.
   Build systems that vendor their own `thrift` (e.g. Meta's Buck) then hit
   a namespace collision even on the SEA / kernel code paths, which never
   speak Thrift on the wire.
   
   This makes the thrift import lazy without changing any public API or
   install semantics: `thrift` stays a base dependency, but it is only
   imported when the Thrift backend is actually used.
   
   Mechanism, per module on the connect/execute chain:
   - Annotation-only uses (the cloud-fetch download manager/downloader,
     the DatabricksClient ABC's `execute_command`, and the SEA/kernel
     `TSparkParameter` annotations): add `from __future__ import
     annotations` and move the thrift import under `TYPE_CHECKING`, so the
     annotations are never evaluated at runtime.
   - Runtime uses on Thrift-only paths (`from_thrift_state`, the queue
     factory's `TSparkRowSetType`, `TProtocolVersion`, the SEA
     `_convert_to_thrift_link` construction, and every `TSparkParameter*`
     construction in parameters/native): move the import into the function
     body.
   - Backend selection in `Session.open`: resolve `ThriftDatabricksClient`
     / `SeaDatabricksClient` via a module-level `__getattr__` (PEP 562) and
     reference them through the module namespace, so thrift is imported
     only on the branch that needs it. This also preserves the
     `patch("...session.ThriftDatabricksClient")` test seam.
   - Preserve the historical re-exports `parameters.native.TSparkParameter*`
     and `client.ThriftDatabricksClient` via lazy `__getattr__` so existing
     importers (and tests) keep working without importing thrift at load.
   
   Add tests/unit/test_lazy_thrift_import.py, which imports the connector
   and each non-Thrift backend in a fresh subprocess and asserts the
   top-level `thrift` package is absent from sys.modules (and, conversely,
   that the Thrift backend still imports it). This locks the invariant --
   a single stray module-level thrift import re-poisons the whole path.
   
   Verified empirically: importing `databricks.sql.client` and the SEA /
   kernel backend modules loads zero thrift modules, while the Thrift
   backend still loads thrift. Full unit suite passes (the two failures
   present also reproduce unchanged on main: a kernel test-ordering issue
   and a realkernel-marked test).
   
   Co-authored-by: Isaac

   

   vikrantpuppala committed Aug 13, 2026
   Configuration menu

   • View commit details
   • Copy full SHA for 5ea26c2
   • Browse repository at this point

   Copy the full SHA

   5ea26c2 View commit details

   Browse the repository at this point in the history

2. Address PR review: robust guard test + preserve client.py re-exports …

   Follow-up to the lazy-thrift change addressing two review-bot findings.
   
   Medium (test robustness / CI false-positive): the guard test's child
   subprocess exited with code 1 on any failure, which is also Python's
   generic uncaught-exception code -- so an import failure was
   indistinguishable from "thrift was imported". This caused a false CI
   failure: in the "default deps" job (no pyarrow), importing
   `kernel.type_mapping` raises `ModuleNotFoundError: pyarrow` (exit 1),
   which the test misread as a thrift leak. It was also a false *pass* risk
   for the Thrift-backend counterpart test. The child now emits dedicated
   sentinel exit codes only after the import completes, captures stderr,
   and reports import failure separately; the parametrized test skips
   modules that can't import due to a missing optional dependency (rather
   than failing), while still detecting a genuine thrift leak.
   
   Low (back-compat): `client.py`'s `__getattr__` only re-exported
   `ThriftDatabricksClient`. Extend it to also lazily resolve the other
   names `client.py` historically exposed as importable
   (`ThriftResultSet`, `TOpenSessionResp`, `TSparkParameter`,
   `TOperationState`), so `from databricks.sql.client import <name>` keeps
   working without importing the `thrift` package at module load.
   
   Verified: importing `databricks.sql.client` still loads zero thrift
   modules; touching any re-export resolves correctly (and only then pulls
   thrift). Guard test passes with full deps (14/14) and correctly skips
   the kernel modules when pyarrow/kernel are absent.
   
   Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>
   Co-authored-by: Isaac
   Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>

   

   vikrantpuppala committed Aug 13, 2026
   Configuration menu

   • View commit details
   • Copy full SHA for 200201d
   • Browse repository at this point

   Copy the full SHA

   200201d View commit details

   Browse the repository at this point in the history