Skip to content

README.md

Latest commit

 

History

History
97 lines (78 loc) · 4.52 KB

README.md

File metadata and controls

97 lines (78 loc) · 4.52 KB

native/kernel/ — consumer-side directory for the Rust napi binding

The Rust binding source lives in the kernel repo at databricks-sql-kernel/napi/. Building it requires a local checkout of that repo — see "Build for local dev" below. The published npm package is @databricks/databricks-sql-kernel-<triple>.

Workspace topology

The napi crate is a standalone Cargo workspace ([workspace] members = ["."] in napi/Cargo.toml), not a sibling of pyo3/ in the kernel root workspace.

The reason is Cargo feature unification. pyo3 builds the kernel with the default tls-native feature (system OpenSSL via native-tls). The napi crate has to opt INTO tls-rustls instead: napi modules are loaded into Node.js processes that statically link OpenSSL 3.x, and dynamically linking the system's OpenSSL 1.1 (which native-tls pulls in on Linux) collides with Node's symbols at module-load time and segfaults the process before any Rust code runs. rustls is pure Rust + ring and avoids the conflict entirely.

If napi lived in the same workspace as pyo3, cargo build --workspace would unify the kernel's feature set to tls-native ∪ tls-rustls, link both TLS stacks into the resulting napi cdylib, and reintroduce the same clash. Standalone-workspace is the fix.

What lives in this directory

  • index.d.ts — TypeScript declarations consumed by lib/kernel/. Generated by napi-rs from the Rust source; checked in as the consumer-facing type contract.
  • index.js — napi-rs's per-platform router shim. Gitignored; populated by npm run build:native for local dev. In published tarballs it ships alongside the .d.ts and require()s the right @databricks/databricks-sql-kernel-<triple> optional dependency.
  • index.*.node — the actual native binary, one per platform. Gitignored. In production these live in the per-triple optional dependencies (@databricks/databricks-sql-kernel-linux-x64-gnu, etc.); for local dev npm run build:native copies one into this directory.

Build for local dev

# From the nodejs repo root:
export DATABRICKS_SQL_KERNEL_REPO=/path/to/your/databricks-sql-kernel
npm run build:native           # release build (default)
BUILD_PROFILE=  npm run build:native  # debug build (empty BUILD_PROFILE drops --release)

DATABRICKS_SQL_KERNEL_REPO points at the kernel repo root (the directory containing napi/) and is required when your kernel checkout isn't at ../../databricks-sql-kernel relative to the nodejs repo.

Production load path

At release time the kernel's CI publishes @databricks/databricks-sql-kernel-<triple> npm packages — one per supported platform — each containing a single .node binary. native/kernel/index.js (the napi-rs router) require()s the package matching the consumer's process.platform / process.arch at load time.

Status: the per-platform packages are published on npm (kernel 0.2.0) and declared in the driver's optionalDependencies, pinned to that exact version. npm install resolves only the package matching the consumer's process.platform / process.arch; the others are skipped (that is what optionalDependencies tolerates), so installing on an unsupported platform does not fail. Local development can still override the installed binary with npm run build:native, which copies a freshly built index.<triple>.node into this directory — the router prefers that local file over the installed package.

Supported platforms

The driver declares all eight published triples in optionalDependencies:

  • linux-x64-gnu, linux-arm64-gnu
  • linux-x64-musl, linux-arm64-musl
  • darwin-x64, darwin-arm64
  • win32-x64-msvc, win32-arm64-msvc

On any other platform (e.g. linux-arm gnueabihf, riscv64, s390x, FreeBSD) the kernel binding is simply absent: KernelNativeLoader returns undefined from tryGet() / throws a structured MODULE_NOT_FOUND hint from get(), and the driver continues to use the Thrift backend exclusively. This is expected, not a regression — additional triples are added to optionalDependencies as the kernel CI starts publishing them.

Supply-chain note

The triple names not yet built/published (…-linux-arm-gnueabihf, …-riscv64-gnu, etc.) referenced by the router boilerplate are not squat-able: @databricks is a Databricks-owned npm scope, and npm only allows org members to publish under a scope it owns. A third party therefore cannot register @databricks/databricks-sql-kernel-* and have the router autoload it. No placeholder packages are required.