NDMSPC: N-Dimensional SPaCe¶
A ROOT framework extension for N-dimensional storage, analysis, and visualization of high-dimensional data — designed for HEP, applicable to any domain.
What is NDMSPC?¶
NDMSPC is a ROOT-based framework for storing and analyzing data in N-dimensional sparse spaces. It combines ROOT's THnSparse for efficient sparse bin indexing with TTree for flexible per-bin object storage, enabling multi-dimensional analysis workflows that would otherwise be computationally intractable.
The framework supports multiple simultaneous binning resolutions, elastic storage layouts across filesystem boundaries, and a full visualization platform — from a browser-based inspector UI (ndmspc-ui) to 3D immersive views (ndmvr-core/ndmvr) — all driven by a live ROOT HTTP server.
Getting Started¶
- Install NDMSPC: Follow the installation instructions.
- Read the Quickstart: Learn the basics in the Quickstart Guide.
- Explore Examples: Check out example tutorials.
- Join the Community: Reach out via Support if you have questions.
Concepts & Design¶
| Section | Topic |
|---|---|
| 1 — Data Types & Representations | Unbinned, pseudo-unbinned, and categorical data and how each is stored |
| 2 — Binned Data & Sparse Representations | Histograms, the curse of dimensionality, sparsity, and THnSparse |
| 3 — NDMSPC Architecture | Two-component model: THnSparse for spatial indexing, TTree for payload storage |
| 4 — Practical Limits | Why universal HEP datasets are intractable and how experiments use hierarchical decomposition |
| 5 — NGnTree Multi-Binning | Multiple simultaneous binning resolutions and the [rebin, start, bin_id] system |
| 6 — NGnTree Storage Elasticity | Flexible storage across filesystem and ROOT file boundaries to manage memory |
| 7 — The Process Function | Thread-safe bin iteration, deduplication, and per-bin output storage |
| 8 — Preparation for Visualization | Projection hierarchies, NGnNavigator, and DrawSpectra parameter views |
| 9 — Visualization Platform | NGnT HTTP server, JSROOT/ndmvr rendering, ndmspc-ui orchestration, and user interaction flows |
| 10 — Python Interoperability, ML/MVA, and Scalable Sparse Workflows | PyROOT/notebook interoperability, per-bin ML execution, binning versioning, and THnSparse splitting workflows |
| 11 — Sharing and Collaboration | Portable .ndmspc collaboration bundles, exchange workflows, and reproducible team handoff |