Skip to content

Interactive module graph viewers - #82375

Open
crisptrutski wants to merge 3 commits into
masterfrom
modules-tree-explorer
Open

Interactive module graph viewers#82375
crisptrutski wants to merge 3 commits into
masterfrom
modules-tree-explorer

Conversation

@crisptrutski

@crisptrutski crisptrutski commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Part of BEGUILD-18: Module system improvements.

Why

The backend module configuration now contains 209 modules, including 81 whose namespace prefix does not match the module name. The existing inspection tools each cover only part of that system:

  • ./bin/mage modules-tree shows the correct hierarchy, but not module boundaries or dependencies.
  • dev.module-viz shows dependencies, but derives source paths, line counts, and team ownership from module names. That gives incorrect results for custom namespace prefixes and nested modules. It can also count a child module's files as part of its parent.

This PR adds one explorer that combines the hierarchy and dependency data, backed by the shared module resolver. The existing text tree remains available for quick terminal use.

What changed

The explorer has three views:

  • Tree shows the module hierarchy. Selecting a module shows its team, API, dependencies, dependents, exports, friends, and source statistics.
  • Hotspots ranks modules by known dependents, dependencies, or their combined total. With a dev REPL, these counts come from observed namespace edges. The standalone page uses declared module edges and leaves :uses :any unresolved.
  • Graph shows the one, two, or three hop neighborhood around one or more focused modules. Modules can be filtered by team, hidden individually, or excluded with a regular expression.

There are two ways to build it:

Entry point Data included Use case
./bin/mage modules-tree --html --output modules.html Module configuration and source statistics Generate a standalone snapshot without a dev REPL
(dev.module-explorer/open!) The same data, plus namespace-level edges from dev.deps-graph See observed module coupling and which API namespaces each consumer uses

Other changes:

  • Source paths, statistics, and inherited team ownership now use the shared module resolver. Nested and renamed modules are assigned their own files instead of relying on directory names.
  • The explorer and module linter share the same effective API calculation, including the default .api, .core, and .init namespaces when :api is omitted.
  • Unrestricted :uses :any modules remain explicit instead of being expanded into a dependency on every module. The REPL entry point replaces that unknown set with dependencies observed in the source scan.
  • The text tree and HTML explorer use the same display path logic, including enterprise modules nested under their OSS counterparts.
  • The old dev.module-viz HTML, Clojure namespace, and Jetty server have been removed. The explorer embeds its data in one HTML file and runs entirely in the browser.
  • Selection, team filters, and graph state are stored in the URL fragment, so reloads and shared links restore the same view.

The Tree and Hotspots views work offline. The Graph view loads Cytoscape and its layout libraries from unpkg.com when opened.

How to verify

  1. Generate and open the explorer:

    ./bin/mage modules-tree --html --output modules.html
    open modules.html
  2. In Tree, select a nested module with a custom prefix, such as query-processor.cache-backend. Confirm that its source link and statistics belong to that module rather than query-processor.

  3. Switch to Hotspots and rank by dependents, dependencies, and total coupling.

  4. Switch to Graph, focus a module, change the hop count, and filter or hide neighboring modules.

  5. From a dev REPL, run (dev.module-explorer/open!). Expand a module in Graph and confirm that incoming edges identify the API namespaces used by each consumer.

Use --no-stats with the Mage command when a faster snapshot is more useful than source metrics.

Test coverage

The updated tests cover module tree paths, enterprise nesting, custom prefix resolution, effective API defaults, inherited team ownership, unrestricted and observed dependencies, URL-state round trips, Git failures, per-file module assignment, and safe embedding of module data in the generated page.

`./bin/mage modules-tree --html` writes a single page with three views
of the module config: the tree, a coupling ranking, and a dependency
graph around focused modules. `dev.module-explorer/open!` builds the
same page in a dev REPL with namespace-level edges, so the graph can
show which API namespaces each consumer uses.

The graph replaces dev.module-viz and its Jetty server. Source paths,
stats and teams go through the shared resolver, so nested and renamed
modules report correctly. module-team moves into hooks.common.modules.
@ci-conductor

ci-conductor Bot commented Sep 11, 2026

Copy link
Copy Markdown

View on CI Conductor

@linear

linear Bot commented Sep 11, 2026

Copy link
Copy Markdown

BEGUILD-18

@metabase-bot metabase-bot Bot added the .Team/Metabot Metabot team label Sep 11, 2026
@crisptrutski crisptrutski changed the title Consolidate the module viewers into one explorer Add an interactive explorer for backend module boundaries Sep 11, 2026
@crisptrutski crisptrutski changed the title Add an interactive explorer for backend module boundaries Interactive module graph viewers Sep 11, 2026
@crisptrutski
crisptrutski marked this pull request as ready for review September 11, 2026 20:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

.Team/Metabot Metabot team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant