feat: Add ReactFlow visualization for Feast registry metadata #5297

franciscojavierarceo · 2025-04-27T01:13:03Z

Add ReactFlow Visualization for Feast Registry Metadata

This PR adds a new visualization feature to the Feast UI that displays registry metadata as an interactive graph using ReactFlow.

Screenshots

Registry Visualization with DAG and Animated Data Flow

Features

Interactive graph visualization of Feast registry metadata
Shows relationships between different Feast objects (Feature Views, Feature Services, Entities, Data Sources)
Directed graph with arrows showing relationship flow direction
Animated edges with green overlay to show data flow
Left-to-right layout for improved readability
Increased node dimensions to accommodate longer text
Renamed visualization to "Lineage" for better clarity
Vertical division node design with:
- Icons on the left side with colored backgrounds matching their type
- Text on the right side
- Light gray outline around the entire box
Box color now matches the icon color for better visual consistency
Increased DAG spacing for better visual clarity
Clean text boxes with icons that have colored backgrounds for better visibility
Supports both vertical and horizontal layout options
Handles both populated and empty registry configurations

Implementation Details

Added ReactFlow dependency for graph visualization
Created new RegistryVisualization component for rendering the graph
Added RegistryVisualizationTab component for integration with the UI
Updated ProjectOverviewPage to include a tab-based navigation
Added Jest configuration for handling ReactFlow CSS imports
Implemented directed graph with arrows to show data flow direction
Added animated edges with green overlay for better visual feedback
Configured left-to-right layout for improved readability
Increased node dimensions (250x60) to better accommodate longer text
Renamed visualization from "Registry Visualization" to "Lineage"
Replaced colored backgrounds with color-coded icons for better visual clarity
Added matching background colors to icons for better visibility
Updated node design with vertical division and light gray outline
Updated box color to match icon color for better visual consistency
Increased DAG spacing with larger nodesep and ranksep values
Added animated green overlay to edges to show data flow

CI Fixes

Fixed syntax error in yarn.lock file at line 3914 by regenerating the file
The yarn.lock file had a corrupted entry that was causing SyntaxError: Invalid value type 3914:0 during yarn install
Fixed protobuf import resolution by adding --path ../protos parameter to pbjs command
The protobuf compiler was looking for imports at incorrect paths, causing ENOENT: no such file or directory, open '../protos/feast/storage/feast/types/Value.proto' error

Link to Devin run

https://app.devin.ai/sessions/6ec7c851670a4ac6bbfe1aecfd5dbfba

Co-Authored-By: Francisco Javier Arceo <arceofrancisco@gmail.com>

- Added MarkerType import from ReactFlow - Configured edges with arrow markers to show relationship direction - Added source and target handles to CustomNode component - Updated edge style to make arrows more prominent - Fixed ESLint warning about missing dependencies in useEffect Co-Authored-By: Francisco Javier Arceo <arceofrancisco@gmail.com>

Co-Authored-By: Francisco Javier Arceo <arceofrancisco@gmail.com>

…mand Co-Authored-By: Francisco Javier Arceo <arceofrancisco@gmail.com>

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>

ui/.npmrc.local

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>

# [0.49.0](v0.48.0...v0.49.0) (2025-04-29) ### Bug Fixes * Adding brackets to unit tests ([c46fea3](c46fea3)) * Adding logic back for a step ([2bb240b](2bb240b)) * Adjustment for unit test action ([a6f78ae](a6f78ae)) * Allow get_historical_features with only On Demand Feature View ([#5256](#5256)) ([0752795](0752795)) * CI adjustment ([3850643](3850643)) * Embed Query configuration breaks when switching between DataFrame and SQL ([#5257](#5257)) ([32375a5](32375a5)) * Fix for proto issue in utils ([1b291b2](1b291b2)) * Fix milvus online_read ([#5233](#5233)) ([4b91f26](4b91f26)) * Fix tests ([431d9b8](431d9b8)) * Fixed Permissions object parameter in example ([#5259](#5259)) ([045c100](045c100)) * Java CI [#12](#12) ([d7e44ac](d7e44ac)) * Java PR [#15](#15) ([a5da3bb](a5da3bb)) * Java PR [#16](#16) ([e0320fe](e0320fe)) * Java PR [#17](#17) ([49da810](49da810)) * Materialization logs ([#5243](#5243)) ([4aa2f49](4aa2f49)) * Moving to custom github action for checking skip tests ([caf312e](caf312e)) * Operator - remove default replicas setting from Feast Deployment ([#5294](#5294)) ([e416d01](e416d01)) * Patch java pr [#14](#14) ([592526c](592526c)) * Patch update for test ([a3e8967](a3e8967)) * Remove conditional from steps ([995307f](995307f)) * Remove misleading HTTP prefix from gRPC endpoints in logs and doc ([#5280](#5280)) ([0ee3a1e](0ee3a1e)) * removing id ([268ade2](268ade2)) * Renaming workflow file ([5f46279](5f46279)) * Resolve `no pq wrapper` import issue ([#5240](#5240)) ([d5906f1](d5906f1)) * Update actions to remove check skip tests ([#5275](#5275)) ([b976f27](b976f27)) * Update docling demo ([446efea](446efea)) * Update java pr [#13](#13) ([fda7db7](fda7db7)) * Update java_pr ([fa138f4](fa138f4)) * Update repo_config.py ([6a59815](6a59815)) * Update unit tests workflow ([06486a0](06486a0)) * Updated docs for docling demo ([768e6cc](768e6cc)) * Updating action for unit tests ([0996c28](0996c28)) * Updating github actions to filter at job level ([0a09622](0a09622)) * Updating Java CI ([c7c3a3c](c7c3a3c)) * Updating java pr to skip tests ([e997dd9](e997dd9)) * Updating workflows ([c66bcd2](c66bcd2)) ### Features * Add date_partition_column_format for spark source ([#5273](#5273)) ([7a61d6f](7a61d6f)) * Add Milvus tutorial with Feast integration ([#5292](#5292)) ([a1388a5](a1388a5)) * Add pgvector tutorial with PostgreSQL integration ([#5290](#5290)) ([bb1cbea](bb1cbea)) * Add ReactFlow visualization for Feast registry metadata ([#5297](#5297)) ([9768970](9768970)) * Add retrieve online documents v2 method into pgvector ([#5253](#5253)) ([6770ee6](6770ee6)) * Compute Engine Initial Implementation ([#5223](#5223)) ([64bdafd](64bdafd)) * Enable write node for compute engine ([#5287](#5287)) ([f9baf97](f9baf97)) * Local compute engine ([#5278](#5278)) ([8e06dfe](8e06dfe)) * Make transform on writes configurable for ingestion ([#5283](#5283)) ([ecad170](ecad170)) * Offline store update pull_all_from_table_or_query to make timestampfield optional ([#5281](#5281)) ([4b94608](4b94608)) * Serialization version 2 deprecation notice ([#5248](#5248)) ([327d99d](327d99d)) * Vector length definition moved to Feature View from Config ([#5289](#5289)) ([d8f1c97](d8f1c97))

…dev#5297) Signed-off-by: Jacob Weinhold <29459386+j-wine@users.noreply.github.com>

# [0.49.0](feast-dev/feast@v0.48.0...v0.49.0) (2025-04-29) ### Bug Fixes * Adding brackets to unit tests ([c46fea3](feast-dev@c46fea3)) * Adding logic back for a step ([2bb240b](feast-dev@2bb240b)) * Adjustment for unit test action ([a6f78ae](feast-dev@a6f78ae)) * Allow get_historical_features with only On Demand Feature View ([feast-dev#5256](feast-dev#5256)) ([0752795](feast-dev@0752795)) * CI adjustment ([3850643](feast-dev@3850643)) * Embed Query configuration breaks when switching between DataFrame and SQL ([feast-dev#5257](feast-dev#5257)) ([32375a5](feast-dev@32375a5)) * Fix for proto issue in utils ([1b291b2](feast-dev@1b291b2)) * Fix milvus online_read ([feast-dev#5233](feast-dev#5233)) ([4b91f26](feast-dev@4b91f26)) * Fix tests ([431d9b8](feast-dev@431d9b8)) * Fixed Permissions object parameter in example ([feast-dev#5259](feast-dev#5259)) ([045c100](feast-dev@045c100)) * Java CI [feast-dev#12](feast-dev#12) ([d7e44ac](feast-dev@d7e44ac)) * Java PR [feast-dev#15](feast-dev#15) ([a5da3bb](feast-dev@a5da3bb)) * Java PR [feast-dev#16](feast-dev#16) ([e0320fe](feast-dev@e0320fe)) * Java PR [feast-dev#17](feast-dev#17) ([49da810](feast-dev@49da810)) * Materialization logs ([feast-dev#5243](feast-dev#5243)) ([4aa2f49](feast-dev@4aa2f49)) * Moving to custom github action for checking skip tests ([caf312e](feast-dev@caf312e)) * Operator - remove default replicas setting from Feast Deployment ([feast-dev#5294](feast-dev#5294)) ([e416d01](feast-dev@e416d01)) * Patch java pr [feast-dev#14](feast-dev#14) ([592526c](feast-dev@592526c)) * Patch update for test ([a3e8967](feast-dev@a3e8967)) * Remove conditional from steps ([995307f](feast-dev@995307f)) * Remove misleading HTTP prefix from gRPC endpoints in logs and doc ([feast-dev#5280](feast-dev#5280)) ([0ee3a1e](feast-dev@0ee3a1e)) * removing id ([268ade2](feast-dev@268ade2)) * Renaming workflow file ([5f46279](feast-dev@5f46279)) * Resolve `no pq wrapper` import issue ([feast-dev#5240](feast-dev#5240)) ([d5906f1](feast-dev@d5906f1)) * Update actions to remove check skip tests ([feast-dev#5275](feast-dev#5275)) ([b976f27](feast-dev@b976f27)) * Update docling demo ([446efea](feast-dev@446efea)) * Update java pr [feast-dev#13](feast-dev#13) ([fda7db7](feast-dev@fda7db7)) * Update java_pr ([fa138f4](feast-dev@fa138f4)) * Update repo_config.py ([6a59815](feast-dev@6a59815)) * Update unit tests workflow ([06486a0](feast-dev@06486a0)) * Updated docs for docling demo ([768e6cc](feast-dev@768e6cc)) * Updating action for unit tests ([0996c28](feast-dev@0996c28)) * Updating github actions to filter at job level ([0a09622](feast-dev@0a09622)) * Updating Java CI ([c7c3a3c](feast-dev@c7c3a3c)) * Updating java pr to skip tests ([e997dd9](feast-dev@e997dd9)) * Updating workflows ([c66bcd2](feast-dev@c66bcd2)) ### Features * Add date_partition_column_format for spark source ([feast-dev#5273](feast-dev#5273)) ([7a61d6f](feast-dev@7a61d6f)) * Add Milvus tutorial with Feast integration ([feast-dev#5292](feast-dev#5292)) ([a1388a5](feast-dev@a1388a5)) * Add pgvector tutorial with PostgreSQL integration ([feast-dev#5290](feast-dev#5290)) ([bb1cbea](feast-dev@bb1cbea)) * Add ReactFlow visualization for Feast registry metadata ([feast-dev#5297](feast-dev#5297)) ([9768970](feast-dev@9768970)) * Add retrieve online documents v2 method into pgvector ([feast-dev#5253](feast-dev#5253)) ([6770ee6](feast-dev@6770ee6)) * Compute Engine Initial Implementation ([feast-dev#5223](feast-dev#5223)) ([64bdafd](feast-dev@64bdafd)) * Enable write node for compute engine ([feast-dev#5287](feast-dev#5287)) ([f9baf97](feast-dev@f9baf97)) * Local compute engine ([feast-dev#5278](feast-dev#5278)) ([8e06dfe](feast-dev@8e06dfe)) * Make transform on writes configurable for ingestion ([feast-dev#5283](feast-dev#5283)) ([ecad170](feast-dev@ecad170)) * Offline store update pull_all_from_table_or_query to make timestampfield optional ([feast-dev#5281](feast-dev#5281)) ([4b94608](feast-dev@4b94608)) * Serialization version 2 deprecation notice ([feast-dev#5248](feast-dev#5248)) ([327d99d](feast-dev@327d99d)) * Vector length definition moved to Feature View from Config ([feast-dev#5289](feast-dev#5289)) ([d8f1c97](feast-dev@d8f1c97)) Signed-off-by: Jacob Weinhold <29459386+j-wine@users.noreply.github.com>

Add ReactFlow visualization for Feast registry metadata

325aaba

Co-Authored-By: Francisco Javier Arceo <arceofrancisco@gmail.com>

franciscojavierarceo requested a review from a team as a code owner April 27, 2025 01:13

devin-ai-integration bot and others added 22 commits April 27, 2025 01:43

Merge master: Integrate ReactFlow visualization with RegistrySearch

cc765e0

Co-Authored-By: Francisco Javier Arceo <arceofrancisco@gmail.com>

Fix code formatting

16ba51a

Co-Authored-By: Francisco Javier Arceo <arceofrancisco@gmail.com>

Fix ReactFlow imports and UI styling

0c129f7

Co-Authored-By: Francisco Javier Arceo <arceofrancisco@gmail.com>

Fix formatting issues in RegistryVisualization component

2220af0

Co-Authored-By: Francisco Javier Arceo <arceofrancisco@gmail.com>

Update dependencies for ReactFlow visualization

1e5bf79

Co-Authored-By: Francisco Javier Arceo <arceofrancisco@gmail.com>

Refactor visualization to use left-to-right layout with dashed lines

0dfc260

Co-Authored-By: Francisco Javier Arceo <arceofrancisco@gmail.com>

Increase node dimensions and rename visualization to Lineage

2907624

Co-Authored-By: Francisco Javier Arceo <arceofrancisco@gmail.com>

Add color coding for different object types

2e3bff9

Co-Authored-By: Francisco Javier Arceo <arceofrancisco@gmail.com>

Replace colored backgrounds with color-coded icons

ce93241

Co-Authored-By: Francisco Javier Arceo <arceofrancisco@gmail.com>

Add background colors to icons for better visibility

d09b909

Co-Authored-By: Francisco Javier Arceo <arceofrancisco@gmail.com>

Update node design with vertical division and light gray outline

3ab382f

Co-Authored-By: Francisco Javier Arceo <arceofrancisco@gmail.com>

Update box color to match icon color

0ffb2cd

Co-Authored-By: Francisco Javier Arceo <arceofrancisco@gmail.com>

Space out DAG and add animated green overlay to edges

730b62d

Co-Authored-By: Francisco Javier Arceo <arceofrancisco@gmail.com>

Add dummy NPM token for local development to fix CI yarn install issue

e3bc3f5

Co-Authored-By: Francisco Javier Arceo <arceofrancisco@gmail.com>

Fix syntax error in yarn.lock file by regenerating it

a01cbfc

Co-Authored-By: Francisco Javier Arceo <arceofrancisco@gmail.com>

Fix formatting issues in RegistryVisualization.tsx

cc6787d

Co-Authored-By: Francisco Javier Arceo <arceofrancisco@gmail.com>

Fix protobuf import resolution by adding --path parameter to pbjs com…

68ad2e8

…mand Co-Authored-By: Francisco Javier Arceo <arceofrancisco@gmail.com>

chore: Refactor transform on write (feast-dev#5300)

f4ee41f

removing ui/.npmrc.ci

9005a3b

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>

updated

1f1a5d0

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>

linter

7c2fa07

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>

HaoXuAI approved these changes Apr 29, 2025

View reviewed changes

ui/.npmrc.local Outdated Show resolved Hide resolved

franciscojavierarceo and others added 2 commits April 28, 2025 22:56

making items more clickable

2d6ce2e

Signed-off-by: Francisco Javier Arceo <farceo@redhat.com>

Delete ui/.npmrc.local

1719fb6

franciscojavierarceo merged commit 9768970 into feast-dev:master Apr 29, 2025
23 checks passed

franciscojavierarceo mentioned this pull request May 1, 2025

Create dependency graph using reactflow #4058

Closed

jfw-ppi pushed a commit to jfw-ppi/feast that referenced this pull request Jun 7, 2025

feat: Add ReactFlow visualization for Feast registry metadata (feast-…

eea26d2

…dev#5297) Signed-off-by: Jacob Weinhold <29459386+j-wine@users.noreply.github.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: Add ReactFlow visualization for Feast registry metadata #5297

feat: Add ReactFlow visualization for Feast registry metadata #5297

Uh oh!

franciscojavierarceo commented Apr 27, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

feat: Add ReactFlow visualization for Feast registry metadata #5297

feat: Add ReactFlow visualization for Feast registry metadata #5297

Uh oh!

Conversation

franciscojavierarceo commented Apr 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Add ReactFlow Visualization for Feast Registry Metadata

Screenshots

Registry Visualization with DAG and Animated Data Flow

Features

Implementation Details

CI Fixes

Link to Devin run

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

franciscojavierarceo commented Apr 27, 2025 •

edited

Loading