Skip to content

Conversation

@chimeyrock999
Copy link
Contributor

  1. Implemented a new feature registry that reads and writes feature data directly on HDFS using pyarrow.fs.HadoopFileSystem.
  2. Added integration tests using two containers (1 Namenode and 1 Datanode) via Testcontainers and bde2020’s Hadoop images.
  3. To use pyarrow.fs.HadoopFileSystem in client or CI environments, you must properly configure the environment by following the PyArrow HDFS documentation.
    .

@chimeyrock999 chimeyrock999 requested a review from a team as a code owner October 10, 2025 23:29
Signed-off-by: Chimey Rock <trinhvanthoai99@gmail.com>
Signed-off-by: Chimey Rock <trinhvanthoai99@gmail.com>
Signed-off-by: Chimey Rock <trinhvanthoai99@gmail.com>
Signed-off-by: Chimey Rock <trinhvanthoai99@gmail.com>
Signed-off-by: Chimey Rock <trinhvanthoai99@gmail.com>
Signed-off-by: Chimey Rock <trinhvanthoai99@gmail.com>
Signed-off-by: Chimey Rock <trinhvanthoai99@gmail.com>
Copy link
Member

@franciscojavierarceo franciscojavierarceo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for this contribution!!

Can we update the registry documentation to include this?

Signed-off-by: Chimey Rock <trinhvanthoai99@gmail.com>
Signed-off-by: Chimey Rock <trinhvanthoai99@gmail.com>
…running

Signed-off-by: Chimey Rock <trinhvanthoai99@gmail.com>
@chimeyrock999
Copy link
Contributor Author

chimeyrock999 commented Oct 11, 2025

Thank you for this contribution!!

Can we update the registry documentation to include this?

Thank @franciscojavierarceo. I’ve fixed the integration test and updated the docs for the HDFS registry.

Signed-off-by: Chimey Rock <trinhvanthoai99@gmail.com>
Signed-off-by: Chimey Rock <trinhvanthoai99@gmail.com>
Signed-off-by: Chimey Rock <trinhvanthoai99@gmail.com>
Signed-off-by: Chimey Rock <trinhvanthoai99@gmail.com>
Signed-off-by: Chimey Rock <trinhvanthoai99@gmail.com>
Copy link
Member

@ntkathole ntkathole left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good

@ntkathole ntkathole enabled auto-merge (squash) October 14, 2025 13:43
@ntkathole ntkathole merged commit 4c65872 into feast-dev:master Oct 15, 2025
1 check passed
franciscojavierarceo pushed a commit that referenced this pull request Oct 27, 2025
# [0.56.0](v0.55.0...v0.56.0) (2025-10-27)

### Bug Fixes

* Add mode field to Transformation proto for proper serialization ([2390d2e](2390d2e))
* Date wise remote offline store historical data retrieval  ([#5686](#5686)) ([949ba3d](949ba3d))
* Fix STRING type handling in on-demand feature views ([#5669](#5669)) ([dfbb743](dfbb743))
* Fixed torch install issue in CI ([366e5a8](366e5a8))
* ODFV not getting counted in resource count ([1d640b6](1d640b6))
* Skip tag updates if user do not have permissions ([#5673](#5673)) ([0a951ce](0a951ce))

### Features

* Add document of Go feature server. ([#5697](#5697)) ([cbd1dde](cbd1dde))
* Add flexible commandArgs support for complete Feast CLI control ([#5678](#5678)) ([6414924](6414924))
* Add HDFS as a feature registry ([#5655](#5655)) ([4c65872](4c65872))
* Add nodeSelector to service config ([#5675](#5675)) ([9728cde](9728cde))
* Add OTEL based observability to the Go Feature Server ([#5685](#5685)) ([f4afdad](f4afdad))
* Added health endpoint for the UI ([#5665](#5665)) ([3aec5d5](3aec5d5))
* Added kuberay support ([e0b698d](e0b698d))
* Added support for filtering multi-projects ([#5688](#5688)) ([eb0a86e](eb0a86e))
* Batch Embedding at scale for RAG with Ray ([cc2a46d](cc2a46d))
* Optimize SQL entity handling without creating temporary tables ([#5695](#5695)) ([aa2c838](aa2c838))
* Support aggregation in odfv ([#5666](#5666)) ([564e965](564e965))
* Support cache_mode for registries ([021e9ea](021e9ea))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants