Skip to content

Conversation

@ntkathole
Copy link
Member

@ntkathole ntkathole commented Feb 18, 2025

What this PR does / why we need it:

This is the fix to prevent deadlocks on registry cache refresh.

  • Used self._refresh_lock.acquire(blocking=False) to avoid infinite waiting and avoids releasing an unlocked lock.
  • Fixed recursive calls by checking self._refresh_lock.locked() flag to prevent self.refresh() from calling itself indirectly.
  • Handled exceptions by wrapping logic inside try to catch errors and always release locks.

Which issue(s) this PR fixes:

#4898

Tests:

# pytest sdk/python/tests/unit/infra/registry/test_registry.py -svv
sdk/python/tests/unit/infra/registry/test_registry.py::test_cache_expiry_triggers_refresh 02/20/2025 03:36:21 PM feast.infra.registry.caching_registry INFO: Registry cache expired, so refreshing
02/20/2025 03:36:21 PM feast.infra.registry.caching_registry INFO: Skipping refresh if already in progress
02/20/2025 03:36:21 PM feast.infra.registry.caching_registry INFO: Registry cache expired, so refreshing
02/20/2025 03:36:21 PM feast.infra.registry.caching_registry INFO: Skipping refresh if already in progress
PASSED
sdk/python/tests/unit/infra/registry/test_registry.py::test_skip_refresh_if_lock_held 02/20/2025 03:36:21 PM feast.infra.registry.caching_registry INFO: Skipping refresh if lock is already held by another thread
PASSED
==================================================== 2 passed in 0.01s ====================================================

Copy link
Member

@franciscojavierarceo franciscojavierarceo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it looks like CachingRegistry doesn't have tests anywhere at all. It'd be good to add some and start to catch some of this stuff.

Could you add a test under sdk/python/tests/units/infra/registry/test_registry.py?

@lokeshrangineni
Copy link
Contributor

integration test failure seems to be legit. may be better to investigate.

Signed-off-by: ntkathole <nikhilkathole2683@gmail.com>
@ntkathole
Copy link
Member Author

it looks like CachingRegistry doesn't have tests anywhere at all. It'd be good to add some and start to catch some of this stuff.

Could you add a test under sdk/python/tests/units/infra/registry/test_registry.py?

Tests added.

Copy link
Member

@franciscojavierarceo franciscojavierarceo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice thanks for this!

@franciscojavierarceo franciscojavierarceo merged commit f3a24de into feast-dev:master Feb 20, 2025
23 checks passed
franciscojavierarceo pushed a commit that referenced this pull request Mar 10, 2025
# [0.47.0](v0.46.0...v0.47.0) (2025-03-10)

* feat!: Include PUBLIC_URL in defaultProjectListPromise URL in /ui ([2f0f7b3](2f0f7b3))

### Bug Fixes

* Add transformation_service_endpoit to support Go feature server. ([#5071](#5071)) ([5627d7c](5627d7c))
* Adding extra space on the VM to kind cluster to see if this solves the issue with memory not available with operator e2e tests. ([#5102](#5102)) ([e6e928c](e6e928c))
* Allow unencrypted Snowflake key ([#5097](#5097)) ([87a7c23](87a7c23))
* Cant add different type of list types ([#5118](#5118)) ([bebd7be](bebd7be))
* Fixing transformations on writes ([#5127](#5127)) ([95ac34a](95ac34a))
* Identify s3/remote uri path correctly ([#5076](#5076)) ([93becff](93becff))
* Increase available action VM storage and reduce dev feature-server image size ([#5112](#5112)) ([75f5a90](75f5a90))
* Move Feast to pyproject.toml instead of setup.py ([#5067](#5067)) ([4231274](4231274))
* Skip refresh if already in progress or if lock is already held ([#5068](#5068)) ([f3a24de](f3a24de))

### Features

* Add an OOTB Chat uI to the Feature Server to support RAG demo ([#5106](#5106)) ([40ea7a9](40ea7a9))
* Add Couchbase Columnar as an Offline Store ([#5025](#5025)) ([4373cbf](4373cbf))
* Add Feast Operator RBAC example with Kubernetes Authentication … ([#5077](#5077)) ([2179fbe](2179fbe))
* Added docling and pytorch as add on ([#5089](#5089)) ([135342b](135342b))
* Feast Operator example with Postgres in TLS mode. ([#5028](#5028)) ([2c46f6a](2c46f6a))
* Operator - Add feastProjectDir section to CR with git & init options ([#5079](#5079)) ([d64f01e](d64f01e))
* Override the udf name when provided as input to an on demand transformation ([#5094](#5094)) ([8a714bb](8a714bb))
* Set value_type of entity directly in from_proto ([#5092](#5092)) ([90e7498](90e7498))
* Updating retrieve online documents v2 to work for other fields for sq… ([#5082](#5082)) ([fc121c3](fc121c3))

### BREAKING CHANGES

* The PUBLIC_URL environment variable is now taken into account by default
when fetching the projects list. This is a breaking change only if all
these points apply:

1. You're using Feast UI as a module

2. You're serving the UI files from a non-root path via the PUBLIC_URL
   environment variable

3. You're serving the project list from the root path

4. You're not passing the `feastUIConfigs.projectListPromise` prop to
   the FeastUI component

In this case, you need to explicitly fetch the project list from the
root path via the `feastUIConfigs.projectListPromise` prop:

```diff
 const root = createRoot(document.getElementById("root")!);
 root.render(
   <React.StrictMode>
-    <FeastUI />
+    <FeastUI
+      feastUIConfigs={{
+        projectListPromise: fetch("/projects-list.json", {
+            headers: {
+              "Content-Type": "application/json",
+            },
+          }).then((res) => res.json())
+      }}
+    />
   </React.StrictMode>
 );
```

Signed-off-by: Harri Lehtola <peruukki@hotmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants