Is your feature request related to a problem? Please describe.
Current Feast implementation for reading/modifying Registry does not allow for concurrent modifications of the Registry by different processes (or server instances), which will be required in production scenarios to independently scale creation and materialization of feature views. Also, there is no mention of this constraint in the documentation. This severely limits scalability.
Describe the solution you'd like
- Ideally, I would suggest that the data currently stored in the registry be stored in a central relational database (or document storage) instead, so that concurrent modification, reading and locking can happen on individual cells, instead of on the entire repository.
- If the above is not feasible, please provide a way to lock the registry while a modification is happening so that the other processes are able to wait for it to complete.
Describe alternatives you've considered
At the suggestion of a Feast Dev, I have considered implementing a locking mechanism, so that different processes can check for an active lock before modifying the registry. But this would put a severe constraint on the scalability, since no parallel actions requiring registry modifications will be allowed.
Additional context
Here is a repo I created to reproduce the concurrency issue when using 2 processes to separate out the materialization from the other create/update requests: https://github.com/DetourToNirvana/mw-feast-samples.
Original Slack post: https://tectonfeast.slack.com/archives/C028V92F1R6/p1640001495050700
Is your feature request related to a problem? Please describe.
Current Feast implementation for reading/modifying Registry does not allow for concurrent modifications of the Registry by different processes (or server instances), which will be required in production scenarios to independently scale creation and materialization of feature views. Also, there is no mention of this constraint in the documentation. This severely limits scalability.
Describe the solution you'd like
Describe alternatives you've considered
At the suggestion of a Feast Dev, I have considered implementing a locking mechanism, so that different processes can check for an active lock before modifying the registry. But this would put a severe constraint on the scalability, since no parallel actions requiring registry modifications will be allowed.
Additional context
Here is a repo I created to reproduce the concurrency issue when using 2 processes to separate out the materialization from the other create/update requests: https://github.com/DetourToNirvana/mw-feast-samples.
Original Slack post: https://tectonfeast.slack.com/archives/C028V92F1R6/p1640001495050700