Skip to content

OnDemandFeatureView: Allow python transformations for offline processing.  #4490

@job-almekinders

Description

@job-almekinders

Is your feature request related to a problem? Please describe.
When running a PythonTransformation for offline feature retrieval, an exception is raised here. I understand that python transformations on very large datasets do not scale well. However, in some cases, a user might still want to apply a python transformation to a smaller dataset in the offline store. Due to this error (and the missing implementation of PythonTransformation().transform_arrow()), this is currently not possible.

Describe the solution you'd like
Instead of raising an exception, we could still run the transformation, and also log a warning indicating that it may not scale well. On top of that we can implement the functionality in transform_arrow to make the transformation happen.

Describe alternatives you've considered

Additional context

Metadata

Metadata

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions