Is your feature request related to a problem? Please describe.
Currently Feast cannot stage data anywhere except GCP storage.
Describe the solution you'd like
Implement a handler for s3 staging URLs, so that data can be staged in AWS
Describe alternatives you've considered
we could use kafka and beam to provide an alternative to file based staging. considering that it is used to ingest the same features, there shouldn't be any reason it couldn't be used for the entity rows or the retrieved features. It would be nice to allow for multiple transfer methods encapsulated behind an pluggable API.
Part of #367
Is your feature request related to a problem? Please describe.
Currently Feast cannot stage data anywhere except GCP storage.
Describe the solution you'd like
Implement a handler for s3 staging URLs, so that data can be staged in AWS
Describe alternatives you've considered
we could use kafka and beam to provide an alternative to file based staging. considering that it is used to ingest the same features, there shouldn't be any reason it couldn't be used for the entity rows or the retrieved features. It would be nice to allow for multiple transfer methods encapsulated behind an pluggable API.
Part of #367