Cassandra as sink option for ingestion job#51
Conversation
khorshuheng
force-pushed
the
cassandra-avro
branch
from
4eda548 to
550c703
Compare
Signed-off-by: Khor Shu Heng <khor.heng@gojek.com>
Signed-off-by: Khor Shu Heng <khor.heng@gojek.com>
Signed-off-by: Khor Shu Heng <khor.heng@gojek.com>
khorshuheng
force-pushed
the
cassandra-avro
branch
from
fe8decd to
c0e2c9d
Compare
Signed-off-by: Khor Shu Heng <khor.heng@gojek.com>
khorshuheng
force-pushed
the
cassandra-avro
branch
from
1e16c39 to
e5a2e47
Compare
| .option("max_age", config.featureTable.maxAge.getOrElse(0L)) | ||
| .save() | ||
|
|
||
| val writer = config.store match { |
There was a problem hiding this comment.
Can this be passed through spark config (in BasePipeline)?
Signed-off-by: Khor Shu Heng <khor.heng@gojek.com>
khorshuheng
force-pushed
the
cassandra-avro
branch
from
e0e3aef to
759814c
Compare
Signed-off-by: Khor Shu Heng <khor.heng@gojek.com>
|
@khorshuheng: The following test failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here. |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: khorshuheng, pyalex The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
3 participants
Signed-off-by: Khor Shu Heng khor.heng@gojek.com
tWhat this PR does / why we need it:
Adding Cassandra as sink option for ingestion job.
Which issue(s) this PR fixes:
Fixes #
Does this PR introduce a user-facing change?: