replace rusoto_s3: part 2 (last part)#245
Conversation
jychen7
changed the title
|
|
||
| tokio = { version = "1", features = ["rt-multi-thread"] } | ||
| futures = "0.3" | ||
| # S3 |
There was a problem hiding this comment.
now s3 is refactor, how about I create a release PR for 0.8.2 after this PR is approved and merged.
after we adds GCS and Azure support, we can release 0.9.0? What do you think?
There was a problem hiding this comment.
that sounds good to me, let's do it!
|
One thing I forgot to mention in the previous PR review, we should enable the aws-profile feature in the object store crate because using profile to inject credential is a very common pattern in AWS land. Without this change, it will break the existing users that are using AWS profiles. |
good point, improve here: #247 |
2 participants
Why
step 2 of #240: replace rusoto_s3 (in use_memory_table: true path) with S3 client from object_store (s3://)
rusoto_s3 is used in
columnq::io::s3andcolumnq::io::s3is used in two places:columnq::partitions_from_table_sourcecolumnq::table::deltawe would like to remove rusoto_s3 and switch to object_store, so can unify S3/GCS/Azure in future
How
following part 1 (#244), this PR (part 2) will focus on migrating
columnq::table::deltaand deletecolumnq::io::s3this PR also remove crate
rusoto_xxxcompletely and close #243Test
Rely on existing unit tests in
roapi/columnq/src/table/delta.rs
Lines 131 to 132 in 1e3644e
roapi/columnq/src/table/delta.rs
Lines 151 to 152 in 1e3644e