Skip to content

docs: note S3-compatible endpoints for s3_endpoint config - #1235

Draft
goanpeca wants to merge 1 commit into
feathr-ai:mainfrom
goanpeca:docs/s3-compatible-endpoint
Draft

docs: note S3-compatible endpoints for s3_endpoint config#1235
goanpeca wants to merge 1 commit into
feathr-ai:mainfrom
goanpeca:docs/s3-compatible-endpoint

Conversation

@goanpeca

Copy link
Copy Markdown

Description

offline_store.s3.s3_endpoint is passed straight through to the Hadoop fs.s3a.endpoint property, so it already works with any S3-compatible endpoint and not only AWS. That is not obvious from the current documentation, which shows s3.amazonaws.com as the only example and describes the setting as "Required if using AWS S3 Storage as an offline store."

This adds one clarifying line in each of the two places the option is documented: the feathr_config.yaml workspace template and the environment variable table in docs/how-to-guides/feathr-configuration-and-env.md. Comments and documentation only. No behavior change, no new config keys, no new dependencies.

Per CONTRIBUTING.md, documentation-only changes do not need an accompanying GitHub issue.

How was this PR tested?

There is no new code path to exercise, so testing was limited to confirming the edited files are still well formed and that the statement being added is accurate.

I confirmed feathr_config.yaml still parses with yaml.safe_load and that offline_store.s3.s3_endpoint still resolves to s3.amazonaws.com, unchanged. I also confirmed the edited Markdown table row still has the same three columns as its neighbors. No links were added, so the broken-link checker is unaffected.

The chain the new note describes: client.py reads offline_store__s3__s3_endpoint, _get_s3_config_str emits it as S3_ENDPOINT in the HOCON job config, and S3ResourceInfoSetter.setupHadoopConfig sets that value as fs.s3a.endpoint on the Spark Hadoop configuration.

Does this PR introduce any user-facing changes?

  • No. You can skip the rest of this section.
  • Yes. Make sure to clarify your proposed changes.

Documentation wording only. The s3_endpoint default and all runtime behavior are unchanged. Readers of either the config template or the environment variable reference now see that the endpoint can point at any S3-compatible service.

Signed-off-by: Gonzalo Peña-Castellanos <goanpeca@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant