Skip to content

Pull the local S3 service from Chainguard's MinIO image - #5631

Merged
obenland merged 1 commit into
mainfrom
s3-service-image
Oct 13, 2026
Merged

obenland merged 1 commit into
mainfrom
s3-service-image

Conversation

@obenland

Copy link
Copy Markdown
Member

Description

MinIO removed its community images from Docker Hub and quay.io in September 2026 after archiving the project. docker pull minio/minio and docker pull minio/mc now fail with pull access denied, so the catalog's s3 and load_to_s3 services cannot start. Everything that depends on them fails with it: ov just up locally, and in CI the "Run tests for the API" job dies in "Start API, ingest and index test data" with service "scheduler" is not running because scheduler depends on s3. The last green run of that job on main was 2026-09-10; no PR has exercised it since, so #5627 was the first to surface the breakage.

Chainguard rebuilds MinIO from source and publishes it free for linux/amd64 and linux/arm64. Its server image ships bash, mkdir, sleep, and mc at /usr/bin/mc, so docker/minio/s3_entrypoint.sh and docker/minio/load_to_s3_entrypoint.sh run unchanged. The same image therefore backs both services. Chainguard's separate minio-client image is distroless and cannot run the loader script. The free tier only publishes a floating latest tag, so the image is pinned by digest, in the spirit of #5602.

The image runs as uid 65532 rather than root. Its /data directory is world-writable, so a fresh minio volume works without changes. A volume created by the old root-running image may carry root-owned metadata; if s3 fails to start on an existing checkout, docker volume rm openverse_minio and bring the stack back up. I could not reproduce that case locally because this machine never held a working MinIO volume.

Verified locally with docker compose up -d s3 load_to_s3: s3 reaches healthy, the three configured buckets are created on startup, and load_to_s3 loads every bucket under catalog/tests/s3-data and exits 0.

Testing Instructions

  1. ov just catalog/up and confirm s3 becomes healthy and load_to_s3 exits 0.
  2. ov just api/init and confirm the sample data step completes.
  3. CI on this PR runs both catalog and API test jobs.

Checklist

  • My pull request has a descriptive title (not a vague title likeUpdate index.md).
  • My pull request targets the default branch of the repository (main) or a parent feature branch.
  • My commit messages follow best practices.
  • My code follows the established code style of the repository.
  • I added or updated tests for the changes I made (if applicable).
  • I added or updated documentation (if applicable).
  • I tried running the project locally and verified that there are no visible errors.
  • I ran the DAG documentation generator (ov just catalog/generate-docs for catalog
    PRs) or the media properties generator (ov just catalog/generate-docs media-props
    for the catalog or ov just api/generate-docs for the API) where applicable.

Developer Certificate of Origin

Developer Certificate of Origin
Developer Certificate of Origin
Version 1.1

Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129

Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.


Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
    have the right to submit it under the open source license
    indicated in the file; or

(b) The contribution is based upon previous work that, to the best
    of my knowledge, is covered under an appropriate open source
    license and I have the right under that license to submit that
    work with modifications, whether created in whole or in part
    by me, under the same open source license (unless I am
    permitted to submit under a different license), as indicated
    in the file; or

(c) The contribution was provided directly to me by some other
    person who certified (a), (b) or (c) and I have not modified
    it.

(d) I understand and agree that this project and the contribution
    are public and that a record of the contribution (including all
    personal information I submit with it, including my sign-off) is
    maintained indefinitely and may be redistributed consistent with
    this project or the open source license(s) involved.

🤖 Generated with Claude Code

MinIO removed its community images from Docker Hub and quay.io, so the
catalog's s3 and load_to_s3 services can no longer start. The API test
job in CI fails in the sample data step as a result, and `ov just up`
fails locally for the same reason.

Chainguard rebuilds MinIO from source and publishes it for amd64 and
arm64. Its server image includes bash and mc, so the existing entrypoint
scripts run unchanged and the same image now backs both services. The
client image is distroless and cannot run the loader script. The free
tier only publishes a floating `latest` tag, so the image is pinned by
digest.

Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings October 13, 2026 13:59
@obenland
obenland requested a review from a team as a code owner October 13, 2026 13:59
@obenland obenland added 🟧 priority: high Stalls work on the project or its dependents 🤖 aspect: dx Concerns developers' experience with the codebase 🧰 goal: internal improvement Improvement that benefits maintainers, not users 🐳 tech: docker Involves Docker 🧱 stack: catalog Related to the catalog and Airflow DAGs labels Oct 13, 2026
@obenland
obenland requested review from krysal and obulat and removed request for a team October 13, 2026 13:59

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@obenland
obenland merged commit 7931ee6 into main Oct 13, 2026
74 checks passed
@obenland
obenland deleted the s3-service-image branch October 13, 2026 14:14
@github-project-automation github-project-automation Bot moved this from 👀 Needs Review to 🤝 Merged in Openverse PRs Oct 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

🤖 aspect: dx Concerns developers' experience with the codebase 🧰 goal: internal improvement Improvement that benefits maintainers, not users 🟧 priority: high Stalls work on the project or its dependents 🧱 stack: catalog Related to the catalog and Airflow DAGs 🐳 tech: docker Involves Docker

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

3 participants