Skip to content

ci: migrate Docker builds to Blacksmith using existing registries - #9715

Draft
mattinannt wants to merge 11 commits into
mainfrom
codex/blacksmith-docker-trial
Draft

mattinannt wants to merge 11 commits into
mainfrom
codex/blacksmith-docker-trial

Conversation

@mattinannt

@mattinannt mattinannt commented Oct 20, 2026 •

Copy link
Copy Markdown
Member

No ticket: infrastructure evaluation.

What & why

Was: PR and release Docker builds used Depot builders and its private intermediate registry.

Now: Blacksmith builds directly into the existing ECR/GHCR repositories; native checks gate release tags on the tested digest.

Draft pending evaluation of release performance/cost; complete publishing trials passed.

Where to look

Breaking changes

  • This PR contains breaking changes

None. Internal CI only.

Migrations & env

  • Existing ECR/GHCR repositories and credentials; no new infrastructure, IAM grants, lifecycle policies, or variables.
  • Community jobs use package permissions; cloud jobs retain AWS OIDC access.
  • PR builds have a separate cache; forks use isolated GitHub runners.

How this was tested

219 Docker tests and workflow lint passed. Rerun: pnpm --dir docker test; publishing commands are in the runbook.

Coverage

Behaviour How Outcome
Repository selection and digest binding unit (guard) image-promotion.test.ts: rejects changed digests and cross-repository promotion before writing tags.
Registry permissions and native validation gates unit (guard) image-promotion.test.ts: correct credentials; both native checks gate release tags.
Native Blacksmith release images manual AMD64 4m38–4m51s; ARM64 9m25–9m57s including upload; Node cached; smoke passed.
PR Docker builds and fork routing manual + unit (guard) Warm builds roughly three minutes; image-promotion.test.ts verifies fork isolation.
Complete ECR publishing manual Passed: both architectures, native smoke, digest-preserving promotion.
Complete experimental GHCR publishing/signing manual Passed: native smoke, promotion, independently verified signature.

Open gaps

Release-speed improvements and monthly savings remain unproven; no production cutover has occurred.

Risks

  • GHCR candidate tags are visible before validation; release and deployment tags remain gated.
  • Run-specific tags follow existing retention; partial publication requires forward recovery using the tested digest.
Complete publishing trials — October 21, 2026

Both workflows passed on 53619d171e5991ae77f28628ec0d419ee696dd18:
ECR and
experimental GHCR.
They built the actual cloud/community profiles, uploaded candidates to the existing repositories,
assembled multi-platform indexes, passed runtime and browser checks on native AMD64 and ARM64,
and promoted the tested digests without rebuilding. Sentry source-map uploads succeeded.

Profile Architecture Build and upload Complete build job
Community AMD64 4m38s 5m02s
Community ARM64 9m57s 10m29s
Cloud AMD64 4m51s 5m21s
Cloud ARM64 9m25s 9m59s

All four builds reused the compiled Node layer and rebuilt the application.
The workflows ran concurrently for functional validation, so these are observations rather than a controlled performance comparison.

The ECR trial published only 0.0.0-blacksmith.20261021131225; production/staging tags were disabled.
The GHCR trial used the existing experimental package and its trial-branch alias.
Independent registry reads confirmed that the candidate, version, and branch tags all point to
sha256:82b0db27a97fee7571a56122ad3c720de53c6d612ece18b45d84b5f4df7cbba9, containing both architectures.
Independent Cosign verification passed with the expected workflow identity and GitHub OIDC issuer.
No new registry repository, IAM grants, lifecycle policy, or repository variable was used.

Local validation: all 219 Docker tests, actionlint on changed workflows, full repository formatting, and git diff --check passed.

PR Docker validation, full unit tests, E2E tests, type checks, and linting passed on this revision.

Native build and PR trial evidence

AMD64 cold: 42m49s build, 44m01s job, native runtime/browser smoke passed.

AMD64 warm: 3m build, 4m09s job. Another warm rebuild: 3m37s build. Both smoke checks passed.

ARM64 32-vCPU cold: 39m28s build, 41m52s job. Warm rebuild: 18m51s build, 22m27s job. Native smoke passed both times; Node compilation was cached in the warm run, while the application rebuild took 969.9s.

An earlier ARM64 follow-up restored an old cache snapshot. A later inspection, about eight minutes after the successful cold build, restored matching database hashes and the compiled Node layer. The initial miss remains unexplained; a commit request alone does not establish immediate cache availability.

PR cold build: 20m20s build, 22m23s job. Warm PR build: 2m49s build, 4m48s job. The preceding Depot PR run took 7m34s and 10m06s respectively. This compares CI-only revisions, not controlled release workloads.

Later PR validation: 3m16s build, 5m15s job; cache cleanup reclaimed 1.483 GB and reported 31.97 GB remaining. The 30gb pruning setting is not a guaranteed billing cap.

The native benchmarks omitted telemetry credentials. The October 21 full workflows above also exercised Sentry uploads and the actual cloud/community profiles.


Note

AI model used — unknown, reasoning effort unknown.

@coderabbitai

coderabbitai Bot commented Oct 20, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@mattinannt mattinannt changed the title ci: trial native Blacksmith Docker release builds ci: trial Blacksmith builds and private image staging Oct 20, 2026
@mattinannt mattinannt changed the title ci: trial Blacksmith builds and private image staging ci: migrate Docker builds to Blacksmith with private ECR staging Oct 20, 2026
@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

🚨 PR Size Warning

This PR has approximately 1051 lines of changes (624 additions, 427 deletions across 19 files).

Large PRs (>800 lines) are significantly harder to review and increase the chance of merge conflicts. Consider splitting this into smaller, self-contained PRs.

💡 Suggestions:

  • Split by feature or module - Break down into logical, independent pieces
  • Create a sequence of PRs - Each building on the previous one
  • Branch off PR branches - Don't wait for reviews to continue dependent work

📊 What was counted:

  • ✅ Source files, stylesheets, configuration files
  • ❌ Excluded 2 files (tests, locales, locks, generated files)

📚 Guidelines:

  • Ideal: 300-500 lines per PR
  • Warning: 500-800 lines
  • Critical: 800+ lines ⚠️

If this large PR is unavoidable (e.g., migration, dependency update, major refactor), please explain in the PR description why it couldn't be split.

@blacksmith-sh

This comment has been minimized.

@mattinannt mattinannt changed the title ci: migrate Docker builds to Blacksmith with private ECR staging ci: migrate Docker builds to Blacksmith using existing registries Oct 21, 2026
@sonarqubecloud

Copy link
Copy Markdown

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant