Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 37 additions & 10 deletions .github/workflows/docker-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,14 +24,35 @@ on:

# nodejs/node's daily workflow dispatches a `node-daily` event with the commit it builds successfully.
#
# Each platform builds in its own job on a native runner and pushes its own nightly-linux-<arch> tag.
# Then, the merge workflow combines the platform tags into `nightly`.
# The resolve job picks the commit once, so every platform builds the same one.
# Each platform builds in its own job on a native runner and pushes its own nightly-linux-<arch> and nightly-<sha>-linux-<arch> tags.
# Then, the merge workflow combines the platform tags into `nightly` and the immutable `nightly-<sha>`.
#
# If any platform fails, the others still push their own nightly-linux-<arch> tags, but `nightly` is not updated.
# If any platform fails, the others still push their own platform tags, but `nightly` is not updated.
# The failed run flags the broken platform in an issue.
jobs:
docker:
resolve:
if: ${{ github.ref == format('refs/heads/{0}', vars.NIGHTLY_BRANCH) }}
runs-on: ubuntu-slim
outputs:
sha: ${{ steps.resolve.outputs.sha }}
steps:
- name: Resolve the nodejs/node commit
id: resolve
env:
NODE_REF: ${{ github.event.client_payload.sha || inputs.node_ref }}
run: |
if [ -z "$NODE_REF" ]; then
NODE_REF=$(git ls-remote https://github.com/nodejs/node.git refs/heads/main | cut -f1)
fi
if [[ ! "$NODE_REF" =~ ^[0-9a-f]{40}$ ]]; then
echo "::error::NODE_REF must be a full commit SHA, got: $NODE_REF"
exit 1
fi
echo "sha=$NODE_REF" >> "$GITHUB_OUTPUT"

docker:
needs: [resolve]
strategy:
fail-fast: false
matrix:
Expand All @@ -52,21 +73,27 @@ jobs:
with:
platforms: linux/${{ matrix.arch }}
push: true
build-args: NODE_REF=${{ github.event.client_payload.sha || inputs.node_ref }}
tags: ${{ vars.DOCKERHUB_IMAGE }}:nightly-linux-${{ matrix.arch }}
build-args: NODE_REF=${{ needs.resolve.outputs.sha }}
labels: org.nodejs.node.commit=${{ needs.resolve.outputs.sha }}
tags: |
${{ vars.DOCKERHUB_IMAGE }}:nightly-linux-${{ matrix.arch }}
${{ vars.DOCKERHUB_IMAGE }}:nightly-${{ needs.resolve.outputs.sha }}-linux-${{ matrix.arch }}

# Only runs when every platform in this run built successfully (the default for `needs`).
merge:
needs: [docker]
needs: [resolve, docker]
uses: $/.github/workflows/docker-merge.yml
secrets: inherit
with:
# A single-platform run has no nightly-<sha> image for the other platform, so it only updates `nightly`.
sha: ${{ (inputs.platform || 'all') == 'all' && needs.resolve.outputs.sha || '' }}

# GitHub only notifies whoever triggered a run (for repository_dispatches it is the bot account)
# So when an automatic run fails, open a tracking issue (or comment on the open one), and close it once a run succeeds.
# Manual runs are skipped: GitHub already notifies whoever started them.
report:
needs: [docker, merge]
if: ${{ !cancelled() && github.event_name != 'workflow_dispatch' && needs.docker.result != 'skipped' }}
needs: [resolve, docker, merge]
if: ${{ !cancelled() && github.event_name != 'workflow_dispatch' && needs.resolve.result != 'skipped' }}
runs-on: ubuntu-slim
permissions:
actions: read # to list the failed jobs
Expand All @@ -75,7 +102,7 @@ jobs:
GH_TOKEN: ${{ github.token }}
GH_REPO: ${{ github.repository }}
FAILED: ${{ contains(needs.*.result, 'failure') }}
NODE_REF: ${{ github.event.client_payload.sha || 'head of main' }}
NODE_REF: ${{ needs.resolve.outputs.sha || github.event.client_payload.sha || 'tip of the default branch' }}
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
steps:
- name: Open, update, or close the tracking issue
Expand Down
32 changes: 28 additions & 4 deletions .github/workflows/docker-merge.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,18 @@ name: Merge images

on:
workflow_dispatch: # allow re-merging the platform tags by hand
inputs:
sha:
description: 'nodejs/node commit SHA whose platform images to merge (default: the current nightly-linux-<arch> tags)'
type: string
required: false
default: ''
workflow_call: # allow this workflow to be called by another workflow
inputs:
sha:
type: string
required: false
default: ''

env:
IMAGE: ${{ vars.DOCKERHUB_IMAGE }}
Expand Down Expand Up @@ -30,12 +41,25 @@ jobs:
# It will only succeed when at least one image for each architecture has been built and pushed.
# When a manual run rebuilt only one platform, the other platform's existing image is
# merged in unchanged.
#
# With a commit SHA, it merges that commit's platform images into both `nightly-<sha>` and `nightly`.
# Merging by SHA keeps `nightly` from mixing platforms built from different commits when runs overlap.
- name: Create multi-arch manifest
env:
SHA: ${{ inputs.sha }}
run: |
docker buildx imagetools create \
-t ${REGISTRY}/${IMAGE}:nightly \
${REGISTRY}/${IMAGE}:nightly-linux-amd64 \
${REGISTRY}/${IMAGE}:nightly-linux-arm64
if [ -n "$SHA" ]; then
docker buildx imagetools create \
-t ${REGISTRY}/${IMAGE}:nightly-${SHA} \
-t ${REGISTRY}/${IMAGE}:nightly \
${REGISTRY}/${IMAGE}:nightly-${SHA}-linux-amd64 \
${REGISTRY}/${IMAGE}:nightly-${SHA}-linux-arm64
else
docker buildx imagetools create \
-t ${REGISTRY}/${IMAGE}:nightly \
${REGISTRY}/${IMAGE}:nightly-linux-amd64 \
${REGISTRY}/${IMAGE}:nightly-linux-arm64
fi

- name: Verify multi-arch manifest
run: |
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ Some useful commands:

nodejs/node's [daily workflow](https://github.com/nodejs/node/blob/main/.github/workflows/daily.yml) builds `main` and, when that succeeds, sends a `node-daily` `repository_dispatch` event to this repository with the commit SHA it built successfully.

The [build workflow](./.github/workflows/docker-build.yml) then builds and publishes the images from that commit across both platforms (`arm64` and `amd64`) and then tags them both as `nightly`. If either platform fails, the other is still published as `nightly-linux-<arch>`, but `nightly` stays on the previous build until both succeed. A failed automatic run opens an issue labeled `nightly-build-failure` (or comments on the open one), and the next successful run closes it.
The [build workflow](./.github/workflows/docker-build.yml) then builds and publishes the images from that commit across both platforms (`arm64` and `amd64`) and then tags them both as `nightly` and `nightly-<sha>`, where `<sha>` is the full nodejs/node commit SHA. The `nightly-<sha>` tags are never moved, so you can pin an image to the commit it was built from. Inside an image, `/home/developer/nodejs/node-commit` holds the same SHA, and `docker inspect` shows it as the `org.nodejs.node.commit` label. If either platform fails, the other is still published as `nightly-linux-<arch>`, but `nightly` stays on the previous build until both succeed. A failed automatic run opens an issue labeled `nightly-build-failure` (or comments on the open one), and the next successful run closes it.

In case the `repository_dispatch` events stop arriving, a weekly scheduled run builds the tip of the default branch as a fallback. Builds can also be manually triggered using `workflow_dispatch` from the Actions tab. You can specify a commit SHA (defaults to head of main) and which platform to build (defaults to both of them).

Expand Down
2 changes: 2 additions & 0 deletions scripts/clone.sh
Original file line number Diff line number Diff line change
Expand Up @@ -22,5 +22,7 @@ mkdir -p /home/developer/nodejs/node
git reset --hard FETCH_HEAD
git update-ref refs/remotes/upstream/main FETCH_HEAD
git branch --set-upstream-to=upstream/main main
# Contributors mount their own checkout over this one, so record the commit outside it.
git rev-parse HEAD > /home/developer/nodejs/node-commit
git restore-mtime # Restore file modification times to commit times for build cache to match.
)