ci: Deploy Jaeger and the executor in the smoke test - #1475

Stacked on #1472. The smoke test deployed docker-compose.yaml only. Jaeger never ran, and the executor was a second container started later with docker compose run, so the container count, health check wait, and restart check never covered either one.

smoke-test.sh now passes all three Compose files to every docker-compose command (pull, up, config --services), so the same checks cover all 27 long-running services.

This PR's build fails until #1465 merges, and it should. The first run found that the Jaeger overlay on main crash-loops:

application version: git-commit=0524ae6d, git-version=v1.60.0, build-date=2024-08-06
Error: unknown flag: --config

#1369 changed the command to Jaeger v2's --config in March, but the image stayed at jaeger-all-in-one:6.1.2889, which is Jaeger v1.60.0. The release PR #1465 moves it to 8.1.4690. With #1465's images Jaeger starts, and the test passes.

Changes

• test/smoke-test.sh: one compose_files list, used by every docker-compose command. New check that Jaeger answers on localhost:16686/-/debug/jaeger/.
• test/docker-compose.override.yaml (new): sets EXECUTOR_FRONTEND_PASSWORD and EXECUTOR_QUEUE_NAME. The executor exits without them, and executor.docker-compose.yaml ships them empty.
• test/executor-smoke-test.sh: no longer starts its own executor. It reads the password from the running executor container, sets it as executors.accessToken, and waits for that container to register.
• .buildkite/validate-docker-compose.sh: also validates the combination the smoke test deploys.

Things to know

• The executor starts before executors.accessToken is set, so its first requests are rejected. It keeps running and registers once the test sets the token. "Running with no restarts" therefore does not prove the executor works; the registration check does.
• The executor container is privileged and mounts the Docker socket, as the Compose file specifies. It creates /scratch on the test VM.
• executor.docker-compose.yaml pins executor:6.0.0. Update main: build v8.1.4690 #1465 moves it to 8.1.4690 too.
• Not checked: that traces reach Jaeger, or that the executor can run a job.

Checklist

• All images have a valid tag and SHA256 sum (no image changes)

Test plan

• Build 16840, this branch: failed as described above. expected 27 running containers, found 26, jaeger: Restarting (1), and jaeger.log was uploaded as an artifact.
• Build 16842, this branch plus the image changes from Update main: build v8.1.4690 #1465 on a throwaway branch (since deleted): passed. 27 running containers of 27 expected, no restarts, Jaeger answered, and the executor from the same up registered after 10s. The smoke test job's log spans 324s; ci: Close the gaps in what CI checks #1472's took about 290s. One run each.
• The :docker: step passed in both builds, including the new four-file docker compose config.
• shellcheck is clean on every script.