amp: Add Amp Orb project setup - #1497

Fixes found by running the service in an orb:

- Amp checks a service's port as a normal user, who cannot open port 80, and
  Amp's portal proxy in the orb listens on port 443, which Caddy also publishes.
  An override file now makes Caddy publish only the port Amp chooses.
- When the service process dies while the containers keep running, Amp chooses
  another free port and the script moves Caddy to it.
- The script removes its temporary files when it stops.

Adds the optional Jaeger and executor deployments. Each orb makes its own
executor token. On the first start, the script sets it as
executors.accessToken, and sets externalURL to the portal URL, which removes
the site admin warning about externalURL. Restarts do not change the site
configuration.

The Portal tab now shows the admin password in the note of the Sourcegraph
link, and links to the executors page and Jaeger.

Co-authored-by: Amp <amp@ampcode.com>
Amp-Thread-ID: https://ampcode.com/threads/T-01a1ccc1-035e-77c5-8b50-caa8352fd32d

The service wrote the site configuration only when /-/site-init had just
created the admin, so a first start that failed after that step left the
executor without a token for good. The script now reads the site
configuration and adds externalURL and executors.accessToken whenever the
token is missing, keeping the other settings, and leaves the configuration
alone on every other start.

Co-authored-by: Amp <amp@ampcode.com>
Amp-Thread-ID: https://ampcode.com/threads/T-01a1ccc1-035e-77c5-8b50-caa8352fd32d

Sets EXECUTOR_QUEUE_NAMES to batches, batches-hooks, and codeintel instead of
EXECUTOR_QUEUE_NAME=batches. The base executor file leaves EXECUTOR_QUEUE_NAME
empty, which the executor requires when EXECUTOR_QUEUE_NAMES is set.

Co-authored-by: Amp <amp@ampcode.com>
Amp-Thread-ID: https://ampcode.com/threads/T-01a1ccc1-035e-77c5-8b50-caa8352fd32d

…Jaeger

Found by running jobs in an orb. A server-side batch change already ran. An
auto-indexing job did not: Sourcegraph refused to queue it, and once queued,
its upload step got the portal URL, reached it from outside Amp's sign-in, and
received an HTML page. Sourcegraph also sent Jaeger no traces.

- Caddy also publishes port 80, and the executor sets
  EXECUTOR_DOCKER_ADD_HOST_GATEWAY, so the containers the executor starts
  reach Sourcegraph at http://host.docker.internal, which the script sets as
  executors.frontendURL.
- The script sets codeIntelAutoIndexing.enabled, which lets index jobs be
  queued. With the default policies, it indexes nothing on its own.
- The script sets observability.tracing. Sourcegraph exports the requests
  made with ?trace=1.
- externalURL no longer ends in a slash, which put "//" in trace links.

Co-authored-by: Amp <amp@ampcode.com>
Amp-Thread-ID: https://ampcode.com/threads/T-01a1c7a5-6b65-723b-980c-b85e1e9008b5

zoekt-indexserver-0 and zoekt-webserver-0 set cpus: 8, and Docker refuses a
limit above the host's CPU count, so in a 4 vCPU orb no container started.
The override lowers both to 4. With it, the deployment starts in a 4 vCPU,
8 GB orb, and uses 2.8 GB at idle.

Co-authored-by: Amp <amp@ampcode.com>
Amp-Thread-ID: https://ampcode.com/threads/T-01a1c7a5-6b65-723b-980c-b85e1e9008b5

Images and volumes from earlier versions stay on the orb's disk after an
upgrade. Every container of the deployment exists once the frontend is
healthy, so Docker keeps the images and volumes they use.

Amp-Thread-ID: https://ampcode.com/threads/T-01a1d2a1-43a0-7092-998e-43c135c772e8
Co-authored-by: Amp <amp@ampcode.com>

Setup output goes into the project snapshot, and the images are built
artifacts, not dev dependencies. Docker Compose pulls missing images on
the first start instead.

In a 16 vCPU orb, setup drops from 15 s to under 1 s and the first start
grows from about 37 s to about 54 s. Amp waits 60 s for the service; a
slower start makes amp orb services ensure exit 1, but the service keeps
running and the portal appears once Sourcegraph answers.

Amp-Thread-ID: https://ampcode.com/threads/T-01a1d2a1-43a0-7092-998e-43c135c772e8
Co-authored-by: Amp <amp@ampcode.com>

The Portal tab shows the portal in an iframe, and Sourcegraph refuses
that: its sign-in page, admin actions, and the Jaeger proxy reject
requests with Sec-Fetch-Dest: iframe, and every response carries
X-Frame-Options: DENY. The orb's Caddy now removes both, so Sourcegraph,
the Executors page, and Jaeger load in the tab. Customers' Caddy
configuration is unchanged.

Amp-Thread-ID: https://ampcode.com/threads/T-01a1d2a1-43a0-7092-998e-43c135c772e8
Co-authored-by: Amp <amp@ampcode.com>

People open Sourcegraph in many orbs, and a password that differs per orb
has to be looked up each time. The executor token is still made per orb.

Amp-Thread-ID: https://ampcode.com/threads/T-01a1d2a1-43a0-7092-998e-43c135c772e8
Co-authored-by: Amp <amp@ampcode.com>

SOURCEGRAPH_ADMIN_USERNAME, SOURCEGRAPH_ADMIN_EMAIL, and
SOURCEGRAPH_ADMIN_PASSWORD set the first site admin, so every orb in the
project gets the same one and the repository holds no password. The
service stops with a message when one is missing. The sign-in requests
build their JSON with jq, so any character works in the values.

Amp-Thread-ID: https://ampcode.com/threads/T-01a1d2a1-43a0-7092-998e-43c135c772e8
Co-authored-by: Amp <amp@ampcode.com>

The orb is for developing and testing this repository's Docker Compose
deployment, not Sourcegraph itself, which sg in sourcegraph/sourcegraph
is for. amp orb services ensure prints only the Executors and Jaeger
links: the script adds the Sourcegraph link a few seconds later.

Amp-Thread-ID: https://ampcode.com/threads/T-01a1d2a1-43a0-7092-998e-43c135c772e8
Co-authored-by: Amp <amp@ampcode.com>

…on new versions

- Print why Sourcegraph refused the sign-in, the site admin, or a GraphQL request,
  and say when the site configuration does not parse
- Stop the containers when the service fails or stops before it follows the logs
- Prune unused images only when the deployment's images change, so executor job
  images do not pull again after every restart, and stop pruning volumes, which
  hold the data that upgrade and rename tests need
- Rewrite the site configuration when the executor token or the portal URL changed
- Include docker-compose/docker-compose.override.yaml when it exists
- Replace the orb guidance on every setup run, so changes to it reach new snapshots
- Say in AGENTS.md what .amp/ is, and that it follows the base files

Amp-Thread-ID: https://ampcode.com/threads/T-01a1d323-9e56-72e1-ac97-48cbf9823c29
Co-authored-by: Amp <amp@ampcode.com>

- The health check is /-/amp-ready, which Caddy serves only after the script has
  created the site admin and configured the site. /healthz passed about 2 seconds
  before the admin existed, so a sign-in right after ensure got a 401
- When both sign-in and site-init fail, log both reasons: a wrong password on an
  existing site logged only site-init's "Signup failed unexpectedly."

Amp-Thread-ID: https://ampcode.com/threads/T-01a1d323-9e56-72e1-ac97-48cbf9823c29
Co-authored-by: Amp <amp@ampcode.com>