chore(docker): refactor docker-compose topologies with Hubble - #3149

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 37.76%. Comparing base (8e167c7) to head (9cf8de7).
⚠️ Report is 1 commits behind head on master.

@@             Coverage Diff              @@
##             master    #3149      +/-   ##
============================================
+ Coverage     32.69%   37.76%   +5.07%     
- Complexity     5517     6551    +1034     
============================================
  Files           789      800      +11     
  Lines         67750    68929    +1179     
  Branches       8955     9157     +202     
============================================
+ Hits          22151    26034    +3883     
+ Misses        42987    39838    -3149     
- Partials       2612     3057     +445     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

Pushed 6b6c3a3c. All four blocking findings from the last review are fixed in this PR rather than deferred, and the PR description is updated to match.

• Auth false-green: images pull by default and Server readiness requires 401 unauthenticated plus 200 authenticated, so an image that ignores PASSWORD never becomes healthy.
• Control plane: PD, Store, and Server ports bind to 127.0.0.1, widened only through HUGEGRAPH_CONTROL_PLANE_HOST / HUGEGRAPH_SERVER_PUBLISH_HOST.
• Advertised addresses: each replica's PD-registered URL is configurable, defaults unchanged.
• Render-only CI: a live smoke job starts the cluster, attaches Hubble, and asserts the above.

The cluster also goes back to being authenticated by default, per your point that this is the file people copy. Running without auth is now an explicit opt-in, docker-compose-3x3.non-auth.yml plus a matching Hubble properties file, documented beside the default flow with a copy-paste prompt for people who hand this to an assistant.

Tested on a clean Docker host, from zero images: 9/9 healthy, 401/200 on all three replicas, ports on loopback, three PD peers and three Stores Up, Hubble attached without recreating a single cluster container, H2 state surviving recreation across both flows, and the non-auth path returning 200 with no credentials anywhere in the container environment.

Testing caught three bugs that rendering never would have, one of them a half-authenticated cluster where only server0 lost its credentials because !reset does not survive a YAML anchor on every Compose version. That one is the clearest argument for the smoke job, so I would rather keep it than trim it.

Fixed in 9616037, verified against a real cluster.

The advertised-address point is the important one and you were right: HG_SERVER_REST_URL becomes restserver.url, which is the bind address. Running the cluster with the documented override crash-looped that replica on java.net.BindException: Cannot assign requested address while the two defaults stayed healthy. The three variables are gone and the README documents the limitation instead. My earlier reply on that thread was wrong.

Also fixed: healthcheck no longer pins to the seeded password (401-only, so password rotation cannot brick readiness), Hubble volumes are external so down -v from either flow cannot destroy shared state, pull policy is overridable for offline use, the non-auth flow documents network/volumes/teardown, the dotenv reader rejects unquoted values and runs in a subshell, and CI unsets the variables it asserts defaults for, covers the non-auth properties, and gates the smoke test to contract changes.

Pushed d2a12bc1. Commit-by-commit, since this branch has grown a lot since the last review.

0d1e67ba: back to authenticated by default. The two commits before it had made the 3-node cluster anonymous. Reverted on top rather than rewritten, so every thread here keeps its anchor. Anonymous is still supported, as an explicit opt-in rather than the default.

c3115579: the four blocking findings. Images pull by default and readiness proves the running image enforces auth, so a cached older image cannot look healthy while serving the graph APIs open. PD, Store and Server ports bind to loopback, widened only through HUGEGRAPH_CONTROL_PLANE_HOST / HUGEGRAPH_SERVER_PUBLISH_HOST. And a live smoke job replaces the render-only checks.

10c6c6b5: corrections the smoke job found in itself. Running it against a real cluster showed two of its own assertions were wrong: /v1/cluster returns PD peers, not graph servers, so the registration check could never pass; and the persistence check passed even when the two flows used different volumes. Now it asserts three PD peers and three Stores Up, and proves shared state by writing a marker through one flow and reading it back through the other.

7ea0efe2: the port reference contradicted the code. It still said cluster ports bind every interface after they had moved to loopback. Also, the documented no-auth placeholder token was long enough to pass validation, so copying those two lines into docker/.env would produce a cluster that looked authenticated while signing tokens with a key printed in this repo. The placeholder is now deliberately too short for the Server to accept.

6b6c3a3c: the no-auth override only disabled auth on one Server. The three services shared a YAML anchor, and !reset does not survive an alias on every Compose version: on 5.1.2 only server0 lost its credentials, so the cluster came up half authenticated, while 5.1.4 reset all three. The healthcheck override propagated either way, which is what hid it. Spelled out per service.

96160375: the seams around the contract. Chiefly: HG_SERVER_REST_URL becomes restserver.url, which is the bind address, so the advertised-address option crash-looped a replica on java.net.BindException; those variables are gone and the limitation is documented. Readiness no longer pins to the seeded admin password, which would have bricked every replica after a password rotation. Hubble's volumes became external, because Compose deletes a fixed-name volume on down -v from any project declaring the name, so leaving one flow destroyed the other's database. Plus the offline pull escape hatch, the no-auth flow's missing network and teardown, a dotenv reader that returned an empty password for malformed values, and a verification block whose exit 1 closed the operator's shell.

05715714: a claim that covered files it did not. The version-pinning paragraph promised an image ignoring PASSWORD never reports healthy across three Compose files. Only the cluster has that check; the single-node files probe /versions, which stays open either way.

d2a12bc1: your latest finding. Making the volumes external fixed data loss but broke the flow that needs them most: external volume "hugegraph-hubble-upload-files" not found. The attach section now creates both first, honouring HUBBLE_DB_VOLUME and HUBBLE_UPLOAD_VOLUME, inspect-then-create so it is safe to repeat. Reproduced the failure and confirmed the fix against a running cluster.

Everything above was verified by running it, not by reading. On the current head: ten containers healthy, 401 unauthenticated and 200 authenticated on all three replicas, wrong password 401, three PD peers and three Stores Up, the H2 database inside its mount, Hubble attaching without recreating a cluster container, and shared state surviving the teardown that used to destroy it.

One note on CI: docker-build (Dockerfile-hstore) is red, and it is a Maven build inside that Dockerfile. This branch changes no Java, no pom and no Dockerfile, and the same job passes on current master, so I believe it is unrelated to this diff. This push re-runs it.

@imbajin one open question before this is mergeable, on the PD-registered Server addresses. The inline thread is marked outdated (the lines it pointed at are gone), so raising it here where it is visible: #3149 (comment)

Your ask was a configurable advertised address, or an external client path that resolves. I could not do either from Compose: HG_SERVER_REST_URL becomes restserver.url, which is also the address the REST server binds, so an externally reachable value crash-loops the replica on java.net.BindException: Cannot assign requested address. I confirmed that by running it. Splitting bind from advertise needs a Server-side setting.

So, which way do you want it closed?

A. Accept it as a documented limitation. External clients use the published Server ports; PD discovery stays correct for anything on hugegraph-net, which is the Hubble case this add-on exists for.

B. I open a Server issue for a real advertised-address option and link it here.

I lean A for this PR, since B is a Server change that would hold up the add-on. Happy to file B regardless if you want it tracked.

The latest HStore failure is independent of this Compose PR: the exact base/master SHA and multiple other master SHAs fail the same VertexCoreTest.testQueryByJointIndexesWithSearchAndTwoRangeIndexesAndWithin assertion (expected 3, actual 1), while other heads on this branch pass. The recurring HStore issue, test history, and investigation boundary are tracked in #3180. I am rerunning the failed HStore job; no PR code change is warranted for this failure.