Skip to content

Add standalone pgsql-exporter for external databases - #969

Draft
marcleblanc2 wants to merge 2 commits into
mainfrom
marc/external-db-exporters
Draft

marcleblanc2 wants to merge 2 commits into
mainfrom
marc/external-db-exporters

Conversation

@marcleblanc2

@marcleblanc2 marcleblanc2 commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Open issues

Found in review on 2026-11-13 (review thread). This PR is a draft until the first three are settled.

  1. Merged with main, the feature fails to render. Add opt-in selector labels to adopt existing workloads #1007 changed sourcegraph.selectorLabels to take (list . .Values.<component>); this branch calls it with $top. The only textual conflict is CHANGELOG.md, so the templates merge cleanly and then any standalone: true fails helm template, and postgresExporterStandalone_test.yaml fails. Default values still render. Fix: rebase, call it with (list $top $database), and move the changelog entry under main's ## Unreleased
  2. Renaming a database breaks its dashboards. The exporter Service takes its app label and name from <database>.name; the dashboards match the literals codeinsights-db and codeintel-db. With codeInsightsDB.name: insights-ext the Service renders as insights-ext-exporter with app: insights-ext. The in-cluster database Services hardcode app. Fix: take app and the Service name from a fixed map
  3. The Deployment can name a ServiceAccount the chart does not create. With pgsql.enabled: false, standalone: true, and pgsql.serviceAccount.create: true, the Deployment renders serviceAccountName: pgsql, and no ServiceAccount, because that template is gated on pgsql.enabled
  4. sslmode values the application accepts can break the exporter. The exporter's driver accepts only disable, require, verify-ca, and verify-full; with prefer or allow, pg_up stays 0. verify-* needs a CA bundle, and the new pod has no extraVolumes. Document the accepted values at least. Read from the driver source, not run
  5. Tests do not assert the DATA_SOURCE_SSLMODE secret key, the Service selector against the pod labels, the disableKubernetesSecrets case, or the queries file
  6. Small ones:
    • values.yaml still lists the existing secret's keys without sslmode
    • <database>.postgresExporter: null with an external database now fails the render
    • the exporter inherits the retired database's nodeSelector, tolerations, and priorityClassName, with no value of its own

Not tested on a cluster: merged with main the feature does not render, so there is nothing to deploy yet. The first three were reproduced with helm template.

Closes #388

Checklist

What

Adds <database>.postgresExporter.standalone (default false) for pgsql, codeIntelDB and codeInsightsDB. When a database is external (<database>.enabled: false) and the toggle is set, the chart deploys pgsql-exporter as its own Deployment + Service, so the Postgres panels in the Grafana dashboards work against managed databases such as AWS RDS.

pgsql:
  enabled: false
  auth:
    host: pgsql.abc.us-east-1.rds.amazonaws.com
    sslmode: require
  postgresExporter:
    standalone: true
  • sourcegraph.dataSource now builds DATA_SOURCE_URI from the auth secret's host and sslmode keys when the database is external. The sidecar on internal databases still renders localhost:...?sslmode=disable, unchanged.
  • The Service carries app: <database name>, which the Prometheus relabel config maps to the job label the dashboards group by (job="codeinsights-db" is matched literally in the dashboard queries).
  • The Deployment reuses the database's podSecurityContext, nodeSelector, affinity, tolerations, priorityClassName, serviceAccount, podLabels/podAnnotations and postgresExporter.env, plus the global postgresExporter.image / resources.
  • When sourcegraph.disableKubernetesSecrets is true, no DATA_SOURCE_* env is rendered; supply the DSN via postgresExporter.env.
  • The external-databases example README listed the secret key as pgsslmode; the chart reads sslmode (already required by the application containers), so it's corrected.

Test plan

  • helm lint charts/sourcegraph/: clean
  • helm unittest charts/sourcegraph: 202 passed, including new tests/postgresExporterStandalone_test.yaml (no render by default, no render for internal DB with toggle on, no render for external DB with toggle off, one Deployment+Service per enabled DB, env wiring with existingSecret, Service labels/annotations, sidecar DSN unchanged)
  • helm template with default values diffed against origin/main: identical
  • helm template with all three databases external and standalone: true: three Deployments and Services render as expected (see charts/sourcegraph/examples/external-databases/override.yaml)
  • ./scripts/helm-docs.sh: README regenerated, tree clean

Adds <database>.postgresExporter.standalone (default false) for pgsql,
codeIntelDB and codeInsightsDB. When the database is external
(<database>.enabled: false) and the toggle is set, the chart deploys
pgsql-exporter as its own Deployment and Service, so the Postgres
panels in the Grafana dashboards work against e.g. AWS RDS.

The exporter connects with the database's auth credentials; the
sourcegraph.dataSource helper now builds DATA_SOURCE_URI from the
secret's host and sslmode keys when the database is external, and is
unchanged for the sidecar on internal databases. The Service carries
app: <database name>, which Prometheus maps to the job label the
dashboards group by.

Also corrects the external-databases example README, which named the
secret key pgsslmode; the chart reads sslmode.

Closes #388

Amp-Thread-ID: https://ampcode.com/threads/T-01a1572d-c025-75ec-bf69-eaddc520e31c
Co-authored-by: Amp <amp@ampcode.com>
@github-actions

Copy link
Copy Markdown

…positive

helm-unittest v1.0.2 short-circuits containsDocument with any: true when
run with -q, as CI does, so assert by documentIndex instead. Both
templates now iterate the databases in the same list order, so the
indexes are predictable. Use the same existingSecret value as
pgsqlAuth_test.yaml, since Checkov flagged the previous one as a secret.

Amp-Thread-ID: https://ampcode.com/threads/T-01a1572d-c025-75ec-bf69-eaddc520e31c
Co-authored-by: Amp <amp@ampcode.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add toggle to deploy db exporters for external databases

2 participants