Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -290,6 +290,10 @@ StackQL is available for Windows, MacOS, Linux, Docker, GitHub Actions and more.
docker pull stackql/stackql
```

> Want findings, not infrastructure? Run a read-only cross-cloud security &
> FinOps audit with the standard image in one command — see the
> [__Docker audit quickstart__](/docs/audit.md).

</details>

<details>
Expand Down
37 changes: 37 additions & 0 deletions cicd/audit/.env.audit.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# Docker audit quickstart — fill in the cloud(s) you have; blank ones are skipped.
# cp cicd/audit/.env.audit.example cicd/audit/.env.audit # then fill in the values
# docker compose -f compose.audit.yaml run --rm audit

# --- AWS (temporary/session credentials) ---
AWS_ACCESS_KEY_ID=
AWS_SECRET_ACCESS_KEY=
AWS_SESSION_TOKEN=
AWS_REGION=us-east-1

# --- GCP (service-account key JSON, inline on one line) ---
# Whole-org sweep: set GOOGLE_ORG_ID (digits only) to descend all projects.
GCP_SA_JSON=
GOOGLE_ORG_ID=

# --- Azure / Entra ---
# Whole-tenant sweep by default; set AZURE_MGMT_GROUP to scope to one mgmt group.
AZURE_TENANT_ID=
AZURE_CLIENT_ID=
AZURE_CLIENT_SECRET=
AZURE_MGMT_GROUP=

# --- Pinned audit engine (queries + scripts downloaded at run time) ---
# Per-target wall-clock cap in seconds (prevents a slow target hanging the run)
STACKQL_DEEP_TIMEOUT=300

AUDIT_ENGINE_REF=4d02dccad5d9b9a79d04c4c48af88e1fb4149aff

# --- Pinned provider registry versions (reproducible audits) ---
AWS_PROVIDER_VERSION=v26.07.00414
GOOGLE_PROVIDER_VERSION=v25.12.00357
AZURE_PROVIDER_VERSION=v26.07.00420
ENTRA_PROVIDER_VERSION=v26.06.00405

# keep the audit light touch by default
STACKQL_AUDIT_SKIP='*storage* s3/*' python3 scripts/discover.py gcp-org

2 changes: 2 additions & 0 deletions cicd/audit/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@

.env.audit
25 changes: 25 additions & 0 deletions cicd/audit/fetch.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
#!/usr/bin/env python3
"""Download the pinned audit engine (run.sh + queries + scripts) from
stackql/stackql-audit-action into /tmp/engine. Stdlib only — no curl/git needed.
Pinned by AUDIT_ENGINE_REF; subdir/repo overridable via env."""
import io
import os
import tarfile
import urllib.request

repo = os.environ.get("AUDIT_ENGINE_REPO", "stackql/stackql-audit-action")
ref = os.environ.get("AUDIT_ENGINE_REF", "main")
sub = os.environ.get("AUDIT_ENGINE_SUBDIR", "cicd/audit/quickstart").strip("/") + "/"

url = f"https://codeload.github.com/{repo}/tar.gz/{ref}"
req = urllib.request.Request(url, headers={"User-Agent": "curl/8"}) # github 403s default urllib UA
data = urllib.request.urlopen(req).read()

prefix = f"{repo.split('/')[-1]}-{ref}/{sub}"
with tarfile.open(fileobj=io.BytesIO(data), mode="r:gz") as t:
for m in t.getmembers():
if m.name.startswith(prefix):
m.name = m.name[len(prefix):]
if m.name:
t.extract(m, "/tmp/engine", filter="data")
print(f"fetched {repo}@{ref} ({sub}) -> /tmp/engine")
2 changes: 2 additions & 0 deletions cicd/audit/output/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
*
!.gitignore
12 changes: 12 additions & 0 deletions cicd/audit/run.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
#!/usr/bin/env bash
# Docker audit quickstart bootstrap (this repo's only audit logic).
# Downloads the pinned audit engine — run.sh + queries + scripts — from
# stackql/stackql-audit-action (cicd/audit/quickstart/) and hands off to its
# run.sh. All run logic lives in that engine; here we only pin the version.
# The stackql server + provider pins live in docker-compose.audit.yaml.
set -euo pipefail

python3 /audit/fetch.py

[ -f /tmp/engine/run.sh ] || { echo "FATAL: engine download failed (no run.sh in ${AUDIT_ENGINE_SUBDIR:-cicd/audit/quickstart})"; exit 1; }
exec bash /tmp/engine/run.sh
32 changes: 32 additions & 0 deletions docker-compose.audit.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# Docker audit quickstart — cross-cloud StackQL security/FinOps audit.
# cp cicd/audit/.env.audit.example cicd/audit/.env.audit # add read-only creds
# docker compose -f compose.audit.yaml run --rm audit
#
# stackql/stackql runs as the query server; a stock python image downloads the
# pinned audit engine and drives it over the Postgres wire. Empty clouds are
# skipped. Findings -> cicd/audit/output/.
services:
stackql:
image: ${STACKQL_IMAGE:-stackql/stackql:latest}
env_file: [cicd/audit/.env.audit]
command:
- bash
- -c
- |
for pv in aws:$${AWS_PROVIDER_VERSION:-v26.07.00414} google:$${GOOGLE_PROVIDER_VERSION:-v25.12.00357} azure:$${AZURE_PROVIDER_VERSION:-v26.07.00420} entra_id:$${ENTRA_PROVIDER_VERSION:-v26.06.00405}; do
stackql exec "registry pull $${pv%%:*} $${pv##*:};"
done
exec stackql srv --pgsrv.address=0.0.0.0 --pgsrv.port=5444 \
--auth='{"aws":{"type":"aws_signing_v4","keyIDenvvar":"AWS_ACCESS_KEY_ID","credentialsenvvar":"AWS_SECRET_ACCESS_KEY"},"google":{"type":"service_account","credentialsenvvar":"GCP_SA_JSON"},"azure":{"type":"azure_default"}}'

audit:
image: python:3.11-slim
depends_on: [stackql]
env_file: [cicd/audit/.env.audit]
environment:
- STACKQL_DSN=postgresql://stackql:stackql@stackql:5444/stackql?sslmode=disable
volumes:
- ./cicd/audit/run.sh:/audit/run.sh:ro
- ./cicd/audit/fetch.py:/audit/fetch.py:ro
- ./cicd/audit/output:/audit/output:rw
command: ["bash", "/audit/run.sh"]
49 changes: 49 additions & 0 deletions docs/audit.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# Docker audit quickstart

Run a read-only cross-cloud (AWS / GCP / Azure + Entra) security & FinOps audit
with one command.

```bash
cp cicd/audit/.env.audit.example cicd/audit/.env.audit
# fill in read-only creds for the cloud(s) you want to audit
docker compose -f docker-compose.audit.yaml pull # get the standard stackql image
docker compose -f docker-compose.audit.yaml run --rm audit
```

> Pin the image for reproducibility by setting `STACKQL_IMAGE` (e.g.
> `STACKQL_IMAGE=stackql/stackql:v0.5.888`) in `cicd/audit/.env.audit`; it
> defaults to `stackql/stackql:latest`.

Findings land in [`cicd/audit/output/findings.json`](../cicd/audit/output/).
A cloud you don't configure is simply skipped.

## What it runs

Two stock images ([docker-compose.audit.yaml](../docker-compose.audit.yaml)):

| Service | Image | Role |
| --- | --- | --- |
| `stackql` | `stackql/stackql` | runs as a Postgres-wire **server** holding your read-only `--auth` |
| `audit` | `python:3.11-slim` | downloads the audit engine and drives the queries against the server |

The audit engine — the check `queries/`, `filters.py`, and `scripts/discover.py`
(S3 deep scan, AWS region sweep, GCP org descent, Azure management-group descent) —
is **not** vendored here. `run.sh` downloads it at run time from
[`stackql/stackql-audit-action`](https://github.com/stackql/stackql-audit-action)
(`cicd/audit/quickstart/`), pinned by `AUDIT_ENGINE_REF` in `.env.audit`. The
action repo is the single source of truth; this repo carries only the thin
quickstart (compose + `run.sh` + docs).

## Configure credentials

Edit `cicd/audit/.env.audit` — fill in the plain credential values for the
cloud(s) you have. A cloud left blank is skipped; no JSON to assemble.

- **AWS** — `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` (+ `AWS_SESSION_TOKEN`
for temporary creds) and `AWS_REGION`.
- **GCP** — `GCP_SA_JSON` (whole SA key JSON, inline on one line); `GOOGLE_ORG_ID`
(digits) to sweep the whole org's projects.
- **Azure / Entra** — `AZURE_TENANT_ID` / `AZURE_CLIENT_ID` / `AZURE_CLIENT_SECRET`
(+ optional `AZURE_MGMT_GROUP` to scope; blank = whole tenant).

Required read-only roles per cloud: [`stackql-audit-action/docs/required-auth.md`](https://github.com/stackql/stackql-audit-action/blob/main/docs/required-auth.md).
Loading