Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 15 additions & 1 deletion .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -1307,6 +1307,19 @@ jobs:
- name: Check out code into the Go module directory
uses: actions/checkout@v7.0.1

- name: Setup Go for signing fixtures
uses: actions/setup-go@v7.0.0
with:
go-version-file: go.mod
cache: false

- name: Build Linux signing fixture generator
env:
GOOS: linux
GOARCH: amd64
CGO_ENABLED: '0'
run: go build -o build/signing-fixtures ./test/tools/signing-fixtures

- name: Download Artifact
# uses: actions/download-artifact@v3
uses: actions/download-artifact@v8.0.1
Expand Down Expand Up @@ -1341,6 +1354,7 @@ jobs:
name: Stackql permissions
run: |
chmod a+rwx build/stackql
chmod a+rx build/signing-fixtures
ls -al build/stackql

- shell: wsl-bash -u root {0}
Expand Down Expand Up @@ -1381,7 +1395,7 @@ jobs:
fi
export PYTHONPATH="$(pwd)/test/python"
source .venv/bin/activate;
python3 cicd/python/build.py --robot-test --config='{ "variables": { "IS_WSL": true } }'
python3 cicd/python/build.py --robot-test --config='{ "variables": { "IS_WSL": true, "SIGNATURE_GENERATOR": "build/signing-fixtures" } }'

- shell: wsl-bash {0}
name: Output from mocked functional tests
Expand Down
20 changes: 20 additions & 0 deletions docs/registry_contribution.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,26 @@ Iterate upon steps 2 and 3 until API coverage and functionality fulfill your req

As per [the Provider Registry Contributor Guide](https://github.com/stackql/stackql-provider-registry/blob/main/.github/CONTRIBUTING.md). The team will review as rapidly as possible and work with you to complete the integration. Once this is done, your functionality with be available via `registry pull...`

## Quantum-safe signature verification

StackQL continues to disable verification by default. Explicitly set
`"verifyConfig":{"nopVerify":false}` in `--registry` to require embedded
ML-DSA-65 signatures on provider and service document reads, including
omnisdk's preview/document-backed paths. This uses FIPS 204 `crypto/mldsa`
from Go 1.27; certificate options and `.sig` sidecars are no longer used.

For private registries, `verifyConfig.publicKeys` is a JSON object mapping
key IDs to hex ML-DSA-65 public keys. It replaces the embedded production
roots; an explicit empty keyset is an error.

The registry signs exact YAML payload bytes using its shared `signing/pqsig`
module. Edits, including formatting or newline changes, invalidate signatures.
Use the registry's `pqsig sign-doc` tool to re-sign a document.
Unsigned local development documents must keep `nopVerify: true`.
Old sidecar-only archives require republishing before verified use.
Archive download/extraction authentication and rollback protection are separate
work; this option authenticates documents when they are read.

## Example integration

In this walkthrough, we demonstrate the bare bones of integrating a simple Provider,
Expand Down
5 changes: 3 additions & 2 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,12 @@ require (
github.com/spf13/cobra v1.10.2
github.com/spf13/pflag v1.0.10
github.com/spf13/viper v1.10.1
github.com/stackql-labs/omnisdk v0.1.6-alpha05
github.com/stackql/any-sdk v0.6.1-beta01
github.com/stackql-labs/omnisdk v0.1.7-alpha01
github.com/stackql/any-sdk v0.6.1-beta02
github.com/stackql/go-suffix-map v0.0.1-alpha02
github.com/stackql/psql-wire v0.1.2-beta02
github.com/stackql/stackql-parser v0.0.16-beta01
github.com/stackql/stackql-provider-registry/signing v0.1.0-alpha01
github.com/stretchr/testify v1.11.1
golang.org/x/mod v0.38.0
golang.org/x/sync v0.22.0
Expand Down
10 changes: 6 additions & 4 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -367,10 +367,10 @@ github.com/spf13/viper v1.10.1 h1:nuJZuYpG7gTj/XqiUwg8bA0cp1+M2mC3J4g5luUYBKk=
github.com/spf13/viper v1.10.1/go.mod h1:IGlFPqhNAPKRxohIzWpI5QEy4kuI7tcl5WvR+8qy1rU=
github.com/spiffe/go-spiffe/v2 v2.7.0 h1:uXe1MflJoHw58wAUvxVlcM7WpKtijWG7I1UidcGh6g4=
github.com/spiffe/go-spiffe/v2 v2.7.0/go.mod h1:47Q0Q9/AqGha8QLHp+kxpH4Wca7X7EnOtlIJy3mxZ3U=
github.com/stackql-labs/omnisdk v0.1.6-alpha05 h1:o4dcS21CFLRvBH4cNtTWyVroTrRELz/42/tgKGrIs4E=
github.com/stackql-labs/omnisdk v0.1.6-alpha05/go.mod h1:bMRymDVvjy9j7v7nm01LugE5mF2I85ClmsjiUzURZ+I=
github.com/stackql/any-sdk v0.6.1-beta01 h1:2Jqru76BDS87Jd95n5P56ruUoB0NWQZEbVQIXa3wlOs=
github.com/stackql/any-sdk v0.6.1-beta01/go.mod h1:xUR2t/dy8d01Ut4n3WdMe1lfxG5ipq/VNwa9dXDXI/o=
github.com/stackql-labs/omnisdk v0.1.7-alpha01 h1:DZ8wLZSZymbeJNxfIM0+bsgm49q0geEhowhKgQ0qp9g=
github.com/stackql-labs/omnisdk v0.1.7-alpha01/go.mod h1:P3pr9kvDTMox+4xCLcFdzOS1wMMe3ho9i6j2Xl+/Yog=
github.com/stackql/any-sdk v0.6.1-beta02 h1:O+PiVsbjMh2165p6ek5vD7xnO+8ji7zxPpT2rEtAHos=
github.com/stackql/any-sdk v0.6.1-beta02/go.mod h1:M9b57jwOLa7afqZE5ZSTzN3YnaRGslJBIAdD5sz83Nw=
github.com/stackql/go-suffix-map v0.0.1-alpha02 h1:JI4aiq/OfHW4a/BiNqK31NbyaR94zlUOhGZof3G/OUo=
github.com/stackql/go-suffix-map v0.0.1-alpha02/go.mod h1:6BSMz4K/gIYfqdVRb7V8xL165wiPYyCjD9Kx7hLMd9Y=
github.com/stackql/psql-wire v0.1.2-beta02 h1:7DfliBpyyCaUHVJ1Jxk37q3zVwVLyh6+FCHeM0/gbAQ=
Expand All @@ -381,6 +381,8 @@ github.com/stackql/stackql-parser v0.0.16-beta01 h1:U3r/zFgnFq4PKYIcFGvwds20tw2+
github.com/stackql/stackql-parser v0.0.16-beta01/go.mod h1:Ar4KzxFUFA6oLcXIZGVr1RIeEH47H3xRP08LJCry924=
github.com/stackql/stackql-provider-registry v0.0.2-alpha01 h1:fPz6MhfXaMC/VPNJnjUo0sZyjTYhlifzaUWZ9Yj8NVE=
github.com/stackql/stackql-provider-registry v0.0.2-alpha01/go.mod h1:87rVxnS2aRASK20lBQgoYA0o7FSJTZBGGRaWFR7IDm4=
github.com/stackql/stackql-provider-registry/signing v0.1.0-alpha01 h1:KtrwwTJLMWn5+yEzguADFx96pA59k7P2JxmpAKXae6g=
github.com/stackql/stackql-provider-registry/signing v0.1.0-alpha01/go.mod h1:2rrVmn6zLIonwq72QX50qtixWRZqx61tSDxPeOt/E1s=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.2.0/go.mod h1:qt09Ya8vawLte6SNmTgCsAVtYtaKzEcn8ATUoHMkEqE=
Expand Down
26 changes: 21 additions & 5 deletions internal/stackql/intrinsic/doc.go
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,11 @@ func docResourceTables(ctx queryContext, bundle, service string) ([]table, error
if err != nil {
return nil, err
}
resources, err := omnisdk.DocResources(dir, bundle, service)
opts, err := documentOptions(ctx)
if err != nil {
return nil, err
}
resources, err := omnisdk.DocResources(dir, bundle, service, opts...)
if err != nil {
return nil, err
}
Expand All @@ -129,7 +133,11 @@ func docMethods(ctx queryContext, bundle, service, resource string) ([]relationM
if err != nil {
return nil, err
}
methods, err := omnisdk.DocMethods(dir, bundle, service, resource)
opts, err := documentOptions(ctx)
if err != nil {
return nil, err
}
methods, err := omnisdk.DocMethods(dir, bundle, service, resource, opts...)
if err != nil {
return nil, err
}
Expand Down Expand Up @@ -223,6 +231,10 @@ func openDocQuery(ctx queryContext, translated docQuery) (omnisdk.Rows, string,
}

func openDocQueryIn(ctx queryContext, translated docQuery, runCtx context.Context) (omnisdk.Rows, string, error) {
opts, err := documentOptions(ctx)
if err != nil {
return nil, "", err
}
registry := registryRoot(ctx)
q := translated.getQuery()
relations := queryRelations(translated)
Expand All @@ -233,7 +245,7 @@ func openDocQueryIn(ctx queryContext, translated docQuery, runCtx context.Contex
if relation, ok := relations[join.Resource().Handle()]; ok {
tbl = relation
} else {
tbl, describeErr = omnisdk.DescribeTable(registry, join.Resource().Handle())
tbl, describeErr = omnisdk.DescribeTable(registry, join.Resource().Handle(), opts...)
}
if describeErr != nil {
return nil, "", describeErr
Expand All @@ -243,7 +255,7 @@ func openDocQueryIn(ctx queryContext, translated docQuery, runCtx context.Contex
var relation string
if target := q.Target(); target != nil {
tbl, describeErr := omnisdk.DescribeMutation(
registry, target.Resource().Handle(), target.Verb().String())
registry, target.Resource().Handle(), target.Verb().String(), opts...)
if describeErr != nil {
return nil, "", describeErr
}
Expand Down Expand Up @@ -340,7 +352,11 @@ func showDocServices(
if err != nil {
return internaldto.NewErroneousExecutorOutput(err)
}
services, err := omnisdk.DocServices(dir, bundle)
opts, err := documentOptions(ctx)
if err != nil {
return internaldto.NewErroneousExecutorOutput(err)
}
services, err := omnisdk.DocServices(dir, bundle, opts...)
if err != nil {
return internaldto.NewErroneousExecutorOutput(err)
}
Expand Down
6 changes: 5 additions & 1 deletion internal/stackql/intrinsic/doc_views.go
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,11 @@ func expandDocumentViews(
if err != nil {
return nil, err
}
view, exists, err := omnisdk.DocView(root, bundle, service, resource)
opts, err := documentOptions(ctx)
if err != nil {
return nil, err
}
view, exists, err := omnisdk.DocView(root, bundle, service, resource, opts...)
if err != nil {
return nil, err
}
Expand Down
5 changes: 5 additions & 0 deletions internal/stackql/intrinsic/preview_services.go
Original file line number Diff line number Diff line change
Expand Up @@ -153,12 +153,17 @@ func streamPlan(
// over after the control predicates, the credential for the cloud in play, the
// backend tuning, and the backend's SQL dialect.
func previewArgs(ctx queryContext, cloud string, params map[string]string) (omnisdk.Args, error) {
verifier, err := documentVerifier(ctx.GetRuntimeContext())
if err != nil {
return omnisdk.Args{}, err
}
dialect, err := backendDialect(ctx)
if err != nil {
return omnisdk.Args{}, err
}
input := previewCfg
return omnisdk.Args{
DocumentVerifier: verifier,
Params: params,
AuthByProvider: providerAuthByProvider(ctx, cloud),
Endpoint: input.getEndpoint(),
Expand Down
7 changes: 5 additions & 2 deletions internal/stackql/intrinsic/relation_reads.go
Original file line number Diff line number Diff line change
Expand Up @@ -388,8 +388,11 @@ func declaredRelationColumns(
if relation, present := relations[handle]; present {
table = relation
} else {
var err error
table, err = omnisdk.DescribeTable(registryRoot(ctx), handle)
opts, err := documentOptions(ctx)
if err != nil {
return nil, err
}
table, err = omnisdk.DescribeTable(registryRoot(ctx), handle, opts...)
if err != nil {
return nil, err
}
Expand Down
2 changes: 1 addition & 1 deletion internal/stackql/intrinsic/staged_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -161,7 +161,7 @@ func newStagingTestCtx(t *testing.T, endpoint string) *stagingTestCtx {
return &stagingTestCtx{
engine: engine,
runtime: dto.RuntimeCtx{
RegistryRaw: `{"url":"file:` + filepath.ToSlash(filepath.Join(dir, "registry")) + `"}`,
RegistryRaw: `{"url":"file:` + filepath.ToSlash(filepath.Join(dir, "registry")) + `","verifyConfig":{"nopVerify":true}}`,
SQLBackendCfgRaw: sqlBackendRaw,
},
typCfg: typCfg,
Expand Down
38 changes: 38 additions & 0 deletions internal/stackql/intrinsic/verification.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
package intrinsic

import (
"fmt"

"github.com/stackql-labs/omnisdk/pkg/docparse/stackqldoc"
"github.com/stackql/any-sdk/pkg/dto"
"github.com/stackql/any-sdk/public/formulation"
"github.com/stackql/stackql-provider-registry/signing/pqsig"
"gopkg.in/yaml.v2"
)

//nolint:nilnil // A nil verifier is the explicit, successful opt-out state.
func documentVerifier(runtime dto.RuntimeCtx) (pqsig.Verifier, error) {
var cfg formulation.RegistryConfig
if err := yaml.Unmarshal([]byte(runtime.RegistryRaw), &cfg); err != nil {
return nil, fmt.Errorf("document verification config: %w", err)
}
var keys map[string]string
if cfg.VerifyConfig != nil {
if cfg.VerifyConfig.NopVerify {
return nil, nil
}
keys = cfg.VerifyConfig.PublicKeys
}
return pqsig.NewVerifier(keys)
}

func documentOptions(ctx queryContext) ([]stackqldoc.Option, error) {
verifier, err := documentVerifier(ctx.GetRuntimeContext())
if err != nil {
return nil, err
}
if verifier == nil {
return nil, nil
}
return []stackqldoc.Option{stackqldoc.WithVerifier(verifier)}, nil
}
44 changes: 44 additions & 0 deletions internal/stackql/intrinsic/verification_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
package intrinsic //nolint:testpackage // tests the unexported verification configuration

import (
"crypto/mldsa"
"encoding/hex"
"encoding/json"
"testing"

"github.com/stackql/any-sdk/pkg/dto"
"github.com/stackql/stackql-provider-registry/signing/pqsig"
)

func TestDocumentVerificationConfig(t *testing.T) {
v, err := documentVerifier(dto.RuntimeCtx{RegistryRaw: `{"verifyConfig":{"nopVerify":true}}`})
if err != nil || v != nil {
t.Fatalf("verification opt-out: %v", err)
}
key, err := mldsa.GenerateKey(mldsa.MLDSA65())
if err != nil {
t.Fatal(err)
}
config, err := json.Marshal(map[string]any{"verifyConfig": pqsig.VerifierConfig{
PublicKeys: map[string]string{"test": hex.EncodeToString(key.PublicKey().Bytes())},
}})
if err != nil {
t.Fatal(err)
}
v, err = documentVerifier(dto.RuntimeCtx{RegistryRaw: string(config)})
if err != nil {
t.Fatal(err)
}
doc, err := pqsig.Sign([]byte("name: example\n"), "test", key)
if err != nil {
t.Fatal(err)
}
if _, err := v.Verify(doc); err != nil {
t.Fatal(err)
}
for _, cfg := range []string{`{`, `{"verifyConfig":{"publicKeys":{}}}`, `{"verifyConfig":{"publicKeys":{"test":"invalid"}}}`} {
if _, err := documentVerifier(dto.RuntimeCtx{RegistryRaw: cfg}); err == nil {
t.Fatalf("accepted invalid config %s", cfg)
}
}
}
Loading
Loading