Skip to content

test(coderd): remove low-value cron and chatwebhook tests - #32164

Merged
ethanndickson merged 2 commits into
mainfrom
ethan/prune-cron-chatwebhook-tests
Nov 18, 2026
Merged

ethanndickson merged 2 commits into
mainfrom
ethan/prune-cron-chatwebhook-tests

Conversation

@ethanndickson

@ethanndickson ethanndickson commented Nov 13, 2026 •

Copy link
Copy Markdown
Member

The tests from #32063 and #32062 are two to three times the size of the code they cover, and a lot of them can't fail for a reason we'd care about. This PR removes those, with no production changes.

In coderd/schedule/cron, two tests rebuilt the schedule rule a second time to compare against, which was also the only use of export_test.go, so that goes too. Most of the rest, in both packages, are table rows that take the same branch as a row that stays. I also dropped a copy of the webhook scheme list and a repeated-request test that belongs at the route.

What's left is a row per reject branch, the RFC cases, the regression test for the Next hang and the constant-time scan. I replayed each fix commit from the two PRs against the smaller suite, and the pre-fix code still fails a test for every one, other than a fix to the wording of a zone error.

One row that stays is changed: the bearer wrong-scheme case used Basic, which was only rejected because of where its space lands, so nothing covered the scheme word. It's now Digest.

#32162 overlaps with this in cron. With the shim's users gone it no longer needs to touch that package, so it's now stacked on top of this PR.

@ethanndickson
ethanndickson temporarily deployed to main November 13, 2026 09:15 — with GitHub Actions Inactive
@ethanndickson
ethanndickson temporarily deployed to main November 13, 2026 09:15 — with GitHub Actions Inactive
@ethanndickson
ethanndickson temporarily deployed to main November 13, 2026 09:19 — with GitHub Actions Inactive

@ethanndickson ethanndickson left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Notes on why each test or group of rows goes, since the diff alone doesn't show it. Most are rows that take the same branch as a row that stays. The rest are tests that compared the code with a second implementation written in the test, tests that pinned behaviour nothing asks for, and one test-only export. Where a removal means some change would no longer fail a test, the comment says so.

Generated by an agent on behalf of the user

Comment on lines -33 to -46
{
name: "EveryFiveMinutes",
spec: "*/5 * * * *",
zone: "UTC",
wantCron: "*/5 * * * *",
wantString: "CRON_TZ=UTC */5 * * * *",
},
{
name: "Weekdays",
spec: "0 9 * * 1-5",
zone: "America/New_York",
wantCron: "0 9 * * 1-5",
wantString: "CRON_TZ=America/New_York 0 9 * * 1-5",
},

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These two rows, and ZoneUnderEtc and LeapDay below, take the same accept path as the two rows that stay, and the assertions only echo the spec and the zone back. AllFieldsRestricted sets every field, and ExtraSpacesAreNormalized is the one row where the output differs from the input.

zone string
wantErr string
}{
{name: "EmptySpec", spec: "", zone: "UTC", wantErr: "empty"},

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

EmptySpec takes the same branch as BlankSpec: both have no fields after the split. Two more rows below go for the same reason. MinuteOutOfRange is a robfig parser error that Standard passes on, like NotANumber, and NeverRunsInShortMonth hits the same never-runs check as NeverRuns.

Comment on lines -118 to -123
{name: "PathToLocaltimeDoubleSlash", spec: "0 9 * * *", zone: ".//localtime", wantErr: "is not a zone name"},
{name: "PathToLocaltimeTwoDots", spec: "0 9 * * *", zone: "././localtime", wantErr: "is not a zone name"},
{name: "PathToPosixrules", spec: "0 9 * * *", zone: "./posixrules", wantErr: "is not a zone name"},
{name: "PathToFactory", spec: "0 9 * * *", zone: "./Factory", wantErr: "is not a zone name"},
{name: "PathThroughParent", spec: "0 9 * * *", zone: "Etc/./../localtime", wantErr: "is not a zone name"},
{name: "PathToZone", spec: "0 9 * * *", zone: "./America/New_York", wantErr: "is not a zone name"},

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isZoneName rejects a name for three reasons: an empty segment, a . segment or a .. segment. One row for each stays (DoubleSlashInZone, PathToLocaltime, ParentPath). These six rows, and the five removed just below, spell the same three reasons in other ways.

Comment on lines -135 to -138
{name: "RightUTC", spec: "0 9 * * *", zone: "right/UTC", wantErr: "is not a zone name"},
{name: "PosixUTC", spec: "0 9 * * *", zone: "posix/UTC", wantErr: "is not a zone name"},
{name: "PosixFactory", spec: "0 9 * * *", zone: "posix/Factory", wantErr: "is not a zone name"},
{name: "RightFactory", spec: "0 9 * * *", zone: "right/Factory", wantErr: "is not a zone name"},

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same check as RightZone and PosixZone above. Only the first segment is read, so what follows right/ or posix/ makes no difference.

{name: "PosixFactory", spec: "0 9 * * *", zone: "posix/Factory", wantErr: "is not a zone name"},
{name: "RightFactory", spec: "0 9 * * *", zone: "right/Factory", wantErr: "is not a zone name"},
// The files that are not zones, in any directory.
{name: "FactoryInADirectory", spec: "0 9 * * *", zone: "Etc/Factory", wantErr: `"Etc/Factory" is not supported`},

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isNotZone reads the last segment only, so LocaltimeInADirectory covers the directory case for Factory and posixrules too. Each file name keeps its own top-level row.

Comment on lines -113 to -119
{name: "signature empty", header: hdr(id, ts, ""), body: body, secret: secret},
{name: "signature version only", header: hdr(id, ts, "v1"), body: body, secret: secret},
{name: "signature with no MAC", header: hdr(id, ts, "v1,"), body: body, secret: secret},
{name: "signature not base64", header: hdr(id, ts, "v1,not base64!"), body: body, secret: secret},
{name: "signature with trailing data", header: hdr(id, ts, valid+",extra"), body: body, secret: secret},
{name: "signature with the version in upper case", header: hdr(id, ts, "V1"+strings.TrimPrefix(valid, "v1")), body: body, secret: secret},
{name: "signature with another version", header: hdr(id, ts, "v2"+strings.TrimPrefix(valid, "v1")), body: body, secret: secret},

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Each of these is an entry that the loop skips or that fails the comparison, and a remaining row takes each of those paths. other versions and junk before the valid entry has an entry with no comma, another version and bad base64, and signature truncated has a short MAC. The same goes for signature without a version, the v1a beside v1 row and extra spaces between entries below. The upper-case V1 and trailing-data rows also pinned extra strictness, and no test fails now if that's relaxed.

Comment on lines -136 to -144
{name: "body emptied", header: hdr(id, ts, valid), body: nil, secret: secret},
{name: "signature for another id", header: hdr(id, ts, sign(key, "msg_other", ts, body)), body: body, secret: secret},
{name: "signature for another timestamp", header: hdr(id, ts, sign(key, id, stamp(time.Second), body)), body: body, secret: secret},
{name: "signature over timestamp and body only", header: hdr(id, ts, "v1,"+base64.StdEncoding.EncodeToString(hmacSHA256(key, []byte(ts+"."+string(body))))), body: body, secret: secret},
{name: "signature over id and body only", header: hdr(id, ts, "v1,"+base64.StdEncoding.EncodeToString(hmacSHA256(key, []byte(id+"."+string(body))))), body: body, secret: secret},
{name: "signature over the parts in another order", header: hdr(id, ts, "v1,"+base64.StdEncoding.EncodeToString(hmacSHA256(key, []byte(ts+"."+id+"."+string(body))))), body: body, secret: secret},
{name: "signature over the parts without dots", header: hdr(id, ts, "v1,"+base64.StdEncoding.EncodeToString(hmacSHA256(key, []byte(id+ts+string(body))))), body: body, secret: secret},
{name: "signature over the body only", header: hdr(id, ts, "v1,"+base64.StdEncoding.EncodeToString(hmacSHA256(key, body))), body: body, secret: secret},
{name: "signature with the prefix inside the key", header: hdr(id, ts, sign([]byte(secret), id, ts, body)), body: body, secret: secret},

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All of these, and signature for another body above, are a MAC over something other than id.timestamp.body, so they fail the one comparison that body tampered and wrong secret fail. Leaving the id, the timestamp or the body out of the signed string still fails the valid rows.

Comment on lines -148 to -153
{name: "secret without the prefix", header: hdr(id, ts, valid), body: body, secret: strings.TrimPrefix(secret, "whsec_")},
{name: "secret with the prefix in upper case", header: hdr(id, ts, valid), body: body, secret: "WHSEC_" + strings.TrimPrefix(secret, "whsec_")},
{name: "secret with the spare bits of the last base64 character set", header: hdr(id, ts, valid), body: body, secret: "whsec_" + withSpareBitsSet(t, base64.StdEncoding.EncodeToString(key))},
{name: "secret not base64", header: hdr(id, ts, valid), body: body, secret: "whsec_not base64!"},
{name: "secret empty", header: hdr(id, ts, valid), body: body, secret: ""},
{name: "secret prefix only", header: hdr(id, ts, valid), body: body, secret: "whsec_"},

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The format of the secret is checked by one function, which TestStandardWebhooksValidateSecret covers directly, so these rows repeated that table through Verify. signature made with an empty key still takes the reject for an unusable secret. The upper-case prefix and spare-bits rows are the extra strictness described at the top of this file.

Comment on lines -169 to -171
"64 bytes": "whsec_" + base64.StdEncoding.EncodeToString(make([]byte, 64)),
"base64 with padding": "whsec_" + base64.StdEncoding.EncodeToString([]byte("0123456789abcdef0123456789abcdef")),
"base64 with plus and slash": "whsec_" + base64.StdEncoding.EncodeToString([]byte{0xfb, 0xff, 0xbf, 0xfb, 0xff, 0xbf}),

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

official secret and one byte stay (they only show as changed because of the alignment). These three are more valid base64 through the same decode. one byte is the row that matters: it shows we don't enforce the 24-byte minimum from the spec.

Comment on lines -187 to -199
"empty": "",
"prefix only": "whsec_",
"no prefix": strings.TrimPrefix(official, "whsec_"),
"prefix in upper case": "WHSEC_" + strings.TrimPrefix(official, "whsec_"),
"another prefix": "whsk_" + strings.TrimPrefix(official, "whsec_"),
"prefix twice": "whsec_" + official,
"not base64": "whsec_not base64!",
"base64 url alphabet": "whsec_" + base64.URLEncoding.EncodeToString([]byte{0xfb, 0xff, 0xbf}),
"base64 without padding": "whsec_" + base64.RawStdEncoding.EncodeToString([]byte("0123456789abcdef0123456789abcdef")),
"base64 with a newline": official + "\n",
"base64 with a space": official + " ",
"base64 with a leading space": " " + official,
"non-ASCII": "whsec_é",

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One row per reject reason stays: no prefix, nothing after the prefix, the URL alphabet, missing padding and a line break (they show as removed and added because of the alignment). empty, another prefix and a leading space fail the prefix check. prefix twice, not base64, a trailing space and non-ASCII fail the decode. prefix in upper case pinned that the prefix is case-sensitive, which nothing asks for.

@ethanndickson

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-18T02:48:13.735074Z cd4cb10 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b28d91c9cc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/schedule/cron/standard_test.go Resolved
@ethanndickson

Copy link
Copy Markdown
Member Author

@codex review

re: Retain the cross-zone clock-change oracle

Not taking this one. The clock-change checks didn't narrow to one instant: TestStandard_Next_DST keeps fixed rows for New York, Lord Howe (a 30-minute change) and Chatham (+12:45), and TestStandard_Next_InsideRepeatedPeriod keeps Juneau 1867 (a 24-hour repeat). TestStandard_Next_Terminates is only the regression for the hang.

We checked whether the model saw anything those rows miss:

  • Of 31 faults we put into the clock-change code, the model caught 10. The remaining rows catch all 10, and 7 more that the model missed.
  • On the code that hung, the model passed in all 80 clock-change windows, in every zone and year and in both copies of the zone data. Its only failures were the hang on 31 December of a leap year, which the remaining case still reproduces.

resolveWallClock only asks Go for the offset at an instant, so it can't tell a rule-based year from a listed one. The one bug that depended on rule-based data came from reading period bounds, and that keeps its regression. A compact oracle would be a second implementation of the schedule rules in the test, with no known regression that only it fails on.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b28d91c9cc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/schedule/cron/standard_test.go Resolved
@ethanndickson

Copy link
Copy Markdown
Member Author

@codex review

re: Keep the legacy DST behavior covered

Not taking this one. Both changes you name still fail a test. Weekly and Daily share parse and the one branch in Next, so either change moves Daily too, and TestCalculateAutoStop/DaylightSavings/QuietHoursFallsOnDstSwitch/In in coderd/schedule fails. It expects Sydney quiet hours at 02:30 to skip to the next day, and gets 03:00 on the day of the change.

We made each change and ran the tests:

  • parse sets wallClock: that test fails.
  • Next always takes the wall-clock path: that test fails.
  • Only Daily sets wallClock: that test fails.
  • Only Weekly sets wallClock: we found no test that fails.

So what's no longer covered is a change to Weekly alone. That would be new code in Weekly, not a slip in a refactor. The removed subtest pinned that a repeated time runs twice, which is robfig/cron's behaviour and not something autostart asks for. If we want Weekly pinned across a clock change, that test belongs with autostart, the way quiet hours has its own.

Still not taking "Retain the cross-zone clock-change oracle" either: the fixed rows that stay catch every fault the model caught, and the model is a second implementation of the schedule rules.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Already looking forward to the next diff.

Reviewed commit: b28d91c9cc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@ethanndickson
ethanndickson marked this pull request as ready for review November 13, 2026 10:22
@ethanndickson
ethanndickson temporarily deployed to main November 13, 2026 10:22 — with GitHub Actions Inactive
Removes tests that model the code under test a second time, duplicate another row, pin behaviour the RFC does not ask for, or read source text. One test per reject branch, the RFC cases, the hang regression and the secret leak test stay. No production code changes.
…internal test

The shim exported reapJob for one external test. The new internal test runs the locked recheck against a real database, so the package needs no export_test.go.
@ethanndickson

Copy link
Copy Markdown
Member Author

@codex review

New since the last review: a second commit that replaces the ReapPendingJob test shim in coderd/jobreaper with an internal test. The first commit is the same as before, only rebased.

Two earlier decisions still stand:

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Delightful!

Reviewed commit: cd4cb10263

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@ethanndickson
ethanndickson merged commit 166b9d2 into main Nov 18, 2026
28 checks passed
@ethanndickson
ethanndickson deleted the ethan/prune-cron-chatwebhook-tests branch November 18, 2026 03:14
@github-actions github-actions Bot locked and limited conversation to collaborators Nov 18, 2026

This branch had an error being deployed

1 failed and 1 inactive deployments
docs-preview — cd4cb102 Deployed Nov 18, 2026 by ethanndickson via cleanup #196
main — cd4cb102 Deployed Nov 18, 2026 by ethanndickson via cla #259108
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants