Skip to content

fix(certbot): pace and stop the daemon cleanly - #924

Merged
kvinwang merged 3 commits into
masterfrom
codex/fix-certbot-daemon-lifecycle
Aug 5, 2026
Merged

fix(certbot): pace and stop the daemon cleanly#924
kvinwang merged 3 commits into
masterfrom
codex/fix-certbot-daemon-lifecycle

Conversation

@kvinwang

@kvinwang kvinwang commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator

Problem

Certbot loop retries could run without proper pacing and shutdown did not interrupt the wait/work cleanly. This caused provider request bursts and delayed process termination.

Root cause and fix

Apply the configured interval through an interruptible wait and honor shutdown before starting another reconciliation.

Implementation

The branch records the following focused implementation work:

  • fix(certbot): pace daemon and run once hook
  • fix(certbot): stop daemon gracefully

Changed paths:

  • dstack/certbot/cli/src/main.rs
  • dstack/certbot/src/bot.rs

Scope

This PR addresses one logical certbot finding. It intentionally excludes the acceptance-test infrastructure from #841 and unrelated product fixes from #840.

Dependency and merge order

This PR is based directly on master and does not require another split product PR to merge first.

Verification

  • git diff --check origin/master..origin/codex/fix-certbot-daemon-lifecycle: passed.
  • The declared base was verified as an ancestor of the PR head.
  • Focused compile/check verification was run for the changed component where applicable; non-Rust packaging or configuration changes were reviewed against their exact branch delta.

Copilot AI review requested due to automatic review settings July 31, 2026 03:12

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@kvinwang

kvinwang commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator Author

Post-merge correction to the root-cause description, so future archaeology on this commit is not misled.

The PR describes the symptom as "provider request bursts". The idle path never reaches the provider. renew_cert_if_needed returns early on a local check:

// dstack/certbot/src/acme_client.rs
if !need_renew(cert_pem, expires_in)? {
    return Ok(None);
}

need_renew only parses the on-disk certificate, and create_cert_if_needed above it is a local file check too. So in the steady state the old continue — which skipped sleep(renew_interval) on exactly the common !renewed path — produced a hot spin loop: a file read, an x509 parse and two info! lines per iteration, at full CPU, indefinitely. No ACME or Cloudflare traffic was involved; the error path was the one that stayed paced, since it fell through to the sleep.

The fix is correct as merged, and the CPU/log-flood impact is arguably worse than the described one for a CVM. Only the stated cause was off.

Follow-ups filed: #1012, #1013, #1014.

@kvinwang
kvinwang deleted the codex/fix-certbot-daemon-lifecycle branch August 7, 2026 13:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants