Skip to content

Commit 1f82e53

Browse files
chore(dev): gitignore local Gmail gate config
1 parent 8fe184e commit 1f82e53

7 files changed

Lines changed: 81 additions & 14 deletions

File tree

‎.gitignore‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,3 +11,6 @@
1111
blobs/
1212
dev/gate/m4gate-result.txt
1313
dev/gate/*.log
14+
# Local live-gate config carries a real account address + OAuth client id
15+
# (committed file is dev/config-gmail.toml.example). Never commit this.
16+
dev/config-gmail.toml

‎AGENTS.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -139,6 +139,8 @@ Gmail **API mode** (M10) has its own fixture rig with no Google account:
139139
bridge on `http://127.0.0.1:8081/gapi` backed by `dev/gate/gmailfixture`; point
140140
jmap-tui's `TestLiveSessionAndMailboxes` at it the same way. The live API gate
141141
is `python3 dev/gate/gmailapi-live-gate.py` (read-only; see `dev/gate/README.md`).
142+
`dev/config-gmail.toml` (a real address + OAuth client id) is **gitignored**:
143+
copy `dev/config-gmail.toml.example`, never commit the real one.
142144

143145
## Live provider rules of engagement
144146

‎PLAN.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -582,7 +582,7 @@ milestones but deliberately carries no completion state). Rules:
582582
| **M1** | SQLite store (schema §4), read-only IMAP sync (discovery, tier detection, backfill, IDLE), hydration, `/changes` + SSE, preview | real Dovecot account browsable read-only; a flag flipped in another IMAP client appears in jmap-tui ≤ 2 s | FR-S.1–.9, FR-M.1–.8, FR-J.7–.8 | ✅ done 2026-09-29 (live local Dovecot 2.4: `test/live` foreign flag → store 505 ms and jmap-tui engine 563 ms, both ≤ 2 s over IDLE+SSE; backfill, hydration and preview against the real server; jmap-tui live suite + `smoke` green over loopback) |
583583
| **M2** | Write path §7.1, `Mailbox/set`, drafts, IMAP-first commits | jmap-tui triage (star/archive/move/delete/undo) round-trips; changes visible from a second IMAP client | FR-M.9–.13 | ✅ done 2026-09-29 (live local Dovecot 2.4: `test/live` write gate — star/move/copy with undo, `Mailbox/set` create/rename/delete, draft `APPEND` with `\Draft`, destroy — each re-read from an **independent IMAP session**; jmap-tui triage scratch green over loopback 3× — read/star/undo/move/undo/copy/undo/archive/delete-to-trash/destroy, every step confirmed by python3/imaplib; gate caught and fixed `/get` dropping `id` per RFC 8620 §5.1) |
584584
| **M3** | Send §7.2, `Identity/get`, blob upload/download, `EmailSubmission/set` with `onSuccessUpdateEmail` | compose → send → message in Sent **and** delivered to a test sink; attachment round-trip byte-exact | FR-M.14–.17 | ✅ done 2026-09-29 (`test/live` M3 gate against dev Dovecot + in-process SMTP sink: batched draft+submission, `#draft` creation reference, patch files Sent, sink held the message with its attachment byte-exact and no Bcc, second IMAP session saw one `\Seen` copy — plus fixture-tier suites `TestSubmission*`, `internal/submit` against the SMTP fixture, and the jmap-tui cross-client rig: `TestLiveM5Gate` (compose→attach→send→Sent), `TestLiveM5ProbeDraftRoundTrip` and the M1/M2 bridge gates green over loopback) |
585-
| **M4** | Gmail profile: OAuth2 bootstrap, XOAUTH2 IMAP/SMTP, `X-GM-LABELS`↔mailboxes, All Mail/archive, `X-GM-THRID`, CONDSTORE tier validation, rate limits | live Gmail: folders+labels both ways, compose/send, archive from jmap-tui, no rate-limit warnings | FR-A.5–.10, FR-S.10, FR-S.12, FR-M.18 | ✅ done 2026-10-01 (live Gmail `you@gmail.com`, `dev/gate/m4gate.py` once, **14/14 checks, no `[THROTTLED]`**, every claim re-read from an independent `imaplib` client: a label applied in Gmail appears in the bridge; archiving from the bridge drops INBOX membership and the undo restores it; compose/send over XOAUTH2 SMTP delivers exactly one copy and files exactly one Sent copy. The gate caught and fixed **five real Gmail bugs**: a label write sourced from the folder being removed is accepted and silently ignored (archive now sources from All Mail); a fresh message with no cached All Mail copy is addressed there by a Message-ID search (`imapdrv.FindUID`); filing a draft into Sent tombstoned it (non-implicit label adds now commit locally); Gmail's draft label is `\Draft`, not `\Drafts`; and Gmail delays an SMTP submission whose Message-ID matches the upstream draft, so the submission path expunges that draft before SMTP and restores it on failure. Fixture coverage: per-folder UIDs in `internal/sync/gmail_test.go`, `TestGmailArchiveSearchesAllMailWhenUncached`, `TestGmailSendExpungesDraftBeforeSMTP`) |
585+
| **M4** | Gmail profile: OAuth2 bootstrap, XOAUTH2 IMAP/SMTP, `X-GM-LABELS`↔mailboxes, All Mail/archive, `X-GM-THRID`, CONDSTORE tier validation, rate limits | live Gmail: folders+labels both ways, compose/send, archive from jmap-tui, no rate-limit warnings | FR-A.5–.10, FR-S.10, FR-S.12, FR-M.18 | ✅ done 2026-10-01 (live Gmail the user's Gmail account, `dev/gate/m4gate.py` once, **14/14 checks, no `[THROTTLED]`**, every claim re-read from an independent `imaplib` client: a label applied in Gmail appears in the bridge; archiving from the bridge drops INBOX membership and the undo restores it; compose/send over XOAUTH2 SMTP delivers exactly one copy and files exactly one Sent copy. The gate caught and fixed **five real Gmail bugs**: a label write sourced from the folder being removed is accepted and silently ignored (archive now sources from All Mail); a fresh message with no cached All Mail copy is addressed there by a Message-ID search (`imapdrv.FindUID`); filing a draft into Sent tombstoned it (non-implicit label adds now commit locally); Gmail's draft label is `\Draft`, not `\Drafts`; and Gmail delays an SMTP submission whose Message-ID matches the upstream draft, so the submission path expunges that draft before SMTP and restores it on failure. Fixture coverage: per-folder UIDs in `internal/sync/gmail_test.go`, `TestGmailArchiveSearchesAllMailWhenUncached`, `TestGmailSendExpungesDraftBeforeSMTP`) |
586586
| **M5** | FTS5 + search-driven backfill, filter/sort/anchor/collapseThreads correctness, `PREVIEW`/partial-fetch, COMPRESS, 100k soak | jmap-tui live search cases green; cold browse of a 100k mailbox stays responsive; soak within NFR bounds | FR-X.1–.8, FR-S.11, NFR-1, NFR-2, NFR-8 | ✅ done 2026-09-30 (dev Dovecot 100k-message corpus, `test/live TestLiveSoak100k`: warm `Email/query` p95 browse 31.1 ms / collapseThreads 32.0 ms / body-token text 82.8 ms, all < NFR-1's 150 ms; hydration 133.4 bodies/s (≥ 15); idle RSS 29.0 MB (< 150 MB, NFR-8); 3 engine-reconnect cycles flat at the 3-goroutine baseline; cold reopen browse 37.7 ms / collapse 32.6 ms — and the same full workload under the race detector with **zero data races** (the soak relaxes its NFR budgets only under `-race`, which it also enforces verbatim otherwise); jmap-tui cross-client gate `TestLiveSearchVerification` green over loopback: 3000-message text search first page 505 ms, from/subject/hasKeyword/after/before/all-mailbox filters, engine `SearchOpen`/`SearchClose`, and both substring fallbacks (partial-word + fielded-subject scan matched 3000/3000); FTS5 index at ingest + hydration, tombstones, search-driven backfill and batched hydration covered by `internal/store/fts_test.go`, `internal/sync/backfill_search_test.go` and `internal/imapdrv/fetchbodies_test.go`) |
587587
| **M6** | CardDAV §8: discovery, sync-collection, PUT/DELETE, vCard↔JSContact, photo blobs, capability gating | jmap-tui `contacts_live_test` suite green against a real CardDAV server; create/edit/delete contact round-trips | FR-P.1–.13 | ✅ done 2026-09-30 (loopback Radicale 3.8.1 as the real server — `dev/gate/radicale-start.sh`: jmap-tui `TestLiveContactsGate` green over loopback (capability + lazy load, create → SSE ContactCard push < 8 s → rename → destroy, card proven gone from the server); bridge-side `test/live TestLiveContactsGate` green against the same Rig (create/update/move/destroy each re-read from an **independent DAV session**, RFC 6578 sync-token replay + 412/refetch write guard); the gate caught and fixed a real bug — an update re-minted the card UID (Radicale answered 409 no-uid-conflict), now the uid is canonicalized into the stored JSContact on create and reused on update. Fixture-tier coverage: `internal/dav` (sync/getctag/bare tiers, well-known redirect, conditional PUT/DELETE, multiget/GET), `test/fixturecarddav`, `internal/convert` golden pairs (v3+v4, groups, inline photos, malformed), `internal/store/contacts_test.go` (bookkeeping never moves states, cascade tombstones), `internal/jmapapi` set/patch/error-mapping tests, `internal/httpapi` FR-P.3 session gating test)
588588
| **M7** | Packaging: multi-account paths, credential encryption, metrics/health (`[metrics]` + `/metrics`), UIDVALIDITY recovery drill, Docker image + k8s manifests, README deployment verified, `JMAP-TestSuite` run | fresh `docker run` + `kubectl` install works end-to-end from the docs; JMAP-TestSuite subset for the implemented surface green (deliberate gaps enumerated below); all gates green | FR-M.19, FR-J.9–.10, FR-D.2–.12, NFR-3–.7, NFR-9–.10 | ✅ done 2026-10-03 (live k3s `v0.1.5` at https://jmap-bridge.geekify.me — cPanel IMAP + Gmail OAuth2 accounts, jmap-tui connected, `/healthz`=ok `/readyz`=ready; `JMAP-TestSuite` vs loopback dev Dovecot: **208/262 required pass**, 28 skip, 54 required fail — **all** in the documented out-of-scope gaps below, zero in-scope failures; `[metrics]` `/metrics` demonstrated `200` enabled vs `404` disabled; FR-D.12 redaction tests; FR-S.6 UIDVALIDITY drill; all local gates green: build, vet, gofumpt, golangci-lint 0 issues, `go test -race`, version smoke) |

‎dev/config-gmail.toml.example‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
# Gmail live-gate config (M4/M10) — EXAMPLE ONLY, copy to
2+
# dev/config-gmail.toml (gitignored) and fill in your own values.
3+
#
4+
# The OAuth client secret is NEVER placed here: it comes from the
5+
# environment (JMAP_BRIDGE_GMAIL_OAUTH2_CLIENT_SECRET, e.g. in .env), as
6+
# does the token-encryption key JMAP_BRIDGE_SECRET_KEY.
7+
#
8+
# Registered redirect URI on the OAuth client:
9+
# http://127.0.0.1:8080/oauth/gmail/callback
10+
11+
listen = "127.0.0.1:8080"
12+
base_url = "http://127.0.0.1:8080"
13+
data_dir = "./data"
14+
log_level = "info"
15+
16+
[sync]
17+
interval = "60s"
18+
19+
[auth]
20+
mode = "token"
21+
22+
[[accounts]]
23+
id = "gmail"
24+
name = "Gmail"
25+
address = "you@gmail.com"
26+
token = "dev-token-gmail-local-only-0123456789"
27+
28+
[accounts.imap]
29+
host = "imap.gmail.com"
30+
port = 993
31+
auth = "oauth2"
32+
username = "you@gmail.com"
33+
34+
[accounts.smtp]
35+
host = "smtp.gmail.com"
36+
port = 465
37+
auth = "oauth2"
38+
39+
[accounts.oauth2]
40+
provider = "google"
41+
client_id = "YOUR_CLIENT_ID.apps.googleusercontent.com"

‎dev/gate/README.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,9 @@ script (JMAP over the bridge + an independent `imaplib` XOAUTH2 client).
77
Secrets live in `.env` (never committed): the OAuth2 client secret and
88
`JMAP_BRIDGE_SECRET_KEY`, which unseals the stored refresh token in
99
`data/bridge.db` — losing that key means re-consenting in a browser.
10+
`dev/config-gmail.toml` holds a real account address and OAuth client id and
11+
is **gitignored**: copy `dev/config-gmail.toml.example` and fill in your own
12+
values before running the Gmail gates.
1013

1114
## What the gate covers
1215

‎dev/gate/gmailapi-live-gate.py‎

Lines changed: 17 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -45,12 +45,19 @@ def load_env():
4545
os.write(fd, key.encode())
4646
os.close(fd)
4747
client_id = None
48-
for line in open(os.path.join(REPO, "dev/config-gmail.toml")):
49-
if "client_id" in line:
50-
client_id = line.split("=", 1)[1].strip().strip('"')
51-
break
48+
address = None
49+
cfg = os.path.join(REPO, "dev/config-gmail.toml")
50+
if os.path.exists(cfg):
51+
for line in open(cfg):
52+
s = line.strip()
53+
if s.startswith("client_id"):
54+
client_id = s.split("=", 1)[1].strip().strip('"')
55+
elif s.startswith("address"):
56+
address = s.split("=", 1)[1].strip().strip('"')
5257
if client_id:
5358
os.environ.setdefault("GMAIL_TOKEN_CLIENT_ID", client_id)
59+
if address:
60+
os.environ.setdefault("GMAIL_ADDRESS", address)
5461

5562

5663
def build():
@@ -113,6 +120,9 @@ def jmap(calls, token=TOKEN, base=BRIDGE):
113120

114121
def write_bridge_config():
115122
os.makedirs(DATA, exist_ok=True)
123+
address = os.environ.get("GMAIL_ADDRESS", "")
124+
if not address:
125+
sys.exit("GMAIL_ADDRESS is not set (add address = ... to dev/config-gmail.toml)")
116126
cfg = f'''listen = "127.0.0.1:8080"
117127
base_url = "http://127.0.0.1:8080"
118128
data_dir = "{DATA}"
@@ -127,7 +137,7 @@ def write_bridge_config():
127137
[[accounts]]
128138
id = "gmail"
129139
name = "Gmail API"
130-
address = "you@gmail.com"
140+
address = "{address}"
131141
token = "{TOKEN}"
132142
backend = "gmail_api"
133143
@@ -231,7 +241,8 @@ def main():
231241
print(f"PASS test label {LABEL}:{label_id}")
232242

233243
# Insert one test message into the label (never touches existing mail).
234-
raw = (f"From: you@gmail.com\r\nTo: you@gmail.com\r\n"
244+
address = os.environ["GMAIL_ADDRESS"]
245+
raw = (f"From: {address}\r\nTo: {address}\r\n"
235246
f"Subject: {SUBJECT}\r\nMessage-ID: <m10-gate-{int(time.time())}@jmap-bridge.test>\r\n"
236247
f"Date: Mon, 02 Jan 2026 15:04:05 -0700\r\nMIME-Version: 1.0\r\n"
237248
f"Content-Type: text/plain; charset=utf-8\r\n\r\nM10 API gate body\r\n").encode()

‎dev/gate/m4gate.py‎

Lines changed: 14 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -48,20 +48,27 @@
4848
os.write(fd, line.split("=", 1)[1].strip().encode())
4949
os.close(fd)
5050
break
51-
for line in open("/home/adam/projects/jmap-bridge/dev/config-gmail.toml"):
52-
if "client_id" in line:
53-
os.environ.setdefault("GMAIL_TOKEN_CLIENT_ID",
54-
line.split("=", 1)[1].strip().strip('"'))
55-
break
51+
CFG = "/home/adam/projects/jmap-bridge/dev/config-gmail.toml"
52+
if not os.path.exists(CFG):
53+
sys.exit("missing dev/config-gmail.toml (copy dev/config-gmail.toml.example and fill in your account)")
54+
for line in open(CFG):
55+
s = line.strip()
56+
if s.startswith("client_id"):
57+
os.environ.setdefault("GMAIL_TOKEN_CLIENT_ID", s.split("=", 1)[1].strip().strip('"'))
58+
elif s.startswith("address"):
59+
os.environ.setdefault("GMAIL_ADDRESS", s.split("=", 1)[1].strip().strip('"'))
5660

5761
BRIDGE = "http://127.0.0.1:8080/gmail"
5862
AUTH = base64.b64encode(b"any:dev-token-gmail-local-only").decode()
59-
USER = "you@gmail.com"
63+
USER = os.environ.get("GMAIL_ADDRESS", "")
64+
if not USER or "@" not in USER:
65+
sys.exit("GMAIL_ADDRESS is not set (add address = ... to dev/config-gmail.toml)")
6066
# The send test goes to a plus-alias of the same account: Gmail dedupes a
6167
# self-send (From == To) against the draft that already carries the same
6268
# Message-ID and does not deliver a separate INBOX copy, but a distinct
6369
# recipient on the same mailbox does deliver one.
64-
RCPT = "you+bridge@gmail.com"
70+
_local, _domain = USER.split("@", 1)
71+
RCPT = os.environ.get("GMAIL_RCPT", f"{_local}+jmapbridgegate@{_domain}")
6572
NONCE = str(int(time.time()))
6673
LABEL = "jmapgate" + NONCE # applied in Gmail, expected in the bridge
6774
SUBJECT = "jmap-bridge M4 gate " + NONCE

0 commit comments

Comments
 (0)