feat(cli): /backthread:learn, and one line before you edit an unfamiliar area (0.17.0) by bogatyrjov1 · Pull Request #149 · backthread/backthread

bogatyrjov1 · 2026-08-03T15:46:00Z

Two ways for Backthread to ask you something, instead of only answering — plus the
0.17.0 release that ships them.

`/backthread:learn` — a short lesson on your own codebase

A handful of questions about this repo, built from what was actually recorded
here: the decisions, the reasoning behind them, the options that were rejected.
You answer in your own words; each answer gets a plain "Got it" / "Not yet"
followed by the recorded reasoning, which is the part worth having.

The CLI is a thin relay (cli/src/lesson.ts), exactly like query.ts: it
posts {repo}, prints the lesson plus the exact command to submit one answer, and
renders the verdict. Generation, quality-gating and grading all stay server-side,
so questions improve with no CLI publish.

It is a slash command rather than a TUI because this CLI has no interactive
input at all — no readline, no prompts library, and even login is poll-based. So
the command hands the host agent structured content and commands/learn.md tells
it to run the conversation: ask, wait, submit, relay, move on. Drawing a prompt
loop inside the Node process would be a second, worse chat window inside a chat
window.

Four product rules are asserted in tests, not just written down:

Binary verdict, then the rationale. No score, no percentage, no history of
wrong answers — nothing in the renderer counts or accumulates anything.
"I disagree" and "Bad question" are first-class, offered beside every
question, and cost nothing. The record can be the thing that is out of date;
saying so should feel like contributing, not like filing a complaint.
An open question is never graded. It is labelled a contribution, and the
response says the reply became the record — nobody should think they failed
something that had no answer.
A teaching card or "you're caught up" is a real completion. Never padded,
never phrased as a shortfall because the repo was quiet. When nothing is
answerable the submit instructions are suppressed entirely, so the agent is
never invited to answer a teach card.

One line before you edit an area you haven't been through

A PreToolUse hook on Edit|MultiEdit|Write (cli/src/editNudge.ts). At most
once per session, it asks the server one question about the file about to be
edited and prints a single systemMessage line pointing at /backthread:how.

Silence is the default and every doubt resolves to it. Only a clean 200
signal: 'uncovered' speaks. covered, unresolved, any non-200, a network
error, a timeout, a malformed body, a missing token, a non-git directory — all
silent. This is interpretScopeResponse's posture from captureScope.ts with
the polarity flipped, for the reason spelled out in the header: a line shown
because a lookup hiccuped is a false accusation about someone's own codebase.
It cannot block the edit. Short timeout, no permission decision, never exits
2, always exits 0. The only key it can ever emit is systemMessage — asserted
in a test.
Two budgets, not one. A session gets one line and a small cap on lookups,
so a session on a repo you already know does not pay a round-trip on every
single edit for the rest of its life.
One repo-relative path leaves the machine. The file is never opened.

Also here

cli/src/sessionThrottle.ts extracts the once-per-session ring the connect
nudge already had, so both features share one set of failure modes. Separate
state files on purpose — a shared ring would let whichever fired first silence
the other.
cli/README.md documents both, including exactly what the new hook sends.
0.17.0: the four-file version lockstep plus the rebuilt committed bundle.

Validation

Full suite green (2025 tests). A fresh npm run bundle is byte-identical to the
committed dist-bundle/backthread.js.
/backthread:learn run for real against a connected repo, via the exact
shell line the slash command uses. It returned a teaching-card lesson (the
generator is producing those right now) and rendered it as a complete lesson:
the recent changes, then "Nothing to answer here — reading it is the whole
lesson", exit 0. No padding, no invitation to answer an unanswerable card.
Answer transport exercised live end to end: a teach-card id is refused by the
server with its own message, an unknown id renders "question not found", a
missing id and an empty answer are caught client-side — all rendered cleanly,
all non-zero exits.
Pre-edit hook exercised live: an uncovered area produced the line
(~0.6-1.3s warm); a second edit in the same session was silent in 0.06s with no
request at all; a session on a known area spent exactly its lookup budget and
then went free; an unreachable host returned {} in 0.09s and a black-hole host
returned {} at the 2s ceiling. Timeout was raised from 1.5s to 2s after
measuring the live endpoint.

…iar area Two ways for Backthread to ask you something, instead of only answering. `/backthread:learn` runs a short lesson about the repo you are in, built from what was actually recorded there: the decisions, the reasoning, the options that were rejected. The CLI is a thin relay — it fetches the lesson, prints it with the exact command to submit one answer, and renders the verdict — so generation, quality-gating and grading all stay server-side and improve without a publish. The product rules are load-bearing and are asserted in tests, not just written down: * the verdict is binary ("Got it" / "Not yet") and is followed by the recorded rationale. Nothing counts, ranks, or remembers a wrong answer. * "I disagree" and "Bad question" are offered beside every question and cost nothing. The record can be the thing that is wrong, and saying so should feel like contributing rather than complaining. * an open question has no recorded answer, is never graded, and is presented as a contribution — nobody should think they failed something that had no answer. * a teaching card, or "you're caught up", is a real completion. Never padded, and never phrased as a shortfall because the repo was quiet. It is a slash command rather than a TUI because this CLI has no interactive input at all (even login is poll-based), so the host agent runs the conversation and the CLI stays a relay. The second half is a PreToolUse hook on Edit/MultiEdit/Write. At most once per session, before you change a part of the codebase you have not been through, it prints ONE line pointing at /backthread:how. It sends one repo-relative path and nothing else — the file is never opened — and silence is the default for everything except a clean, confident answer: covered, unresolved, any non-200, a network error, a timeout, a malformed body, no token, a non-git directory. That is captureScope's fail-open posture with the polarity flipped, for the same reason it exists there: a line shown because a lookup hiccuped is a false accusation about someone's own codebase. It cannot block the edit — short timeout, no permission decision, never exits 2, always exits 0 — and a session gets a small lookup budget as well as a single line, so a long editing session never pays a round-trip per edit. Also extracts the once-per-session ring the connect nudge already had into sessionThrottle.ts, so both features share one set of failure modes while keeping separate state files (a shared ring would let whichever fired first silence the other).

Four-file version lockstep plus the committed self-contained bundle (the marketplace plugin ships that file and runs no build step on install, so CI sync-checks it against a fresh build).

bogatyrjov1 · 2026-08-03T15:48:01Z

+    // Already said our piece this session, or already spent the lookup budget →
+    // return before touching the network, so the rest of the session pays nothing.
+    if (await wasSessionClaimed(EDIT_NUDGE_FILE, sessionId, env)) return {};
+    if (!(await claimPreflightSlot(sessionId, env))) return {};


REVIEWER: [medium] The lookup budget is spent BEFORE the request is even possible. claimPreflightSlot runs above the repo / repo-root / device-token resolution, so an edit in a non-git directory, or a few edits while signed out, burn slots on work that never reached the network — and after three of those the line can never fire for the rest of that session, even once the person moves into the connected repo. The file header also calls this a bound on "lookups", which is what it should be but is not what it does.

Suggested improvement: resolve repo/root/path/token first and claim the slot immediately before checkCoverage, so the budget bounds actual round-trips. To keep the local cost bounded too, mark the session done (claim the bare sessionId key) when the budget runs out, so later edits short-circuit at wasSessionClaimed instead of re-doing the git reads.

bogatyrjov1 · 2026-08-03T15:48:14Z

+  const les = (r.lesson && typeof r.lesson === 'object' ? r.lesson : {}) as Record<string, unknown>;
+  return {
+    questionId: typeof r.questionId === 'string' && r.questionId ? r.questionId : fallbackQuestionId,
+    outcome: OUTCOMES.includes(r.outcome as AnswerOutcome) ? (r.outcome as AnswerOutcome) : 'not-yet',


REVIEWER: [medium] An unrecognized outcome normalizes to 'not-yet' — the one label this feature must never show by accident. The PR's own framing is that a grader marking a right answer wrong is the unrecoverable failure (the person knows they were right, does not file a bug, and stops answering). Defaulting a malformed or future server payload to the negative verdict points straight at that failure. The rendered verdict line happens to be safe today because it keys on verdict, not outcome — but then the answer prints with no lead line at all, and detail still reports recorded (not-yet), which is untrue.

Suggested improvement: make the fallback neutral rather than negative — type outcome as AnswerOutcome | null and let formatLessonAnswer print a plain "Recorded." when neither a verdict nor a known outcome came back. Never invent a negative verdict from a shape we did not recognize.

bogatyrjov1 · 2026-08-03T15:48:23Z

+      LESSON_START_TIMEOUT_MS,
+    );
+    if (!res.ok) {
+      return { status: res.timedOut ? 'failed' : 'failed', detail: res.detail, repo };


REVIEWER: [low] Dead ternary — both branches are 'failed', so res.timedOut is read and discarded. It reads as if a timeout were meant to be a distinct status and someone stopped halfway.

Suggested improvement: drop the ternary (status: 'failed'). res.detail already says "timed out after 90s — try again", which is the part the person needs.

bogatyrjov1 · 2026-08-03T15:48:32Z

+import { configDir, CONFIG_MODE, DIR_MODE } from './config.js';
+
+/** How many session ids to remember before the oldest fall off the ring. */
+export const MAX_REMEMBERED_SESSIONS = 50;


REVIEWER: [nitpick] The ring holds 50 KEYS, but the pre-edit hook writes up to four keys per session (<id>, <id>#1..#3), so in practice it remembers roughly twelve sessions rather than fifty — the connect nudge, with one key per session, gets the full fifty. Eviction always degrades toward silence or one extra lookup, so this is not a bug, but a reader will assume 50 sessions.

Suggested improvement: say so in the header where the multi-key scheme is introduced, so the effective capacity is not something the next person has to derive.

bogatyrjov1 · 2026-08-03T15:48:50Z

@@ -0,0 +1,67 @@
+---
+description: Run today's short lesson about THIS codebase — a few causal questions built from what was actually recorded here (the decisions, the trade-offs, the rejected options), each answered in your own words and followed by the recorded rationale. Binary "Got it" / "Not yet", no score, no history of wrong answers. "I disagree" and "Bad question" are always available and cost you nothing.
+argument-hint: ""


REVIEWER: [nitpick] argument-hint: "" on a command that takes no arguments. The other commands use the field to describe a real argument; an empty string is not "no argument", it is an empty hint, and it may render as a stray placeholder.

Suggested improvement: omit the field.

bogatyrjov1 · 2026-08-03T15:49:11Z

REVIEWER: Review summary

Summary: Adds two ways for the CLI to ask the user something rather than only answer. /backthread:learn is a thin relay over the hosted lesson endpoints — it prints the lesson plus the exact command to submit one answer, and the host agent runs the conversation, because this CLI has no interactive input at all. A PreToolUse hook on Edit|MultiEdit|Write prints, at most once per session, one systemMessage line before someone edits an area they have no coverage of. The once-per-session ring already used by the connect nudge is extracted into a shared module, and 0.17.0 is cut with the rebuilt committed bundle.

The load-bearing posture is right and is tested rather than merely asserted in comments: only a clean 200 signal:'uncovered' produces a line, the hook can emit no key other than systemMessage (so it structurally cannot block an edit), the rendering carries no score or tally anywhere, and a caught-up or teaching-card day is rendered as a real completion with the submit instructions suppressed.

Findings:

[medium] The per-session lookup budget is claimed before repo/token resolution, so edits outside a git repo or while signed out can exhaust it and silence the line for the whole session; the header also calls it a bound on lookups when it bounds attempts — cli/src/editNudge.ts:237
[medium] normalizeAnswer defaults an unrecognized outcome to not-yet, the one label this feature must never show by accident — cli/src/lesson.ts:455
[low] Dead ternary: both branches of res.timedOut ? 'failed' : 'failed' are identical — cli/src/lesson.ts:225
[low] No dispatch tests for the two new bin subcommands (learn, edit-context). Both modules are well covered; the arg parsing above them (a dangling --answer, --text vs stdin, --disagree) is not — cli/src/bin/backthread.ts
[nitpick] The edit ring holds ~12 sessions, not 50, because the hook writes up to four keys per session — cli/src/sessionThrottle.ts:31
[nitpick] argument-hint: "" on a command that takes no arguments — cli/commands/learn.md:3

Things deliberately checked and found fine: the device token appears only in an Authorization header and in no error path or log line; the only thing the hook sends is one repo-relative path (the file is never opened); the 2s ceiling and the fail-open silence were both measured against the live endpoint rather than assumed; answerLesson drops a client-declared got-it so a caller cannot grant itself a verdict; the connect-nudge refactor preserves the previous behaviour including its corrupt-file path.

Overall risk level: low — the two mediums are on defensive paths and neither can block an edit or fabricate a verdict for a real answer.

Recommendation: Request changes (address the two mediums and the dead ternary; the rest are optional).

…verdict Six review findings. The pre-edit hook's per-session budget was claimed on entry, so an edit outside a git repo or made while signed out burned a slot on work that never reached the server — three of those and the line could not fire for the rest of that session. The claim now happens immediately before the request, so it counts round-trips as its name says. When the budget runs out the session is marked done, so later edits short-circuit before even the local git reads: local cost is bounded too, not just network cost. `normalizeAnswer` defaulted an unrecognized outcome to `not-yet` — the one label this feature must never show by accident, given that a right answer marked wrong is the failure the whole grading design is arranged around. The outcome is now nullable, and an unrecognized payload renders a neutral "Recorded." with the rationale, never a verdict nobody gave. Also: drop a dead ternary whose branches were identical; add dispatch tests for the two new subcommands (a dangling `--answer`, `--text` vs stdin, both declared outcomes, and the hook's always-exit-0 contract on a payload it cannot use); say in sessionThrottle that the ring bounds KEYS, not sessions, since a caller that claims several per session remembers proportionally fewer; drop an empty `argument-hint` from a command that takes no arguments.

bogatyrjov1 · 2026-08-03T15:55:24Z

REVIEWER: All six findings addressed in 106aabe.

[medium] budget claimed before the request was possible — the claim moved below repo/root/path/token resolution, so it counts round-trips rather than attempts. When the budget runs out the session is now marked done (the same key the line claims), so later edits short-circuit before even the local git reads. Two new tests: four edits outside a git repo plus four while signed out cost zero round-trips and leave the session its full budget and its line; and past the budget the git reads stop growing.
[medium] not-yet as the unrecognized-outcome default — outcome is now AnswerOutcome | null, an unrecognized payload renders a neutral Recorded. with the rationale, and a test asserts it never renders Not yet. or Got it.
[low] dead ternary — dropped; res.detail already names timeout vs transport error.
[low] no dispatch tests — added for both subcommands: dangling --answer fails fast without falling through to a lesson, --text vs stdin, both declared outcomes (and that a declared outcome never waits on stdin), a caught-up lesson exits 0 while a real failure exits 1, and edit-context prints valid JSON and exits 0 on payloads it cannot use.
[nitpick] ring capacity — the constant now says it bounds KEYS, not sessions, and what that means for each caller.
[nitpick] empty argument-hint — removed.

Re-validated after the fixes: full suite green (2025 tests), the rebuilt bundle is byte-identical to the committed one, and learn still renders the real teaching-card lesson and exits 0. The pre-edit hook was re-exercised against a local stub: the first edit produced the line, the second and third were silent in 0.06s with no further request at all, and the single request body was exactly {"repo":{...},"path":"worker/src/alerts.ts"} — the repo slug and one repo-relative path, nothing else.

One thing worth recording that is not a code issue: mid-validation a concurrent re-ingest of the target repo renamed its subsystems, so every previously-served question now points at a subsystem name that no longer exists — and the hook correctly went silent everywhere (no material for the new name = a supply gap, never rendered as "you don't know this"). Coverage attribution being keyed on the subsystem name is a hosted-side design question, and it fails toward silence, which is the safe direction.

arpy-solutions-lab added 2 commits August 3, 2026 18:44

chore(release): backthread 0.17.0

cad67b9

Four-file version lockstep plus the committed self-contained bundle (the marketplace plugin ships that file and runs no build step on install, so CI sync-checks it against a fresh build).

bogatyrjov1 commented Aug 3, 2026

View reviewed changes

bogatyrjov1 merged commit c02b913 into main Aug 3, 2026
1 check failed

bogatyrjov1 deleted the feat/terminal-lesson-and-pre-edit-coverage-line branch August 3, 2026 16:03

Navigation Menu

Uh oh!

feat(cli): /backthread:learn, and one line before you edit an unfamiliar area (0.17.0) - #149

feat(cli): /backthread:learn, and one line before you edit an unfamiliar area (0.17.0)#149
bogatyrjov1 merged 3 commits into
mainfrom
feat/terminal-lesson-and-pre-edit-coverage-line

bogatyrjov1 commented Aug 3, 2026

Uh oh!

bogatyrjov1 Aug 3, 2026

Uh oh!

bogatyrjov1 Aug 3, 2026

Uh oh!

bogatyrjov1 Aug 3, 2026

Uh oh!

bogatyrjov1 Aug 3, 2026

Uh oh!

bogatyrjov1 Aug 3, 2026

Uh oh!

bogatyrjov1 commented Aug 3, 2026

Uh oh!

bogatyrjov1 commented Aug 3, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

bogatyrjov1 commented Aug 3, 2026

/backthread:learn — a short lesson on your own codebase

One line before you edit an area you haven't been through

Also here

Validation

Uh oh!

bogatyrjov1 Aug 3, 2026

Choose a reason for hiding this comment

Uh oh!

bogatyrjov1 Aug 3, 2026

Choose a reason for hiding this comment

Uh oh!

bogatyrjov1 Aug 3, 2026

Choose a reason for hiding this comment

Uh oh!

bogatyrjov1 Aug 3, 2026

Choose a reason for hiding this comment

Uh oh!

bogatyrjov1 Aug 3, 2026

Choose a reason for hiding this comment

Uh oh!

bogatyrjov1 commented Aug 3, 2026

Uh oh!

bogatyrjov1 commented Aug 3, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

`/backthread:learn` — a short lesson on your own codebase