Skip to content

feat: let organization members turn off organization skills in chats - #31134

Merged
ibetitsmike merged 1 commit into
mainfrom
mike/org-skills-opt-out-backend
Oct 29, 2026
Merged

ibetitsmike merged 1 commit into
mainfrom
mike/org-skills-opt-out-backend

Conversation

@ibetitsmike

@ibetitsmike ibetitsmike commented Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator

Stack

Organization skills load into every member's Coder Agents chats in their organization, with no way for a member to opt out. This stack adds a per-organization setting that lets members turn them off for their own chats, bottom to top:

  1. feat: let organization members turn off organization skills in chats #31134 data model, API, and chatd (this PR)
  2. feat: add organization skill opt-out controls to the dashboard #31135 admin and member controls in the dashboard, the / menu, and docs

What changes

Organizations get members_can_disable_skills, off by default. Users who can update organization skills change it with PUT /api/experimental/organizations/{organization}/skill-settings, which writes an organization audit entry. Anyone who can read the organization can GET it.

While the setting is on, each member saves their own choices with PATCH /api/experimental/organizations/{organization}/members/{user}/skill-preferences: all_skills_disabled (a master switch that also covers skills created later) and skills, a sparse map from skill ID to on or off. GET returns the setting, the master switch, and disabled_skill_ids. The routes follow the personal-settings rules: without read_personal or update_personal on that user the caller gets 404, a PATCH while the setting is off gets 403, and a key that is not an organization skill that member can read gets a 400 that names the key. The check runs as the member, not the caller, so an owner or user admin editing someone else's choices is held to that member's skills. Turning the setting off keeps the choices but ignores them.

Migration 000636 adds the column and two tables keyed by organization membership. Both cascade when the membership goes away, and the per-skill table also cascades when the skill is deleted, so a recreated skill loads by default.

chatd reads the preferences as the chat owner when it builds the skill index and when it loads a body for read_skill or for a structured skill invocation (#30861). A skill the member turned off is left out of the index, and both loads report it as not found, including later in a running chat. If the preferences cannot be read, turn preparation fails, as it already does when the skill metadata cannot be read.

Older migration tests created organizations with dbgen.Organization and GetDefaultOrganization, which now select the new column before it exists, so they insert organizations with raw SQL instead.

Review focus

  • Authorization in coderd/organizationskillpreferences.go and the five new dbauthz methods (read_personal and update_personal on the user, organization_skill update for the setting).
  • GetOrganizationSkillMemberPreferencesRow.Excludes is the one rule chatd applies; feat: add organization skill opt-out controls to the dashboard #31135 mirrors it in activeOrganizationSkills.

Testing

  • Go tests cover the queries and cascades, dbauthz, both routes (including the audit diff and each 400 case), and chatd (index, read_skill, skill invocations, master switch, a change during a chat). Each new test fails with its code path removed.
  • Remote dogfood UAT (10 scenarios, real model) passed on the pre-split branch head 186c28d81ab, which has the same code except the fix below. It found one issue: two spellings of one skill ID with opposite values stored a random result. The PATCH now rejects every aliased key with a 400. A second run on the stack's top head at the time, 718dd6d1bd1, passed: every aliased form returns 400 and stores nothing, and chats leave out a skill the member turned off, including in read_skill.
  • Polish and the test audit stayed inside the implementation diff. No new dependencies.
  • Rebased on main as one commit. The migration is now 000636, after main's 000634 and 000635. Every organization skill load passes the chat owner, including main's skill invocation path, so /skill invocations honor the opt-out too. The model's skill index leaves out a skill that is not model-invocable or that the member turned off. Apart from that, the diff is unchanged.

Xum, an AI agent, opened this PR for @ibetitsmike and wrote this description.

@ibetitsmike
ibetitsmike added this pull request to stack #31136 October 21, 2026 18:47
@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Docs preview

Check off each page once it's been reviewed. If a page changes in a later push, its checkbox clears automatically so it gets a fresh look. Pages not yet wired into the docs navigation aren't listed here.

@ibetitsmike ibetitsmike changed the title mike/org skills opt out backend feat: let organization members turn off organization skills in chats Oct 21, 2026
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

Xum, an AI agent, requested this review for @ibetitsmike.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-29T06:14:34.859285Z ef4cc2c Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Hooray!

Reviewed commit: 9819211235

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@ibetitsmike
ibetitsmike marked this pull request as ready for review October 21, 2026 23:34
@pr-queue-manager
pr-queue-manager Bot requested a review from jaaydenh October 21, 2026 23:35
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

Xum, an AI agent, requested this review for @ibetitsmike.

2 similar comments
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

Xum, an AI agent, requested this review for @ibetitsmike.

@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

Xum, an AI agent, requested this review for @ibetitsmike.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Swish!

Reviewed commit: 433c3e1c44

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

Xum, an AI agent, requested this review for @ibetitsmike.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 🚀

Reviewed commit: e5b7b8e729

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@coder-prototypes

Copy link
Copy Markdown
Contributor

UAT finding: skill IDs in the preferences PATCH are checked against the caller's skill access, not the member's

parseOrganizationSkillPreferenceIDs (coderd/organizationskillpreferences.go#L236) lists the organization skills with r.Context(), so dbauthz filters them by what the caller can read. When the caller differs from {user}, which the update_personal check at L137 allows for owners and user admins, the result can be wrong in both directions:

  1. Too permissive. As the owner, PATCH .../members/<ava>/skill-preferences with {"skills":{"<payments-pci>":false}} returns 200 and stores the row, even though payments-pci is shared only with another user and Ava cannot read it. The row has no effect in chats today, but Ava's disabled_skill_ids now lists a skill she can't see. If the skill is shared with her later, it starts turned off without her choosing that.
  2. Too strict. As a user-admin who is not a member of the organization, PATCH .../members/<noah>/skill-preferences with a skill Noah can read returns 400 No organization skill with this ID exists in this organization. The same caller can GET Noah's choices and can change all_skills_disabled, so only the per-skill part fails.

Impact. Low. Nothing leaks and chats stay correct. The dashboard always uses me, so only direct API or SDK calls on another user's behalf can trigger this.

Possible fixes.

  • Validate against the target member's access, for example by listing the skills as that member, the same way chatd reads preferences as the chat owner.
  • Validate only that the ID belongs to an organization skill of this organization, regardless of access.
  • Limit the PATCH to me if editing other users' choices isn't intended.

Found during remote UAT on PR head f8a56f34e9 (the #31135 stack top), local dev deployment with seeded users.

Coder Agents generated this comment on behalf of @jaaydenh.

@ibetitsmike
ibetitsmike force-pushed the mike/org-skills-opt-out-backend branch from e5b7b8e to ef4cc2c Compare October 29, 2026 06:10
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

Xum acted for Mike.

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Another round soon, please!

Reviewed commit: ef4cc2caf2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

Fixed in a100b41. parseOrganizationSkillPreferenceIDs now loads the target member's RBAC subject and lists the organization skills as that member, the same way chatd reads them as the chat owner. Both cases from this report are covered in TestOrganizationSkillPreferences: an owner can no longer turn off a skill the member can't read (400), and a user admin outside the organization can turn off a skill the member can read (200).

Xum posted this on behalf of Mike.

Add a per-organization setting, members_can_disable_skills (off by
default, audited, changeable by users who can update organization
skills). While it is on, each member can turn individual organization
skills off for their own chats, or turn all of them off. chatd and
read_skill skip the skills a member turned off.

- Migration 000635 adds the organization column and two per-member
  tables that cascade on membership and skill deletion.
- GET/PUT /api/experimental/organizations/{org}/skill-settings and
  GET/PATCH /api/experimental/organizations/{org}/members/{user}/skill-preferences.
- chatd reads the preferences as the chat owner when it builds the skill
  index and loads skill bodies.
@ibetitsmike
ibetitsmike force-pushed the mike/org-skills-opt-out-backend branch from 76249e0 to 6bb1eaa Compare October 29, 2026 12:36
@ibetitsmike
ibetitsmike merged commit d1f308d into main Oct 29, 2026
29 checks passed
@ibetitsmike
ibetitsmike deleted the mike/org-skills-opt-out-backend branch October 29, 2026 12:53
@github-actions github-actions Bot locked and limited conversation to collaborators Oct 29, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants