Skip to content

feat(computer-use): enable computer use by default, governed by a permission-group restriction #9944

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
waleedlatif1 merged 4 commits into from
Nov 12, 2026
+247 −368
Merged

feat(computer-use): enable computer use by default, governed by a permission-group restriction #9944

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
4dbdcb9
feat(computer-use): enable computer use by default, governed by a per…
waleedlatif1 Nov 12, 2026
9d1f891
fix(computer-use): hide device settings until the permission group is…
waleedlatif1 Nov 12, 2026
dc5a9df
fix(computer-use): read the permission group with the lightweight que…
waleedlatif1 Nov 12, 2026
aca7646
fix(computer-use): mount the device controls only once the permission…
waleedlatif1 Nov 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 4 additions & 5 deletions apps/desktop/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -234,11 +234,10 @@ Cadence: Electron ships a major every ~8 weeks and supports the latest 3 — bud

## Native Mac computer use

Mothership's `computer` tool is available on macOS 14+ when the global runtime flag
`mothership-computer-use` is enabled and the user opts in under **Settings → Desktop → Computer Use**.
The flag defaults off; local/self-hosted deployments use `MSHIP_COMPUTER_USE=true`, while hosted
rollout uses the existing AppConfig feature-flags document. Older desktop builds omit this optional
bridge and continue to work.
Mothership's `computer` tool is available on macOS 14+ once the user opts in under
**Settings → Desktop → Computer Use**. Enterprise organizations can withhold it from a permission
group with the **Computer Use** restriction. Older desktop builds omit this optional bridge and
continue to work.

The user grants Accessibility and Screen Recording through macOS, then approves each app for the
current task or persistently. Persistent app approvals are listed in Settings and can be revoked.
Expand Down
5 changes: 3 additions & 2 deletions apps/desktop/e2e/local-files.spec.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ test('native file tools remember folder consent across chats and restarts until
stdin: {
contents: `import { createRoot } from 'react-dom/client';
import { ToastProvider } from '@sim/emcn';
import { QueryClient, QueryClientProvider } from '@tanstack/react-query';
import { AppRouterContext } from 'next/dist/shared/lib/app-router-context.shared-runtime';
import { PathParamsContext } from 'next/dist/shared/lib/hooks-client-context.shared-runtime';
import { Desktop } from '@/app/workspace/[workspaceId]/settings/components/desktop/desktop';
Expand All @@ -70,11 +71,11 @@ const router = { bfcacheId: 'fixture', back: () => history.back(), forward: () =
createRoot(document.getElementById('settings')).render(
<AppRouterContext.Provider value={router}>
<PathParamsContext.Provider value={{ workspaceId: 'fixture' }}>
<ToastProvider><SettingsHeaderProvider><SettingsHeaderShell>
<QueryClientProvider client={new QueryClient({ defaultOptions: { queries: { retry: false } } })}><ToastProvider><SettingsHeaderProvider><SettingsHeaderShell>
<FeatureFlagsProvider flags={{ dashboards: false, 'workflow-tests': false, 'mothership-model-selector': false, 'mothership-plan-mode': false }}>
<SettingsSectionProvider plane='workspace' section='desktop'><Desktop /></SettingsSectionProvider>
</FeatureFlagsProvider>
</SettingsHeaderShell></SettingsHeaderProvider></ToastProvider>
</SettingsHeaderShell></SettingsHeaderProvider></ToastProvider></QueryClientProvider>
</PathParamsContext.Provider>
</AppRouterContext.Provider>
);`,
Expand Down
1 change: 1 addition & 0 deletions apps/docs/content/docs/platform/enterprise/access-control.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,7 @@ Most platform restrictions refuse the protected operation at the server as well
| Tools | Custom Tools | Prevents agents from calling custom tools. |
| Tools | Skills | Prevents agents from loading skills. |
| Tools | Tool Auto-Approval | Prevents storing an always-allow preference for tool calls. |
| Tools | Computer Use | Prevents Chat from operating apps on a member's Mac, even when the device allows it. |
| Logs | Trace Spans | Withholds per-block spans from log reads and API responses. |
| Logs | Log Export | Prevents CSV execution-log downloads. |
| Logs | Execution Cost | Omits cost and token spend from governed logs and member exports. Organization-admin data drains are not projected this way. |
Expand Down
74 changes: 54 additions & 20 deletions apps/docs/openapi-v2-resources.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7431,7 +7431,8 @@
"disableKnowledgeBaseExport": false,
"deniedPartialAccessProjectIssues": [],
"defaultAgentModel": null,
"deniedAccessRequests": []
"deniedAccessRequests": [],
"disableComputerUse": false
},
"isDefault": false,
"membershipMode": "inherit",
Expand Down Expand Up @@ -7592,7 +7593,8 @@
"disableKnowledgeBaseExport": false,
"deniedPartialAccessProjectIssues": [],
"defaultAgentModel": null,
"deniedAccessRequests": []
"deniedAccessRequests": [],
"disableComputerUse": false
},
"isDefault": false,
"membershipMode": "inherit",
Expand Down Expand Up @@ -7753,7 +7755,8 @@
"disableKnowledgeBaseExport": false,
"deniedPartialAccessProjectIssues": [],
"defaultAgentModel": null,
"deniedAccessRequests": []
"deniedAccessRequests": [],
"disableComputerUse": false
},
"isDefault": false,
"membershipMode": "inherit",
Expand Down Expand Up @@ -7922,7 +7925,8 @@
"disableKnowledgeBaseExport": false,
"deniedPartialAccessProjectIssues": [],
"defaultAgentModel": null,
"deniedAccessRequests": []
"deniedAccessRequests": [],
"disableComputerUse": false
},
"isDefault": false,
"membershipMode": "inherit",
Expand Down Expand Up @@ -22420,6 +22424,10 @@
"maxLength": 1024
},
"description": "Access requests are unavailable for the listed canonical target keys when access is restricted."
},
"disableComputerUse": {
"type": "boolean",
"description": "Revoke computer use. Chat cannot operate apps on a member's Mac, even when the device allows it."
}
},
"required": [
Expand Down Expand Up @@ -22467,7 +22475,8 @@
"disableKnowledgeBaseExport",
"deniedPartialAccessProjectIssues",
"defaultAgentModel",
"deniedAccessRequests"
"deniedAccessRequests",
"disableComputerUse"
],
"additionalProperties": false,
"description": "Resolved group settings. True disables a boolean capability; null allowlists permit every value and empty allowlists permit none. defaultAgentModel selects the default for new Agent blocks without restricting model selection."
Expand Down Expand Up @@ -22600,7 +22609,8 @@
"disableKnowledgeBaseExport": false,
"deniedPartialAccessProjectIssues": [],
"defaultAgentModel": null,
"deniedAccessRequests": []
"deniedAccessRequests": [],
"disableComputerUse": false
},
"isDefault": false,
"membershipMode": "inherit",
Expand Down Expand Up @@ -22678,7 +22688,8 @@
"disableKnowledgeBaseExport": false,
"deniedPartialAccessProjectIssues": [],
"defaultAgentModel": null,
"deniedAccessRequests": []
"deniedAccessRequests": [],
"disableComputerUse": false
},
"isDefault": false,
"membershipMode": "inherit",
Expand Down Expand Up @@ -22962,6 +22973,10 @@
"maxLength": 1024
},
"description": "Access requests are unavailable for the listed canonical target keys when access is restricted."
},
"disableComputerUse": {
"type": "boolean",
"description": "Revoke computer use. Chat cannot operate apps on a member's Mac, even when the device allows it."
}
},
"additionalProperties": false,
Expand Down Expand Up @@ -23056,7 +23071,8 @@
"disableKnowledgeBaseExport": false,
"deniedPartialAccessProjectIssues": [],
"defaultAgentModel": null,
"deniedAccessRequests": []
"deniedAccessRequests": [],
"disableComputerUse": false
},
"isDefault": false,
"membershipMode": "inherit",
Expand Down Expand Up @@ -23132,7 +23148,8 @@
"disableKnowledgeBaseExport": false,
"deniedPartialAccessProjectIssues": [],
"defaultAgentModel": null,
"deniedAccessRequests": []
"deniedAccessRequests": [],
"disableComputerUse": false
},
"isDefault": false,
"membershipMode": "inherit",
Expand Down Expand Up @@ -23423,6 +23440,10 @@
"maxLength": 1024
},
"description": "Access requests are unavailable for the listed canonical target keys when access is restricted."
},
"disableComputerUse": {
"type": "boolean",
"description": "Revoke computer use. Chat cannot operate apps on a member's Mac, even when the device allows it."
}
},
"additionalProperties": false,
Expand Down Expand Up @@ -25920,6 +25941,10 @@
"maxLength": 1024
},
"description": "Access requests are unavailable for the listed canonical target keys when access is restricted."
},
"disableComputerUse": {
"type": "boolean",
"description": "Revoke computer use. Chat cannot operate apps on a member's Mac, even when the device allows it."
}
},
"required": [
Expand Down Expand Up @@ -25967,7 +25992,8 @@
"disableKnowledgeBaseExport",
"deniedPartialAccessProjectIssues",
"defaultAgentModel",
"deniedAccessRequests"
"deniedAccessRequests",
"disableComputerUse"
],
"additionalProperties": false
},
Expand Down Expand Up @@ -26727,7 +26753,8 @@
"disableToolAutoApproval",
"hideSandboxesTab",
"disableOAuthAppAccess",
"disableKnowledgeBaseExport"
"disableKnowledgeBaseExport",
"disableComputerUse"
],
"description": "Feature restriction key returned by access discovery."
}
Expand Down Expand Up @@ -27039,7 +27066,8 @@
"disableToolAutoApproval",
"hideSandboxesTab",
"disableOAuthAppAccess",
"disableKnowledgeBaseExport"
"disableKnowledgeBaseExport",
"disableComputerUse"
],
"description": "Feature restriction key returned by access discovery."
}
Expand Down Expand Up @@ -27439,7 +27467,8 @@
"disableToolAutoApproval",
"hideSandboxesTab",
"disableOAuthAppAccess",
"disableKnowledgeBaseExport"
"disableKnowledgeBaseExport",
"disableComputerUse"
],
"description": "Feature restriction key returned by access discovery."
}
Expand Down Expand Up @@ -27837,7 +27866,8 @@
"disableToolAutoApproval",
"hideSandboxesTab",
"disableOAuthAppAccess",
"disableKnowledgeBaseExport"
"disableKnowledgeBaseExport",
"disableComputerUse"
],
"description": "Feature restriction key returned by access discovery."
}
Expand Down Expand Up @@ -28193,7 +28223,8 @@
"disableToolAutoApproval",
"hideSandboxesTab",
"disableOAuthAppAccess",
"disableKnowledgeBaseExport"
"disableKnowledgeBaseExport",
"disableComputerUse"
],
"description": "Feature restriction key returned by access discovery."
}
Expand Down Expand Up @@ -28452,7 +28483,7 @@
"description": "Access request being reviewed."
},
"changes": {
"maxItems": 43,
"maxItems": 44,
"type": "array",
"items": {
"type": "object",
Expand Down Expand Up @@ -28502,7 +28533,8 @@
"hideSandboxesTab",
"disableOAuthAppAccess",
"disableKnowledgeBaseExport",
"deniedPartialAccessProjectIssues"
"deniedPartialAccessProjectIssues",
"disableComputerUse"
],
"description": "Permission restriction changed by approval."
},
Expand Down Expand Up @@ -28755,7 +28787,8 @@
"disableToolAutoApproval",
"hideSandboxesTab",
"disableOAuthAppAccess",
"disableKnowledgeBaseExport"
"disableKnowledgeBaseExport",
"disableComputerUse"
],
"description": "Feature restriction key returned by access discovery."
}
Expand Down Expand Up @@ -29014,7 +29047,7 @@
"description": "Access request being reviewed."
},
"changes": {
"maxItems": 43,
"maxItems": 44,
"type": "array",
"items": {
"type": "object",
Expand Down Expand Up @@ -29064,7 +29097,8 @@
"hideSandboxesTab",
"disableOAuthAppAccess",
"disableKnowledgeBaseExport",
"deniedPartialAccessProjectIssues"
"deniedPartialAccessProjectIssues",
"disableComputerUse"
],
"description": "Permission restriction changed by approval."
},
Expand Down
21 changes: 0 additions & 21 deletions apps/sim/app/api/desktop/computer/availability/route.ts
View file
Open in desktop

This file was deleted.

1 change: 0 additions & 1 deletion apps/sim/app/extension/providers.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,6 @@ export function ExtensionProviders({
changelog: false,
'mothership-model-selector': modelSelectorEnabled,
'mothership-plan-mode': planModeEnabled,
'mothership-computer-use': false,
}}
>
<SettingsNavigationProvider navigationTarget='_blank'>
Expand Down
4 changes: 0 additions & 4 deletions apps/sim/app/o/[organizationId]/layout.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ import { getSession } from '@/lib/auth'
import { getActiveOrganizationId } from '@/lib/auth/session-response'
import { canUseBenchmarks } from '@/lib/benchmarks/config'
import { isChangelogEnabled } from '@/lib/changelog/feature-flag'
import { isComputerUseAvailable } from '@/lib/computer-use/availability.server'
import { isDashboardsEnabled } from '@/lib/dashboards/feature-flag'
import { isMothershipModelSelectorEnabled } from '@/lib/mothership/feature-flags'
import { organizationRoutes, WORKSPACE_SETTINGS_PATH } from '@/lib/navigation/paths'
Expand Down Expand Up @@ -82,7 +81,6 @@ export default async function OrganizationLayout({
modelSelectorEnabled,
dashboardsEnabled,
benchmarkEnabled,
computerUseEnabled,
workflowTestsEnabled,
changelogEnabled,
projectsEnabled,
Expand All @@ -102,7 +100,6 @@ export default async function OrganizationLayout({
logger.warn('Could not resolve benchmark navigation access')
return false
}),
isComputerUseAvailable(organizationId),
isWorkflowTestsEnabled(organizationId),
isChangelogEnabled(organizationId),
isProjectsEnabled(),
Expand All @@ -124,7 +121,6 @@ export default async function OrganizationLayout({
'workflow-tests': workflowTestsEnabled,
changelog: changelogEnabled,
'mothership-model-selector': modelSelectorEnabled,
'mothership-computer-use': computerUseEnabled,
'mothership-plan-mode': benchmarkEnabled,
'mothership-memory-spaces': benchmarkEnabled,
projects: projectsEnabled,
Expand Down
1 change: 0 additions & 1 deletion ...sim/app/workspace/[workspaceId]/home/components/mothership-chat/rollout-controls.test.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,6 @@ async function render(advanced: boolean, workspaceId?: string) {
changelog: false,
'mothership-model-selector': advanced,
'mothership-plan-mode': true,
'mothership-computer-use': false,
}}
>
<MothershipChat
Expand Down
13 changes: 1 addition & 12 deletions apps/sim/app/workspace/[workspaceId]/home/hooks/use-chat.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -772,7 +772,6 @@ export function useChat(
): UseChatReturn {
const { toast } = useToast()
const modelSelectorEnabled = useFeatureFlag('mothership-model-selector')
const computerUseEnabled = useFeatureFlag('mothership-computer-use')
const workspaceId = typeof owner === 'string' ? owner : undefined
const organizationId = typeof owner === 'string' ? undefined : owner.organizationId
const scopeKey = typeof owner === 'string' ? owner : `organization:${owner.organizationId}`
Expand Down Expand Up @@ -4027,16 +4026,7 @@ export function useChat(
const desktopChatCapabilities =
options?.requestMode === 'assistant'
? {}
: await getDesktopChatCapabilities(
desktopScopeIdRef.current,
!computerUseEnabled
? null
: organizationId
? { organizationId }
: workspaceId
? { workspaceId }
: null
)
: await getDesktopChatCapabilities(desktopScopeIdRef.current)

admissionUnknown = true
/** A reload from here on may find the server holding this id. */
Expand Down Expand Up @@ -4398,7 +4388,6 @@ export function useChat(
queryClient,
upsertChatHistory,
modelSelectorEnabled,
computerUseEnabled,
processSSEStream,
finalize,
resumeOrFinalize,
Expand Down
Loading
Loading