Repository navigation
Description
Our team's developers need to access their local USB devices in their Coder workspaces via USB/IP, and to minimize the latency between a USB/IP server running on a developer's local machine (which serves the USB hardware they have connected to their machine) and the USB/IP client running in a Coder workspace (which attaches that USB hardware remotely over the network), we'd like our developers to be able to provision Coder workspaces to run on their local machines (e.g. using the Docker Terraform provider).
I was describing this use case to @kylecarbs on Slack, and he mentioned:
Coder v2 has the concept of a provisioner daemon. It accepts provision operations (like workspace start/stop) and performs them. Developers could run a provisioner daemon on their local machine, and then Terraform would execute against it. This would enable Terraform to run locally, so you could use
unix:///var/run/docker.sock. Our architecture certainly enables this.
We'll have to work with y'all to get the UX nice, but we can certainly provide a flow like:
- Download the Coder CLI.
- Run "coder provisioner" in the background.
- Have the workspace provisioned on your local machine, using that provisioner daemon.
We'll have to add some targeting to provisionerd, but that should be simple enough!
That sounds exactly like what we were hoping for, so I've created this GitHub issue to formally request this feature in Coder. 😃
Building on what @kylecarbs suggested above, I think enabling something like the following would meet our requirements:
- A developer can run a Coder CLI command (e.g.
coder provisioner) to launch a local provisioner that is associated/authenticated with coderd using the credentials/URL the developer previously provided viacoder login. - A developer can specify that workspace operations (e.g. creation, stopping, etc.) should use a particular provisioner daemon so that they can ensure that a workspace will be created/provisioned/stopped on their local machine as opposed to other machines that would not immediately have the ability to provision/control Terraform resources on the developer's local machine.
- For the CLI, maybe this could be specified via an option when running
coder workspace createlike--provisioner-daemon <PROVISIONER_DAEMON_ID>where a developer specifies the name/reference to the provisioner daemon of interest via<PROVISIONER_DAEMON_ID>? - For the web UI, maybe this could be a drop-down on the workspace creation webpage from which the user could select a provisioner daemon from a list of provisioner daemons that they are authorized to use (see below)?
- For the CLI, maybe this could be specified via an option when running
- A developer can restrict other Coder users from creating workspaces using a provisioner daemon they are running on their local machine. I think this would help prevent one developer from accidentally (or perhaps maliciously) creating a workspace on a different developer's machine as well as avoid having too many options for selecting a provisioner daemon in the new workspace creation web UI (see above about the proposed drop-down option) when many developers maintain their own provisioner daemons.
- I think something that would work well for our team would be if the default behavior of Coder would be to give a developer the ability to create workspaces using a provisioner daemon they've spawned themselves but restrict other users from utilizing it unless the provisioner daemon has been associated/registered to be shared throughout a Coder organization. And maybe that association/registration could be performed by the developer via a CLI command or web UI page?
What do you think?
Activity
kylecarbs commentedon Apr 12, 2022
@kconley-sq I was able to get this working. It's not scoped to a specific user right now, but we are able to use the Docker Terraform Provider to spin up a workspace. This is a basic template that lets you specify an image. Pretty neat!
bpmct commentedon May 5, 2022
@arivictor would something like this work for your use case of local machines?
tjcran commentedon May 16, 2022
I am adding this one into Community MVP milestone. It unblocks specific use cases we've heard from multiple people now.
misskniss commentedon May 17, 2022
Please add your planning poker estimate with ZenHub @kylecarbs
7 remaining items
dwahler commentedon Jul 28, 2022
It sounds like there are a few prerequisites to being able to implement this as more than just a demo:
First of all, we need to decide on a notion of identity for provisioners, so that we can distinguish between different instances of the Terraform provisioner. Simplest thing is probably just to assign each of them a UUID, but we might also want to consider namespacing them, so that you'd have e.g.
globalanddwahler/local.Secondly, we need to think about security (authn/authz). #44 covers authentication between coderd and provisionerd. For authorization, @kconley-sq's proposed defaults sound great, but I can think of some other potential things to consider:
And then there are lifecycle issues. It probably doesn't make sense to delete a provisioner instance if there are existing workspaces associated with it. But if the provisioner goes away unexpectedly (e.g. your laptop dies) then we probably need a way to clean up its state.
github-actions commentedon Oct 22, 2022
This issue is becoming stale. In order to keep the tracker readable and actionable, I'm going close to this issue in 7 days if there isn't more activity.
kylecarbs commentedon Nov 7, 2022
#4935
kylecarbs commentedon Nov 9, 2022
Provisioner daemons and template versions will gain the concept of tags. Tags must match for a provision job to execute on the daemon.
Tags will be defined in the frontmatter of a template's README:
When a provisioner job is queued (like template creation or workspace build), it will be clear to the user that the job is pending on a set of tags. If
provisioner: useris specified, we'll make a unique exception in the UI directing the user to start a provisioner daemon. All users can create provisioner daemons, but they are useless unless templates target them.# Registers and starts a new provisioner daemon with a set of tags $ coder provisionerd start --tag environment=staging --tag scope=organizationThe
scopetag specifies whether the provisioner daemon is registered for the organization or the user. If not provided, the default value isuser. If the user doesn't have permission to create an organization-wide provisioner daemon, the request to start will fail.The following tags will be specified by default but can be overridden by the user:
arch: Specifies the architecture of the host system.os: Specifies the operating system of the host system.kylecarbs commentedon Dec 14, 2022
This is done!
matifali commentedon Jan 4, 2024
@bpmct @kylecarbs, We currently do not have the ability to run a workspace on the developer's local docker daemon (as requested originally in the issue); I think we should reopen this.