You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Developers want to see additional metadata about resources in Coder. This could be an instance type, the number of cores an instance has, or a link to the cloud resource itself (say in a GCP dashboard).
As a developer, I want to see information about my workspace, both information that is dynamic (e.g external IP) and the most relevant aspects of the template (no. CPUs).
Definition of done
We should add arbitrary key/value pairs that allow template authors to expose additional metadata. This should be added in our Terraform Provider, allowing customers to expose arbitrary values from existing resources that are provisioned. The key/value pairs would be available from the CLI and the dashboard.
$ coder metadata my-workspace
RESOURCE_NAME KEY VALUE
about workspace-id acf2a70d-6347-4b8c
about quicklinks [onboarding guide](https://mywiki.com/onboarding), [troubleshooting](https://mywiki.com/troubleshooting)
main-dev type kubernetes_pod
main-dev api_key (hidden)
Some values are sensitive. Use `coder metadata myworkspace <key>` to fetch individual values.
In the future, perhaps resource_id = "global" or global = true could be used to display metadata workspace-wide in another panel if the hack is widely used.
I'd like to give customers more help here. Are there certain common items we can discover automatically? Or perhaps naming conventions that tell us to pull out particular variables?
Also a limitation (perhaps) of the above is around units/types and naming. For instance we measure RAM and disk in GB, but they're separate concepts. And they may want to measure CPU in a strict count.
I wouldn't assume in terms of limitations on what can be relevant. But if we make it zero effort to see how much e.g. S3 space is in use, I think that'd be good. And we should be able to do it with no coupling worth noting.
We can't really make arbitrary API calls (like getting the s3 bucket used) to these clouds. That would couple us heavily with the provisioner environment. So, we're limited to what terraform is exposing as variables.
If we want some "default metadata" to expose, we can just put those blocks in our example templates. That way the user can easily discover how to disable them and how to expose more metadata.
Sorry, "use" was the wrong word. I meant "allocated," which is something we can know from just our local information. We're targeting quotas to work on workspaces not in them, so that should be fine.
I don't want to give up on actual usage one day; I think it's pretty doable, but that's not what I meant above.
Sorry, "use" was the wrong word. I meant "allocated," which is something we can know from just our local information. We're targeting quotas to work on workspaces not in them, so that should be fine.
Ok, in that case I think we should include default metadata blocks in our examples.
I think the way we should provide access to the secrets in the workspace is via a command coder env get <key name> (run in the workspace that has access to the values). That is flexible enough to work on multiple operating systems in a consistent way and gives users a starting point where they can then store the values in anything else they want.
I believe this can be done in a satisfactory manner with no web UI. The template files and the above coder command should do it pretty well. Perhaps there's future work that would justify a web UI, but I think to start we don't need any of CLI, API, or web UI.
If you're suggesting that we add a coder env command in addition to environment variables, check out how Vercel approaches the problem.
TL;DR they don't have a vercel env get command, probably because it leads to an anti-pattern in scripts where you call out to vercel instead of the environment. This breaks the repo when running in CI or another environment without the coder command present. Also: there is no compatibility risk with scripts? All shell interpreters and most programming languages abstract environment variables to a portable interface.
To my knowledge, we haven't had any user requests for this feature. It does unlock some valuable use cases:
display CPU cores to workspace user
display links to relevant docs to workspace user
display necessary values from Terraform-generated resources (e.g API key) for consumption elsewhere
hide resource type (unnecessary complexity in dashboard)
display the instance type to the workspace user
down the road: custom icons for resources
I think it's still valuable to define the feature & schema but I suggest we don't prioritize this in coming weeks, especially as we focus on "blocking" issues. For now, I consider it a "nice-to-have"
After a bit more research, I think metadata is still the best name for this feature. As Ammar mentioned, it is a broad term but the feature itself will likely expand to support more than simply key/value pairs in future iterations so the term is "encompassing".
I also think it is distinct enough from parameters.
Also spoke with @kylecarbs some more and since it's on the roadmap let's keep it moving! Over the next week or so, we might adjust as the roadmap changes but spoke with @dwahler and @BrunoQuaresma and seems like we're good to go.
For the initial backend implementation, here's what I'm thinking:
To keep things simple, for now let's assume that metadata is always associated with a resource, which might be a null_resource
Metadata values are always either strings or null (integers etc. would be supported in templates, but stringified)
The provisioner API will not directly return coder_metadata resources in job results (since they're an implementation detail of the Terraform provisioner) and instead extracts their key/value data and returns it along with the "target" resource
Metadata is stored in a new workspace_resource_metadata DB table and included as a field of the WorkspaceResource object that we return from /api/v2/workspacebuilds/{id}/resources:
@@ -28,6 +28,12 @@ type WorkspaceResource struct {
Type string `json:"type"`
Name string `json:"name"`
Agents []WorkspaceAgent `json:"agents,omitempty"`
+ Metadata []WorkspaceResourceMetadata `json:"metadata,omitempty"`
}
+type WorkspaceResourceMetadata struct {+ Key string `json:"key"`+ Value string `json:"value"`+ Sensitive bool `json:"sensitive,omitempty"`+ // in future: Icon? Order? Type? LastUpdated?+}
The API response includes both "explicit" metadata fields returned from the provisioner, and "implicit" ones like type, with explicit taking precedence and null-valued fields omitted from the response
How does that sound? If I don't run into any unexpected snags in the implementation, I expect to be able to get the backend and CLI parts of this done within the next week.
may not actually be valid Terraform code. If the target resource is something like a pod whose lifecycle is tied to the workspace start/stop, then it has a count attribute which may be either 0 or 1, and you get an error like this when trying to directly reference its id field:
Error: Missing resource instance key
Because kubernetes_pod.main-dev has "count" set, its attributes must be accessed on specific instances.
The error is from Terraform itself, not from our provider, and I haven't yet found anything in the language that would allow us to make the original syntax work. As a workaround, you can do this instead, but it's kind of ugly:
Activity
[-]Resource Metadata[/-][+]Resource Metadata + Secrets[/+]ketang commentedon Jul 14, 2022
I'd like to give customers more help here. Are there certain common items we can discover automatically? Or perhaps naming conventions that tell us to pull out particular variables?
Also a limitation (perhaps) of the above is around units/types and naming. For instance we measure RAM and disk in GB, but they're separate concepts. And they may want to measure CPU in a strict count.
ammario commentedon Jul 14, 2022
I don't think we can easily assume which parts of the resource object are most relevant to the user.
ketang commentedon Jul 14, 2022
I wouldn't assume in terms of limitations on what can be relevant. But if we make it zero effort to see how much e.g. S3 space is in use, I think that'd be good. And we should be able to do it with no coupling worth noting.
ammario commentedon Jul 14, 2022
We can't really make arbitrary API calls (like getting the s3 bucket used) to these clouds. That would couple us heavily with the provisioner environment. So, we're limited to what terraform is exposing as variables.
If we want some "default metadata" to expose, we can just put those blocks in our example templates. That way the user can easily discover how to disable them and how to expose more metadata.
ketang commentedon Jul 14, 2022
Sorry, "use" was the wrong word. I meant "allocated," which is something we can know from just our local information. We're targeting quotas to work on workspaces not in them, so that should be fine.
I don't want to give up on actual usage one day; I think it's pretty doable, but that's not what I meant above.
ammario commentedon Jul 14, 2022
Ok, in that case I think we should include default metadata blocks in our examples.
ketang commentedon Jul 19, 2022
I think the way we should provide access to the secrets in the workspace is via a command
coder env get <key name>(run in the workspace that has access to the values). That is flexible enough to work on multiple operating systems in a consistent way and gives users a starting point where they can then store the values in anything else they want.I believe this can be done in a satisfactory manner with no web UI. The template files and the above
codercommand should do it pretty well. Perhaps there's future work that would justify a web UI, but I think to start we don't need any of CLI, API, or web UI.ammario commentedon Jul 19, 2022
Are you suggesting that on top of the original post's suggestion or instead of?
Also environment secrets can already be injected without any new feature, which is why I left the environment injection outside of my suggestion:
ammario commentedon Jul 19, 2022
If you're suggesting that we add a
coder envcommand in addition to environment variables, check out how Vercel approaches the problem.TL;DR they don't have a
vercel env getcommand, probably because it leads to an anti-pattern in scripts where you call out to vercel instead of the environment. This breaks the repo when running in CI or another environment without thecodercommand present. Also: there is no compatibility risk with scripts? All shell interpreters and most programming languages abstract environment variables to a portable interface.31 remaining items
bpmct commentedon Jul 20, 2022
To my knowledge, we haven't had any user requests for this feature. It does unlock some valuable use cases:
I think it's still valuable to define the feature & schema but I suggest we don't prioritize this in coming weeks, especially as we focus on "blocking" issues. For now, I consider it a "nice-to-have"
cc @misskniss @tjcran
ketang commentedon Jul 20, 2022
Where a subset of this may relate to demonstrated customer needs is with respect to resource quotas.
bpmct commentedon Jul 21, 2022
After a bit more research, I think
metadatais still the best name for this feature. As Ammar mentioned, it is a broad term but the feature itself will likely expand to support more than simply key/value pairs in future iterations so the term is "encompassing".I also think it is distinct enough from
parameters.Also spoke with @kylecarbs some more and since it's on the roadmap let's keep it moving! Over the next week or so, we might adjust as the roadmap changes but spoke with @dwahler and @BrunoQuaresma and seems like we're good to go.
ammario commentedon Jul 21, 2022
Thanks @bpmct. What is the expected timeline for an MVP into
main?dwahler commentedon Jul 21, 2022
For the initial backend implementation, here's what I'm thinking:
null_resourcecoder_metadataresources in job results (since they're an implementation detail of the Terraform provisioner) and instead extracts their key/value data and returns it along with the "target" resourceworkspace_resource_metadataDB table and included as a field of theWorkspaceResourceobject that we return from/api/v2/workspacebuilds/{id}/resources:type, with explicit taking precedence and null-valued fields omitted from the responseHow does that sound? If I don't run into any unexpected snags in the implementation, I expect to be able to get the backend and CLI parts of this done within the next week.
kylecarbs commentedon Jul 21, 2022
That seems perfect to me!
tjcran commentedon Jul 25, 2022
@ammario @bpmct @kylecarbs we now just need to weigh this against the other roadmap items for August, but this seems like a solid candidate.
bpmct commentedon Jul 26, 2022
@tjcran This is currently in this sprint, the July roadmap, and being worked on. We can consider prioritizing against the August roadmap if necessary
dwahler commentedon Jul 27, 2022
I've gotten the core of this (provisioner and API only) implemented in PRs coder/terraform-provider-coder#34 and #3242.
Unfortunately, I hit a noticeable snag, which is that the syntax described in the requirements:
may not actually be valid Terraform code. If the target resource is something like a pod whose lifecycle is tied to the workspace start/stop, then it has a
countattribute which may be either 0 or 1, and you get an error like this when trying to directly reference itsidfield:The error is from Terraform itself, not from our provider, and I haven't yet found anything in the language that would allow us to make the original syntax work. As a workaround, you can do this instead, but it's kind of ugly:
Any suggestions for better ways to deal with this are welcome.
bpmct commentedon Jul 27, 2022
@dwahler that seems reasonable to me. Template admins could add a
countto the metadata to avoid the try function 🤷🏼Another thing that comes to mind is using Terraform's
depends_onto establish the relation, but I think that introduces some more problems:depends_onor legitimately need it to depend on a resourcebpmct commentedon Jul 28, 2022
Note: I'll write the docs for this once the frontend comes in and close the issue once that is complete.