Skip to content

Decrease size of coder slim binary #9380

Closed

Description

@mafredri

As of v2.1.3, our slim/agent binary size is pretty big:

68M coder*
23M coder.gz*

With a bit of hacking (essentially preventing the coderd package from being imported when building the slim binary and eliminating a few other imports), I was able to get the size down significantly:

33M coder*
12M coder.gz*

The size was essentially halved. Reducing the size of the slim binary here should help reduce the full binary (with coder server) size as well, albeit to a lesser extent (via reduced size of bundled slim binaries). First entry into workspaces via e.g. VSCode or JetBrains extensions should also be faster.

This will require a bit of refactoring in multiple places, essentially pulling out shared types from packages that would also pull in coderd.

This issue is for tracking the progress as we slowly make small changes to prevent pulling in all that weight.

Activity

  1. self-assigned this
    on Aug 28, 2023
  2. sharkymark commented on Aug 28, 2023

    @sharkymark
    Contributor

    Customers with requirements that control plane is not in the same region will value this. I presume the slim is used in the VS Code extension and JetBrains plugin.

  3. mafredri commented on Sep 1, 2023

    @mafredri
    MemberAuthor

    For reference, this is the current state of things:

    ❯ make build
    
    ❯ ls -lha build
    total 2.6G
    drwxr-xr-x  2 coder coder 4.0K Sep  1 12:42 ./
    drwxr-xr-x 30 coder coder 4.0K Sep  1 12:38 ../
    -rw-r--r--  1 coder coder 455M Sep  1 12:40 coder-slim_2.1.4-devel+bc9fdd15f.tar
    -rw-r--r--  1 coder coder 117M Sep  1 12:40 coder-slim_2.1.4-devel+bc9fdd15f.tar.zst
    -rw-r--r--  1 coder coder  434 Sep  1 12:40 coder-slim_2.1.4-devel+bc9fdd15f_checksums.sha1
    -rwxr-xr-x  1 coder coder  67M Sep  1 12:40 coder-slim_2.1.4-devel+bc9fdd15f_darwin_amd64*
    -rwxr-xr-x  1 coder coder  67M Sep  1 12:40 coder-slim_2.1.4-devel+bc9fdd15f_darwin_arm64*
    -rwxr-xr-x  1 coder coder  70M Sep  1 12:39 coder-slim_2.1.4-devel+bc9fdd15f_linux_amd64*
    -rwxr-xr-x  1 coder coder  67M Sep  1 12:40 coder-slim_2.1.4-devel+bc9fdd15f_linux_arm64*
    -rwxr-xr-x  1 coder coder  66M Sep  1 12:40 coder-slim_2.1.4-devel+bc9fdd15f_linux_armv7*
    -rwxr-xr-x  1 coder coder  62M Sep  1 12:40 coder-slim_2.1.4-devel+bc9fdd15f_windows_amd64.exe*
    -rwxr-xr-x  1 coder coder  60M Sep  1 12:40 coder-slim_2.1.4-devel+bc9fdd15f_windows_arm64.exe*
    -rwxr-xr-x  1 coder coder 236M Sep  1 12:41 coder_2.1.4-devel+bc9fdd15f_darwin_amd64*
    -rwxr-xr-x  1 coder coder 237M Sep  1 12:42 coder_2.1.4-devel+bc9fdd15f_darwin_arm64*
    -rwxr-xr-x  1 coder coder 238M Sep  1 12:41 coder_2.1.4-devel+bc9fdd15f_linux_amd64*
    -rwxr-xr-x  1 coder coder 234M Sep  1 12:41 coder_2.1.4-devel+bc9fdd15f_linux_arm64*
    -rwxr-xr-x  1 coder coder 233M Sep  1 12:41 coder_2.1.4-devel+bc9fdd15f_linux_armv7*
    -rwxr-xr-x  1 coder coder 230M Sep  1 12:42 coder_2.1.4-devel+bc9fdd15f_windows_amd64.exe*
    -rwxr-xr-x  1 coder coder 227M Sep  1 12:42 coder_2.1.4-devel+bc9fdd15f_windows_arm64.exe*
    
    ❯ git log -1
    commit bc9fdd15f22ef84a893a4ca70cc64b16d3df9907 (HEAD -> main, origin/main, origin/HEAD)
    Author: Cian Johnston <cian@coder.com>
    Date:   Fri Sep 1 10:37:09 2023 +0100
    
        fix(enterprise/cli): correctly set default tags for PSK auth (#9436)
    
        * provisionerd: unconditionally set tag scope to org for psk auth
        * provisionerd: add unit tests for MutateTags
        * cli: add some informational logging around provisionerd tags
        * cli: respect CODER_VERBOSE when initializing logger
    

    I will post updates as we manage to decrease size.

  4. 18 remaining items

  5. mafredri commented on Sep 4, 2023

    @mafredri
    MemberAuthor

    That's a motivation to remove it entirely given it helps reduce the size further.

    @matifali I think that makes sense, however, removing it doesn't help reduce size (tried it). All wsproxy commands (except server), basically only rely on codersdk which is already a common import in the CLI, so we can't shed any weight by removing them.

  6. mafredri commented on Sep 4, 2023

    @mafredri
    MemberAuthor

    With todays PRs merged, we managed to reduce size further by ~7 MB for a total of 33 MB reduction.

    We also introduced a compile-time guard that should help keep the size down. If we noticed any packages that bloat the size, we could add guards to those as well. We could also add a binary size check to tests/CI to ensure it doesn't grow beyond a fixed limit.

    I believe this also marks the point where there may be diminishing returns with regard to effort/size reduction. There's still 2 MB we could remove by taking away the scaletest command (#9485), but it being part of the slim build is currently relied upon for scaletesting.

    Agent before/after:

    -rwxr-xr-x 1 coder coder  44M Sep  1 19:28 coder-slim_2.1.4-devel+18c9f33de_linux_amd64*
    -rwxr-xr-x 1 coder coder  37M Sep  4 20:53 coder-slim_2.1.4-devel+76ab22f53_linux_amd64*
    

    Compressed, we're down to 14 MB (gzip -9) for a total of 9 MB reduction:

    -rwxr-xr-x 1 coder coder 14M Sep  4 20:53 build/coder-slim_2.1.4-devel+76ab22f53_linux_amd64.gz*
    

    ❯ ls -lh build
    total 1.8G
    -rw-r--r-- 1 coder coder 253M Sep  4 20:53 coder-slim_2.1.4-devel+76ab22f53.tar
    -rw-r--r-- 1 coder coder  68M Sep  4 20:53 coder-slim_2.1.4-devel+76ab22f53.tar.zst
    -rw-r--r-- 1 coder coder  434 Sep  4 20:53 coder-slim_2.1.4-devel+76ab22f53_checksums.sha1
    -rwxr-xr-x 1 coder coder  39M Sep  4 20:53 coder-slim_2.1.4-devel+76ab22f53_darwin_amd64*
    -rwxr-xr-x 1 coder coder  39M Sep  4 20:53 coder-slim_2.1.4-devel+76ab22f53_darwin_arm64*
    -rwxr-xr-x 1 coder coder  37M Sep  4 20:53 coder-slim_2.1.4-devel+76ab22f53_linux_amd64*
    -rwxr-xr-x 1 coder coder  36M Sep  4 20:53 coder-slim_2.1.4-devel+76ab22f53_linux_arm64*
    -rwxr-xr-x 1 coder coder  35M Sep  4 20:53 coder-slim_2.1.4-devel+76ab22f53_linux_armv7*
    -rwxr-xr-x 1 coder coder  36M Sep  4 20:53 coder-slim_2.1.4-devel+76ab22f53_windows_amd64.exe*
    -rwxr-xr-x 1 coder coder  34M Sep  4 20:53 coder-slim_2.1.4-devel+76ab22f53_windows_arm64.exe*
    -rwxr-xr-x 1 coder coder 187M Sep  4 20:54 coder_2.1.4-devel+76ab22f53_darwin_amd64*
    -rwxr-xr-x 1 coder coder 187M Sep  4 20:54 coder_2.1.4-devel+76ab22f53_darwin_arm64*
    -rwxr-xr-x 1 coder coder 181M Sep  4 20:53 coder_2.1.4-devel+76ab22f53_linux_amd64*
    -rwxr-xr-x 1 coder coder 178M Sep  4 20:53 coder_2.1.4-devel+76ab22f53_linux_arm64*
    -rwxr-xr-x 1 coder coder 176M Sep  4 20:54 coder_2.1.4-devel+76ab22f53_linux_armv7*
    -rwxr-xr-x 1 coder coder 181M Sep  4 20:54 coder_2.1.4-devel+76ab22f53_windows_amd64.exe*
    -rwxr-xr-x 1 coder coder 177M Sep  4 20:54 coder_2.1.4-devel+76ab22f53_windows_arm64.exe*
    
    ❯ git log -1
    commit 76ab22f539785596d8105f8ba8a734603757417a (HEAD -> main, origin/main, origin/HEAD)
    Author: Mathias Fredriksson <mafredri@gmail.com>
    Date:   Mon Sep 4 22:01:11 2023 +0300
    
        chore(coderd/database): introduce compile-time guard against import in slim build (#9521)
    
        This change introduces a compile-time error when `coderd/database` is
        imported into the slim build. This is to guard against accidentally
        growing the binary size via import.
    
        Ref: #9380
    
  7. mafredri commented on Oct 9, 2023

    @mafredri
    MemberAuthor

    We have now removed exp scaletest from the slim binary via #9934. With this change, we dropped 2 MB.

    -rwxr-xr-x  1 coder coder  35M Oct  9 08:30 coder-slim_2.2.1-devel+38bb854c8_linux_amd64*
    

    gzip -9:

    -rwxr-xr-x 1 coder coder 13M Oct  9 08:30 build/coder-slim_2.2.1-devel+38bb854c8_linux_amd64.gz*
    

    ❯ git log -1
    commit 38bb854c8bb45dc5ecf4246919b8915f431f6dfd (HEAD -> main, origin/main, origin/HEAD)
    Author: Michael Smith <throwawayclover@gmail.com>
    Date:   Fri Oct 6 19:40:37 2023 -0400
    
        fix: update ErrorDialog logic and tests (#10111)
    
    	[...]
    
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions