Skip to content

The coder server cache dir defaults to /tmp, should be ~/.cache #2534

Closed

Description

@mafredri

Storing cache in /tmp is not ideal since it will be wiped on system restarts, as such I'm proposing we move it to ~/.cache as per Deans suggestion.

EDIT: We might also consider defaulting to ~/Library/Caches on macOS, otherwise XDG_CACHE_DIR if set. If there's an equivalent on Windows, consider using that.


cacheDir defaults to /tmp/ on my system. We should probably change that to not use /tmp at all and instead use ~/.cache so we're not dumping 300MB into RAM on startup

Originally posted by @deansheather in #2533 (comment)

Activity

  1. self-assigned this
    on Jun 30, 2022
  2. kotx commented on Aug 20, 2022

    @kotx

    This would be great as Coder server averages around only 200MB of ram usage, but requires me to allocate 512MB on startup, which raises VM costs. Even something like coder server extract-cache would be useful for running during Docker builds instead of runtime.
    Edit: unrelated but does the extracting of the .tar.zst leak? Memory usage hovered around 200MB after extracting, but if it was not extracted, it hovers around 90MB.

  3. mafredri commented on Aug 20, 2022

    @mafredri
    MemberAuthor

    @kotx If you want a quick fix for changing the folder, you can set this in your Dockerfile today:

    ENV CODER_CACHE_DIRECTORY=/data/cache
    

    This would be great as Coder server averages around only 200MB of ram usage, but requires me to allocate 512MB on startup, which raises VM costs. Even something like coder server extract-cache would be useful for running during Docker builds instead of runtime.

    It's not a bad idea, although it sounds like your scenario would benefit more from not using the full binary (that bundles binaries for all platforms). In the build process, we create binaries called "slim", you could use a slim binary for running coder server and manually place the slim binaries in /data/cache/site/bin/ (or whichever path you define).

    Unfortunately, we currently don't have a slim bundle available for download as part of a release.

    (PS. Feel free to open a ticket about this feature, or request slim binaries in releases, or both.)

    Edit: unrelated but does the extracting of the .tar.zst leak? Memory usage hovered around 200MB after extracting, but if it was not extracted, it hovers around 90MB.

    I do get ~90MB even with extraction. The relevant code lives here https://github.com/coder/coder/blob/57f3410009201df2038f06a65c796e9690c2e617/site/site.go/#L583-L630 and I can't see anything that would cause a leak (assuming it's not in the library). Which platform are you running coder server on? (I tested in a docker container based on Alpine, running on amd64.)

    Edit: How are you measuring the memory usage, btw? I wonder if what you may be seeing is the systems filesystem cache being used?

  4. kotx commented on Aug 20, 2022

    @kotx

    Edit: How are you measuring the memory usage, btw? I wonder if what you may be seeing is the systems filesystem cache being used?

    Filesystem cache is definitely possible, I was just checking the memory usage metric on Fly.io which measure the whole VM. So there probably isn't a leak.

  5. kotx commented on Aug 20, 2022

    @kotx

    If you want a quick fix for changing the folder, you can set this in your Dockerfile today:

    Yup, this is my workaround for now:
    https://github.com/kotx/coder-fly/blob/master/make-cache.sh

    #!/bin/sh
    echo "Generating coder server cache..."
    
    set -euxo pipefail
    
    if [ -z $CODER_CACHE_DIRECTORY ]; then
    	echo "CODER_CACHE_DIRECTORY unset, exiting..."
    	exit
    fi
    
    timeout 120 /opt/coder server >/dev/null 2>&1 &
    until [ -d $CODER_CACHE_DIRECTORY/site ] || [ ! -d /proc/$! ]
    do
    	echo "Waiting for $CODER_CACHE_DIRECTORY/site..."
    	sleep 5
    done
    
    echo "Detected coder server cache!"
    sleep 2

    It's not ideal, but it seems to have worked so far.

  6. bpmct commented on Aug 22, 2022

    @bpmct
    Member

    I occasionally lose my https://coder-access-url/bin/ folder when I update my Coder deployment (dpkg -i coder.deb && systemctl restart coder). @mafredri and I chatted and suspect this is the issue. I'll report back if changing the cache dir helped

  7. added a commit that references this issue on Sep 23, 2022
    e254581
  8. added a commit that references this issue on Sep 23, 2022
    f160830
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

apiArea: HTTP API

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions