django-redis persisting json data

Question

I have a small django site which controls an anstronomy dome and house automation. On start up the project loads 3 json files: relays, conditions and homeautomation. To avoid constant reading and writing to the Pi4's ssd I load the json files into REDIS (on start up in apps, see below). I already have REDIS running in a docker as the project uses celery.

My problem is that within a few minutes of loading the json into REDIS it clears the data out of cache.

I load the json file in the form of a dictionary (dict) in apps

cache.set("REDIS_ashtreeautomation_dict", dict, timeout=None)

and set

CACHES = {
"default": {
    "BACKEND": "django_redis.cache.RedisCache",
    "LOCATION": "redis://redis:6379",
    "OPTIONS": {
        "CLIENT_CLASS": "django_redis.client.DefaultClient",
        "SERIALIZER": "django_redis.serializers.json.JSONSerializer",
        "TIMEOUT": None
    }
}

}

I don't need the data to persist if the dockers go down and I don't need db functions. Caching these files is ideal but I need them to 'stay alive' for the lifetime of the server.

Thank you.

A typical way to handle data with those requirements be to simply load the data into a Python dictionary one time at startup of the process. Is there a reason you need to put it in a "cache", and use Redis? — Kevin Christopher Henry
– Kevin Christopher Henry, Commented Jun 23, 2022 at 15:16
Thank you Kevin. That is how I originally set up the project, the problem is that a 3rd party app provides sensor data in a json file every 15 seconds. I need to merge that with data from another sensor (every 15 seconds). Plus the data is used across a number of apps so sharing / instantiation is an issue. I already have REDIS running in a docker so this seemed an ideal solution. Particularly when the Django documentation says “TIMEOUT.....TIMEOUT to None so that, by default, cache keys never expire.” I would like to get it working if I can. — SgtBilko
– SgtBilko, Commented Jun 23, 2022 at 15:42
Your problem may simply be that you've put TIMEOUT under OPTIONS, when it should be at the same level in the hierarchy. See the example configuration in the documentation. Note that None isn't a guarantee that your data won't ever get purged. Redis, for example, has its own set of (configurable) eviction policies that it will turn to when it starts to run out of memory. — Kevin Christopher Henry
– Kevin Christopher Henry, Commented Jun 23, 2022 at 16:01

SgtBilko · Accepted Answer · 2022-06-23 17:16:05Z

0

Thank you Kevin.

Moving TIMEOUT solved the issue.

CACHES = {
"default": {
    "BACKEND": "django_redis.cache.RedisCache",
    "LOCATION": "redis://redis:6379",
    "TIMEOUT": None,
    "OPTIONS": {
        "CLIENT_CLASS": "django_redis.client.DefaultClient",
        "SERIALIZER": "django_redis.serializers.json.JSONSerializer",
    }
}

}

I am going to include some code to catch the long term REDIS 'eviction' policies (i.e. reload the json data). I don't want to delve into the REDIS docker.

Thanks

Ian

answered Jun 23, 2022 at 17:16

SgtBilko

553 silver badges12 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

django-redis persisting json data

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related