The RAM bill for Redis-backed online stores — some measured numbers, and a question about hot-set sizes #6708

I've been measuring the memory footprint of feature serving, and I'd like a
sanity check from people running Feast in production.

The numbers. On a real corpus, a bit-packed, arithmetically-addressed
read-only artifact holds the same retained fields in 13.28× less space than
Redis dense encoding, and 32.27× less than Redis hash encoding — the hash
shape being what Feast's Redis online store writes (per the online store format
spec). Footprint is deterministic, so unlike latency claims there are no
benchmark caveats to argue about: byte counts are byte counts, and I'm happy to
share the exact methodology.

Why it might matter. Managed Redis-class memory runs roughly $20–36 per
GB-month before HA (HA roughly doubles it), so a 100 GB hot feature set is
~$25k–85k/year per workload. At 32×, the same features fit in ~3 GB —
in-process on the serving host, no separate cluster. With a lot of teams
re-evaluating their caching layer after the Redis/Valkey licensing churn, the
RAM line item seems worth measuring rather than assuming.

What this is and isn't, stated plainly. It's a derived, read-only, mmap'd
artifact rebuilt from the offline store — which happens to match Feast's
materialization model almost exactly (materialize = rebuild). It is not a
database: no writes (rebuild-only by design), no range scans, and zstd
compresses smaller at rest — the property is packed and random-accessible
(~0.1 µs in-process lookups), not smallest-possible. The point isn't beating
Redis on latency; a network hop dominates either way. It's the RAM bill.
Build interruption and power-loss behavior are tested — incomplete or invalid
artifacts are detected and refused at serving — and the construction work is
on file (U.S. provisional filed Aug 2026).

The question. Does anyone here run online stores with hot sets in the
~50 GB+ range where Redis/Valkey RAM cost is an actual line item you track? If
so, I'm considering building this as a custom online store connector (the
plugin interface looks like it fits cleanly) and would rather validate the
pain first than build in a vacuum. If your hot set is small, this genuinely
doesn't matter for you — that's the honest boundary.

Methodology, full numbers, and the retirement record for figures that didn't
survive my own audit are all available if anyone wants to dig in.

— Wayne F. Brasco Jr.