Question about cache eviction

[blueberry6401]

Currently there're 3 options about cache eviction: max, ttl, and ttlAutopurge.
In my specific case: the keys are in very wide range

• Use ttl: because the cache key is not being hit again, setting ttl is not doing much, and memory grows over time.
• Use max: this is fine but I have to define max, in my app this value depends on many factors that changes over time, so guessing a number is not what I want.
• Use ttlAutopurge. This option will trigger a setTimeout to delete cache after ttl => this is cause a performance degrade if my cache has about 100k keys for example.

So I kinda stuck to choose which one to do, currently I'm choosing (2 - guessing a number for max) but I really don't like it.
What if I care about my memory, I want the cache to be evicted, but no need to be in exact timing like setTimeout. Maybe I just use an interval of 60s to loop through the whole cache and delete anything which are expired. Is this way faster than setTimeout, and does lru-cache have built in way to do it?

[isaacs]

Currently there're 3 options about cache eviction: max, ttl, and ttlAutopurge.

There's 4 actually 😅 (Or three, but ttlAutopurge and ttl are really the same thing).

It sounds like the best option for you is to set a maxSize and sizeCalculation rather than max.

Then your sizeCalculation function can adjust as needed for whatever various factors you care about.

Or, you can skip the sizeCalculation function, and provide an explicit size value for each set. (If you set a maxSize, and don't have a sizeCalculation function set, then you must set a size explicitly for each set.)

If you go that route, you can of course also set a max that's fairly high, just to pre-allocate the memory up front. That buys you some added performance, which you pay for with a bit of an up-front perf hit to instantiate the cache, plus some memory that's locked up for the lifetime of the cache, and it also sets a safety limit that you know it won't go past.

So, yeah, this is the problem with flexible lowlevel libs like this, you get a lot of flexibility, but it means you have to think a little harder about how to model things in some cases.

What you can also do in addition to a max or maxSize, if you want to be somewhat conservative on memory usage, but don't care so much about ttl for data validity, is to set a ttl value that's fairly high, with { allowStale: true, noDeleteOnStaleGet: true } on it. Then, on some fairly infrequent interval (like maybe once a minute, like you mentioned) call cache.purgeStale().

So all that being said, and apologies if this is a dumb question, but if you are concerned about memory usage and you don't have a lot of key collisions, why are you using a cache? A cache like this is really only a good idea in cases where the data access pattern is heavily skewed to a fairly small subset of keys at any given time.

[blueberry6401]

explicit

I don't want to set max or maxSize to limits the cache, because I will monitor the cache size by memory usage of pod and if it reaches the limit, autoscaler will do its job. Setting a max value, if too low, will affect app's functionality; if too high, will affect memory usage.
My use case for example, is to cache transaction data. Let's say I have 1-5k transactions per second, I want to cache transaction ID with transaction data, and expire them in 10 minutes. Since the ID is randomly, I want to make sure they will be evicted in 10 minutes to release memory, otherwise memory leak will occur. Also I can't anticipate the max transaction count in the cache, it depends on our users' activities.
I also find you're having a different repo ttlcache, which I think is more suitable for my use case. I hestitate to use it in production because it has not widely used compared to lru-cache. But also it has nicer idea about maintaining only one setTimeout; why don't you implement this idea in lru-cache? Or setting only one interval to clear stale keys periodically, will it have better performance than call multiple setTimeout?

[isaacs]

I also find you're having a different repo ttlcache, which I think is more suitable for my use case. I hestitate to use it in production because it has not widely used compared to lru-cache.

True. There are some folks using it in real life scenarios, but it hasn't gotten nearly as much play testing or as much of my development attention as lru-cache, so I'd be a bit less confident in it behaving correctly in all possible edge cases.

But also it has nicer idea about maintaining only one setTimeout; why don't you implement this idea in lru-cache? Or setting only one interval to clear stale keys periodically, will it have better performance than call multiple setTimeout?

I've thought about both of those ideas, probably will eventually copy the same approach ttlcache uses. Just hasn't been a high priority, and lru-cache is so popular, I try to be very careful about changes that can potentially have a disruptive impact, which makes it a bit more work to take on, especially since it's already so flexible and thus easy to have unintended consequences.

It does sound like you're looking for something that is primarily ttl based rather than primarily use recency based. Perhaps mnemonist has something that would be a more precise fit? That's another very high quality set of caching and data structure libraries in js, which tend to be somewhat less flexible/ergonomic than lru-cache/ttlcache, but also more targeted to specific use cases and performance characteristics.