TTL should be configurable
coding-bunny opened this issue · comments
Hello,
Right now your TTL for the Cache is forced to one year.
Would it be possbile to have this configurable?
Cause one year is quite a long time for data.
Just a small snippet I had to add to our application.rb to make the cache be 7 days instead of a year:
# HACKY TIME!
# We redefine the constant of the Redis Rack Cache to be 7 days instead of one year.
# Since this gem doesn't like to be configured, Oli forces it to be configured using ruby sugar.
::Redis::Rack::Cache.send(:remove_const, :DEFAULT_TTL)
::Redis::Rack::Cache.send(:const_set, :DEFAULT_TTL, 7.days.to_i)
config.action_dispatch.rack_cache = { metastore: "#{Chamber[:redis][:cache]}/metastore", entitystore: "#{Chamber[:redis][:cache]}/entitystore" }
So yeah, having a configuration option would be cool.
@coding-bunny It might be a little confusing because the documentation for configuring TTL is in rack-cache itself. We didn't document it in this project presumably because the documentation can already be found in rack-cache.
You should probably modify your config as follows:
config.action_dispatch.rack_cache = {
metastore: "#{Chamber[:redis][:cache]}/metastore",
entitystore: "#{Chamber[:redis][:cache]}/entitystore",
default_ttl: 1.week.from_now.to_i
}Doesn't work. I tried setting that value, including the use_native_ttl: true option, but nonetheless the Redis keys keep being set to one year for the Rack Cache
@coding-bunny what version of redis-rack-cache are you running?
latest one as far as I know
@coding-bunny can you tell me the version number? cat Gemfile.lock | grep redis-rack-cache
gem 'redis-rack-cache', '~> 2.0.0.pre'
From what I found in the code, using my snippet it seems the default TTL is always used as the Rails framework etc doesn't set any TTL by default, and we're not explicitly calling the cache methods in our controllers, so we rely on the defaults everywhere.
So when I look at Redis, the entitystore and metastore keys all receive the default TTL of one year, even when I explicitly tell the app to use the native ttl and a timeout of 7 days.
I'm sure it works fine when you explicitly use the fragment cache methods with timeouts.
@coding-bunny Did you try expires_in: as the config setting? It's a little confusing because we haven't really documented this anywhere, but I think that's the correct setting. Sorry for leading you down the wrong path...
https://github.com/redis-store/redis-store/blob/master/lib/redis/store/ttl.rb#L29-L34
The main reason why we made a default TTL was to prevent redis servers that withstand a certain amount of load from getting OOM errors every so often due to the cache ballooning out of control, since expiration was never set on any of the keys. We added the default expiration of a year to prevent the cache from ballooning out of control, but also to prevent any errors that might come about from setting an ultra-short time frame. Some people may have been relying on the "endless" cache keys of yesteryear.
Anyway, let me know if :expires_in fixes your problem. This should really be documented, it's a very useful feature, and I'd like to confirm whether I can add what we talked about here to the README or wiki or something.
will give this a spin tomorrow when im at work and let you know the results.
I've removed the hackey code I have set. I applied to the expires_in: key to our rack cache settings like so:
config.action_dispatch.rack_cache = {
metastore: "#{Chamber[:redis][:cache]}/metastore",
entitystore: "#{Chamber[:redis][:cache]}/entitystore",
expires_in: 7.days.to_i
}
However, when I flushed all keys and looked at the TTL of the entity store and metastore, it's not set to 7 days:
127.0.0.1:6379[1]> TTL metastore:88d6132fad089612cb220247668fa102186becb7
(integer) 31535935
It falls back to one year.
I tried checking if I need to supply it as a separate parameter or something, but it didn't work.
Also tried it once more with the default_ttl and use_native_ttl params, but it keeps falling back to one year unless I add my hacky code.
we solved the same problem by setting use_native_ttl: true in config.action_dispatch.rack_cache.
FYI
https://github.com/rtomayko/rack-cache/blob/master/lib/rack/cache/meta_store.rb#L61
https://github.com/rtomayko/rack-cache/blob/master/lib/rack/cache/meta_store.rb#L95
Hope this helps.
does not help at all.
If you look at the previous comments, you can see we tried this and it makes no difference whatsoever in our case.
And that is probably the issue.
As I pointed out originally : We do not rely on the controller caching methods.
We rely on the default behaviour everywhere for the cache.
Our entities and views use standard rails. We have no explicit calls to cache_for or other caching methods.
And in that the case, I'm pretty sure the response.fresh? will always return false, meaning the default_ttl is ignored, and the rack implementation falls back on the default const, which is one year.
Hey @coding-bunny, I was a little confused between the configuration for this and the one for redis-rails. You're correct though, the default TTL doesn't seem to be configurable on a global scale for Redis::Rack::Cache because we're setting it every time as ttl ||= Redis::Rack::Cache::DEFAULT_TTL. You probably still need use_native_ttl: true in the config but we also need a way to specify a default TTL.
Yes, but right now it's defined as the constant. Would be nicer if it was a class variable for example and allow it to be overwritten. Even if your use the ||=, it would still give us a one time chance in an initializer to set the value for example.
Cause now I hacked it to 7 days.
I'm fine using the use_native_ttl option, as long as it allows me to configure it :P
@coding-bunny yeah that was what I was saying. a change needs to occur in the gem to allow you to do this. :) thanks for letting us know, one of us will try and get to it...but if you want this feature expedited, I encourage you to submit a pull request.
might give it a spin tomorrow when I'm at work ;)
hey sorry this is taking so long @coding-bunny, just got around to rebasing my configurable-default-ttl branch.
No rush from my side :)
New job opportunity, so I'm at the moment not involved anymore with Ruby/Rails, but still following up though :D