Feature Request: Floating-Point ttl

[waterlubber]

I'd love to use this library for very short caching intervals (eg 10-50ms), but ttl only supports integer seconds. I'm currently looking into a way to do this more manually, but are there any ways to conveniently use a subsecond ttl?

I'm not sure if it's possible to add this feature without breaking existing code, either.

[James Kominick]

It's not currently possible, but I'd be happy to accept a PR if you can get to it before I have time to work on it. It would be pretty straightforward to add support to the TimedCache and TimedSizedCache to accept a subsecond_nanos so the pair of (seconds, subsec_nanos) can be compared against the pair that makes a Duration. Then there's several places in the impl of both cache types that need to be updated to include checking subsec nano (ex. here and here), and also adding a macro argument for it (around here)

[Luke Newcomb]

@jaemk
I am interested in implementing this and submitting a pull request. Although I have a few implementation concerns/questions. I'm guessing that changing the outward facing API in a breaking way would be a concern of yours.

What should be done about the following functions to allow subsecond TTL?

• TimedCache::with_lifespan
• TimedCache::with_lifespan_and_capacity
• TimedCache::with_lifespan_and_refresh
• Cached::cache_lifespan
• Cached::cache_set_lifespan

New versions of all of these could be created with the ability to pass in a lifespan with additional subsecond_nanos, but the function names seem to get quite long and redundant. It also seems beneficial to switch the internal representation of the lifespan to a Duration.
It seems like it would be nice to just pass in a duration as the lifespan, but that would be a breaking API change.

[Max Thomson]

+1 for supporting subsecond cache timeouts. I have a few applications that have boatloads of concurrent database calls on every HTTP request, yet I need data that is fresh within ~250ms.

Although breaking changes aren't the prettiest, it seems like the best way forward is for the macros to accept a Duration type for time. Switching to a Duration representation for the lifespan makes sense and would allow for more precise (and readable) control over cache invalidation. It would also align with Rust's standard practice of using Duration for time-related operations.

The upgrade path is relatively straight forward™️ (happy to contribute a codemod if wanted)

- #[cache(time = 1)]
+ #[cache(time = Duration::from_secs(1))]

[Omid Rad]

Just to add some notes here:

1. Some stores (backends) may not support ms. like Redis (Edit: Redis supports ms, with PX flag)
2. Working with time is a bit critical, for example, if you set it a very low number, like 1ms or even 10ms you most probably will always miss the cache. It can be worse for non-memory caches, like disk or Redis.
3. AFAIK, we don't have GC at the moment. But I think we should have it some day! In case we have it, GC can be a bit in trouble for auto-flushing short-living caches. Many reallocations. (Do we have GC functionality anywhere in the lib @jaemk for timed stores?)