Documented Behaviour of NumCounters Not Observed

[Nishant Das]

Currently it is stated that

// For example, if you expect your cache to hold 1,000,000 items when full,
// NumCounters should be 10,000,000 (10x). Each counter takes up 4 bits, so
// keeping 10,000,000 counters would require 5MB of memory.

over here. However this behaviour isnt correct when utlising the cache. When utilising the cache with the following config

cache, err := ristretto.NewCache(&ristretto.Config{
		NumCounters: 10000,           // number of keys to track frequency of (10000).
		MaxCost:     1000, // maximum cost of cache (1000 objects).
		BufferItems: 64,             // number of keys per Get buffer.
		Cost: func(value interface{})int64{
			return 1
		},
	})

After the cache is filled with the first 1000 objects and is at its maximum capacity, any new items cannot be added into it. All my following Set fail. None of the old items currently in the cache are evicted to make way for new ones. This is however resolved if the NumCounters is set the same as the maximum size of the cache(1000 objects).

cache, err := ristretto.NewCache(&ristretto.Config{
		NumCounters: 1000,           // number of keys to track frequency of (1000).
		MaxCost:     1000, // maximum cost of cache (1000 objects).
		BufferItems: 64,             // number of keys per Get buffer.
		Cost: func(value interface{})int64{
			return 1
		},
	})

The above allowed the cache to work as normal, with majority of my Sets succeeding and my hit ratio staying at 0.94 instead of rapidly dropping. Is there anything else that needs to be done to get the cache working with 10x the number of keys or is this a gap in the documentation ?

[Karl McGuire]

This is probably because of the TinyLFU admission policy rejecting new Sets in the first case, due to the incoming items not being deemed "valuable" enough. In the second case, the low number of counters is causing a high amount of collisions when TinyLFU is checking for incoming item value and most of them are getting through.

I'm not sure of your use case but in the first case you're probably only checking the initial Set. Right now, Ristretto expects you to call Set multiple times (with missed Get calls as well), this is the process of "building up item value" within TinyLFU. Eventually the Set will succeed when it's found to be a valuable item.

Could you post the full code you're using to check the hit ratio?

[Karl McGuire]

Just checking in to make sure this still isn't an issue?

[Nishant Das]

Hey @karlmcguire, sorry for the late reply, I must have missed this.

This is probably because of the TinyLFU admission policy rejecting new Sets in the first case, due to the incoming items not being deemed "valuable" enough.

In that case, in my situation each new object has a few sets. But I am guessing it doesn't have enough new sets to be valuable enough so that it is admitted to the cache. So once the cache is full basically no new object can be admitted in, as not enough set/gets are called on it. However what about old objects in the cache that do not have anymore gets called on them ? They do not seem to be getting evicted. Would they only be evicted if a new item is 'valuable' enough to be admitted in ?