graphql-java / java-dataloader

The docs for Custom caches say:

You could choose to use one of the fancy cache implementations from Guava or Kaffeine and wrap it in a CacheMap wrapper ready for data loader. They can do fancy things like time eviction and efficient LRU caching.

I've been looking at Caffeine, and started writing a CacheMap wrapper for it, but I'm wondering about a couple of things. Does anyone have experience with using a Caffeine cache with their dataloader(s)?

Here's what I've got so far:

public final class CaffeineCacheMap<U, V> implements CacheMap<U, V> {
  private final Cache<U, V> cache = Caffeine.newBuilder().maximumSize(1000).build();
  
  @Override
  public boolean containsKey(U key) {
      return cache.getIfPresent(key) != null;
  }

  @Override
  public V get(U key) {
      return cache.getIfPresent(key);
  }

  @Override
  public CacheMap<U, V> set(U key, V value) {
      cache.put(key, value);
      return this;
  }

  @Override
  public CacheMap<U, V> delete(U key) {
      cache.invalidate(key);
      return this;
  }

  @Override
  public CacheMap<U, V> clear() {
      cache.invalidateAll();
      return this;
  }
}

and here's how I'm using this to build my DataLoaderOptions:

final CacheMap cacheMap = new CaffeineCacheMap<>();

final DataLoaderOptions options = DataLoaderOptions.newOptions().setCacheMap(cacheMap);

My questions so far are:

Is this right?

  public boolean containsKey(U key) {
      return cache.getIfPresent(key) != null;
  }

Caffeine's Cache doesn't provide a "contains key" method, but what about fields with nullable values? Does this mean they can't be cached using the Caffeine cache, since there's no way to distinguish between a cached null value and a cache miss?

CacheMap has type parameters U and V, which I've added to my CaffeineCacheMap implementation as well. But I'm not sure what values to use for these;DataLoader explicitly casts the cacheMap to a CacheMap<Object, CompletableFuture<V>>:

java-dataloader/src/main/java/org/dataloader/DataLoader.java

Line 344 in 1ab3210

return loaderOptions.cacheMap().isPresent() ? (CacheMap<Object, CompletableFuture<V>>) loaderOptions.cacheMap().get() : CacheMap.simpleMap();

Should I do the same, where V comes from my batch loader's value type?

fwiw, Caffeine offers an asMap() view. While it does not support null values, a negative cache can be implemented by using Optional<V>.

Thanks Ben, appreciate it, and looking forward to trying out Caffeine!

After playing around with it a bit, I realized that the value that the data loader puts in the cache isn't the actual value V, it's a CompletableFuture<V>. Thus if cache.getIfPresent(key) == null, we can safely assume that the value doesn't exist in the cache. If a batch loader returns a null value, that's stored in the cache as a CompletableFuture with a null result.

Oh, interesting. We do also have an AsyncCache which is equivalent to Map<K, CompletableFuture<V>>. However, it automatically discards the entry if it fails or computes to null.

A benefit of cache.asMap().contains(key) is that it doesn't affect the metadata, e.g. frequency / recency, reset expiration, hit rate. Whether that matters though depends on your configuration.

Yeah in this case, we do need the ability to cache a CompletableFuture whose result is null, because some fields are nullable and we don't want to fetch them twice.

The reason I opened this issue is that I'm trying to combine two OSS libraries - this one and caffeine - and I'm not an expert on either one (in fact, I've never used Caffeine before), nor does there appear to be any examples online of doing this. I think it's relatively straightforward to wrap a manually-loaded Caffeine cache with a CacheMap, but I would love to get confirmation from someone like @bbakerman that I'm not missing anything.

I just found #45, which answers my second question about which types to use. Here's what I ended up with, in case it helps anyone else:

public final class CaffeineCacheMap<V> implements CacheMap<Object, CompletableFuture<V>> {

  private final Cache<Object, CompletableFuture<V>> cache;

  ...
}

With both of my questions answered, I'm closing this issue.

Oh and FWIW, here's what the final version of my CacheMap looks like:

import com.github.benmanes.caffeine.cache.Cache;
import com.github.benmanes.caffeine.cache.Caffeine;
import java.util.concurrent.CompletableFuture;
import org.dataloader.CacheMap;

/**
 * A simple CacheMap wrapper around a Caffeine cache, which uses manual loading.
 *
 * @param <V> – type parameter indicating the type of the data that is cached
 */
public final class CaffeineCacheMap<V> implements CacheMap<Object, CompletableFuture<V>> {

  private final Cache<Object, CompletableFuture<V>> cache;

  public CaffeineCacheMap(int cacheSize) {
    this.cache = Caffeine.newBuilder().maximumSize(cacheSize).build();
  }

  @Override
  public boolean containsKey(Object key) {
    return cache.asMap().containsKey(key);
  }

  @Override
  public CompletableFuture<V> get(Object key) {
    return cache.getIfPresent(key);
  }

  @Override
  public CacheMap<Object, CompletableFuture<V>> set(Object key, CompletableFuture<V> value) {
    cache.put(key, value);
    return this;
  }

  @Override
  public CacheMap<Object, CompletableFuture<V>> delete(Object key) {
    cache.invalidate(key);
    return this;
  }

  @Override
  public CacheMap<Object, CompletableFuture<V>> clear() {
    cache.invalidateAll();
    return this;
  }
}

How to wrap Caffeine with CacheMap?