RecordPersister's 'Expiration' is not working

[Gaurav Mittal]

StoreBuilder.<BarCode, BufferedSource, List<ListDto>>parsedWithKey()
        .fetcher(barCode -> responseConverter(
            serverRestClient.getListsSingleForStore()))
        .memoryPolicy(getMemoryPolicy())
        .persister(getDataPersister(context))
        .parser(GsonParserFactory.createSourceParser(gson, new TypeToken<List<ListDto>>() {
        }.getType()))
        .open();

private Persister getDataPersister(Context context) {
    try {
      return RecordPersister.create(getFileSystem(context), 1, TimeUnit.MINUTES);
    } catch (IOException e) {
      e.printStackTrace();
    }
    return null;
  }

 private MemoryPolicy getMemoryPolicy() {
    return MemoryPolicy
        .builder()
        .setExpireAfterWrite(1)
        .setExpireAfterTimeUnit(TimeUnit.DAYS)
        .build();
  }

Data is fetched from the disk(not from network) even after the expiation time(here 1 minute).
Note : I am force killing the application and re-launching, so memory data is not used.

I even tried adding

networkBeforeStale()

But no change.
I read everything I could find online around this. So far no progress.

*Fresh data from network is fetched after i do this
Device Settings -> Applictions -> My Application -> Clear Cache

[Yang]

What's in getMemoryPolicy()? Can you create a small project to show what you're trying to achieve?

[Gaurav Mittal]

@ychescale9 thanks for looking into this

private MemoryPolicy getMemoryPolicy() {
   return MemoryPolicy
       .builder()
       .setExpireAfterWrite(1)
       .setExpireAfterTimeUnit(TimeUnit.DAYS)
       .build();
 }

Should memory policy have any affect on this?
I am force killing the application and re-launching, so that the in-memory data is not used.

Also I can not create a a new project at the moment.

If it helps I can remove the memory policy and resubmit my new findings.

[Sven Bendel]

I am seeing exactly the same (using FileSystemPersister over here). Anyone having any idea what is going wrong here?

[Sven Bendel]

@gaurav-m I think I fixed that thing for me now by using FileSystemRecordPersister (I'm fetching some deal data in the example):

private val DEALS_EXPIRE_AFTER_WRITE = 1L
private val DEALS_EXPIRE_AFTER_WRITE_UNIT = TimeUnit.MINUTES

StoreBuilder.parsedWithKey<Int, BufferedSource, DealsResponse>()
            .fetcher {
                dealsDataSource.getDeals().subscribeOn(Schedulers.io())
            }
            .persister(
                FileSystemPersisterFactory.create(
                    FileSystemFactory.create(application.filesDir),
                    { generateKey(it) },
                    DEALS_EXPIRE_AFTER_WRITE,
                    DEALS_EXPIRE_AFTER_WRITE_UNIT
                )
            )
            .parser(GsonParserFactory.createSourceParser(gson, DealsResponse::class.java))
            .memoryPolicy(
                MemoryPolicy.builder()
                .setExpireAfterTimeUnit(DEALS_EXPIRE_AFTER_WRITE_UNIT)
                .setExpireAfterWrite(DEALS_EXPIRE_AFTER_WRITE)
                .build()
            )
            .networkBeforeStale() // not entirely sure whether this is actually needed
            .open()

@ychescale9 I'm not entirely sure whether the networkBeforeStale() call is actually needed, does it affect both memory cache AND persister cache?

[Yang]

@gaurav-m https://github.com/NYTimes/Store/blob/e4c21b23c80ba1c6a2f5c451066ed7adff3592f0/store/src/main/java/com/nytimes/android/external/store3/base/impl/RealInternalStore.java#L146

Looks like it only applies to persister, but once your memory cache becomes expired (defined by the MemoryPolicy) it will eventually hit the persister cache which is when the StalePolicy kicks in.

Sorry I'm not familiar with the filesystem based persisters so not sure if they have any behavior that might be "unexpected".

[Mike Nakhimovich]

Hello, I have trouble understanding your issue. A store should not return stale data if you are using networkBeforeStale and your persister properly returns Stale as the record state. We have tests covering this use case: https://github.com/NYTimes/Store/blob/e4c21b23c80ba1c6a2f5c451066ed7adff3592f0/filesystem/src/test/java/com/nytimes/android/external/fs3/StoreNetworkBeforeStaleTest.java

Could you mutate the tests to show what your issue is?

[Sven Bendel]

@digitalbuddha with FileSystemRecordPersister it all works perfectly fine (as outlined in my sample).

@gaurav-m can you check what type your RecordPersister.create() call returns and follow up here? I don't have access to the code for the next few days.

[Gaurav Mittal]

@ubuntudroid i'll check and get back. will do that in couple of days.

apologies for the delay. I stopped following this a while back.