Filesystem driver cache

[Razvan]

I'm trying to write a file watcher using \Amp\File\getModificationTime and a custom listFilesRecursive.

This is what listFilesRecursive does:

use function Amp\call;
use function Amp\File\isDirectory;
use function Amp\File\listFiles;

use Amp\Promise;

function listFilesRecursive(string $path):Promise {
    return call(function() use ($path) {
        $items = yield listFiles($path);
        $files = [];
        foreach ($items as $item) {
            $filename = "$path/$item";
            $isDir    = yield isDirectory($filename);
            if ($isDir) {
                foreach (yield listFilesRecursive($filename) as $subItem) {
                    $files[] = $subItem;
                }
                continue;
            }

            $files[] = $filename;
        }
        return $files;
    });
}

I'm running \Amp\File\getModificationTime for each one of those files and everything repeats every X seconds.

The problem is that getModificationTime seems to return the wrong modification time when it runs more than once.

I noticed the filesystem driver is keeping a cache:

file/src/Driver/StatusCachingDriver.php

Lines 19 to 23 in 013f592

 public function __construct(Driver $driver) 
 { 
     $this->driver = $driver; 
     $this->statusCache = new Cache(1000, 1024); 
 } 

which is great for both performance and familiarity since the traditional filemtime also uses a cache.

In this case I don't really care if the \Amp\File\* functions are optimized for performance using that cache, I just care for their async nature, I want to avoid a bottleneck when the number of scanned files increases.

I noticed this method exists but it's private:

file/src/Driver/StatusCachingDriver.php

Lines 127 to 142 in 013f592

 private function invalidate(array $paths, Promise $promise): Promise 
 { 
     foreach ($paths as $path) { 
         $this->statusCache->delete($path); 
     } 
  
     if (!$promise instanceof Success) { 
         $promise->onResolve(function () use ($paths) { 
             foreach ($paths as $path) { 
                 $this->statusCache->delete($path); 
             } 
         }); 
     } 
  
     return $promise; 
 } 

Is there a way to manually invalidate this cache similar to how traditional php does it with clearstatcache?

[Niklas Keller]

Instead of exposing cache invalidation, I suggest using a custom Filesystem without the cache layer in between. If caching is desired per run, you could wrap your existing instance in a new StatusCachingDriver on each run.

[Razvan]

Awesome, it works!

StatusCachingDriver is a wrapper around the default driver with some extra caching logic, so using the default driver directly does what I need it to do.

The whole solution looks like this in my case:

$fs = new \Amp\File\Filesystem(\Amp\File\createDefaultDriver());
$mtime = yield $fs->getModificationTime($filename);