Type errors trying to create a client middleware

[Matt de Young]

Hello, I am trying to great a generic middleware that can be used to add client side caching.

So far I have the current example:

import { ClientMiddleware } from 'nice-grpc';
import NodeCache from 'node-cache';
import objectHash from 'object-hash';
import logger from '@ivy/logger';

/**
 * These options are added to `CallOptions` by
 * `nice-grpc-client-middleware-cache`.
 */
export type CacheOptions = {
  /**
   * Max age of cached responses that will be returned.
   *
   * Defaults to 600.
   *
   * Example: if `cacheMaxAge` is 120, and a 300 second old response exists
   * in the cache, it will not be returned.
   */
  cacheMaxAge?: number;
  /**
   * Max number of responses that will be cached.
   *
   * Defaults to 1000.
   */
  cacheMaxKeys?: number;
};

const responseCache = new NodeCache({ stdTTL: 600, maxKeys: 1200 });

// TODO: benchmark this to see if this hashing is dummy slow
const requestKey = (path: string, request: Request): string => `${path}/${objectHash.sha1(request)}`;

/**
 * Client middleware that adds automatic caching to unary calls.
 */
export const cacheMiddleware: ClientMiddleware<CacheOptions> = async function* cacheMiddleware(call, options) {
  // We don't cache streaming responses
  if (call.method.requestStream) return yield* call.next(call.request, options);

  const key = requestKey(call.method.path, call.request as Request);
  logger.debug(`Checking cache for ${key} newer than ${options.cacheMaxAge}`);

  // TODO: check if response with max age exists in cache
  const cachedResponse: Awaited<Response> | undefined = responseCache.get(key);
  if (cachedResponse) {
    logger.debug(`Cache hit on ${key}`);
    return cachedResponse as any;
  }
  logger.debug(`Cache miss on ${key}`);

  const response = yield* call.next(call.request, options);

  // TODO: yield response & then update cache?

  try {
    // TODO: if at maxKeys, replace the oldest response with this one
    responseCache.set(key, response);
  } catch (e) {
    logger.warn(e, 'Unable to store response in cache.');
  }
  return response;
};

This works, but relies on type assertions on lines 40 & 47 for reasons that I can't really figure out. I am using typescript 5.1.6 & nice-grpc2.1.5

When using this in the client, the only way I can get this to work is by type assertions again:

const client = createClientFactory()
  .use(clientLoggingMiddleware)
  .use(retryMiddleware)
  .use(cacheMiddleware)
  .create(ServiceDefinition, channel, {
    '*': {
      cacheMaxAge: 600,
    } as CallOptions,
  });

with a modified CallOptions interface. The documentation seems to suggest that this shouldn't be necessary, but I can't really work out how this is supposed to work otherwise.

I am not exactly a deep Typescript expert so I am just looking for some guidance on how I can avoid the gross as any, I am probably just missing something here. I can also put together a minimal example project if that would help.

Thanks.

[Daniel Lytkin]

Hey,

You should add generic type parameters and annotate argument types for the async function* cacheMiddleware generator. Like this:

export const cacheMiddleware: ClientMiddleware<CacheOptions> =
  async function* cacheMiddleware<Request, Response>(
    call: ClientMiddlewareCall<Request, Response>,
    options: CallOptions & CacheOptions,
  ) {

Also in the requestKey function you have request: Request where the Request type is part of the fetch API. But inside the middleware the Request stands for a generic type parameter. To fix that you can set request: unknown or even move the hash calculation inside the middleware function.

[Matt de Young]

Thanks for the help I really appreciate it.

Are you saying that the call in the client code should look something like this:

import { CallOptions } from 'nice-grpc';
import { CacheOptions } from '@ivy/nice-grpc-client-middleware-cache';

client.rpc({}, { cacheMaxAge: 120 } as CallOptions & CacheOptions);

Maybe with a reusable intersection type for CallOptions & CacheOptions or should this be handled by adding the typing to the middleware & I am just missing something?

[Daniel Lytkin]

In your case, the client will have type Client<ServiceDefinition, CacheOptions>, so the extra options added by middleware are embedded in its type. You should be able to call it like this:

client.rpc({}, { cacheMaxAge: 120 });

Type casts are almost always a wrong thing, even if it's not any.

[Matt de Young]

Got it, I see the problem now. I was creating the client with let client: Promise<PricingServiceClient>;

Thanks for the help, and I agree about lying to the type system which is why I opened this issue.