Customizing errors

[Michael Hayes]

It would be great to have a way to customize errors. This could potentially be tied into #124, but having some sort of callback that returns (or throws?) errors would be super useful. Customizing the error message would be an improvement, but ideally I think throwing custom errors (so that extensions can be added) would be more useful.

[nullswan]

Do you think we need sort of global callback - or per plugin callback?

[Michael Hayes]

A global callback would make some things easier, but I imagine people will want to customize messages based in how things failed. For example for complexity errors, they might want to include the complexity and/or the limit. It would be hard to have a global callback that provides the right details about an error in a type-safe way that works for every potential error in a variety of plugins. I don't have a great solution here, but I can see a couple of options: a global callback that receives error instances, plugins throw sub-classes of a specific error. In a global handler you could do something like error => error instance of ComplexityError ? new Error('complexity limit: ${error.details.complexity}') : error obviously this is over simplified, but the idea is using specific error classes that are documented and exported from plugins to allow catching/replacing errors. The alternative is to do something on each plugin like onResult(details: PluginSpecificDetails, error: GraphQLError | null) => GraphQLError | null for each plugin. This pattern would allow you to combine the use case of #124 to have a callback for logging/analytics. Each plugin can surface captured information (like calculated complexity) regardless of if the plugin actually failed or not. This solves a lot of important use cases like tracking complexity for things like rate limiting, writing metrics, etc.

I didn't include this in the snippets above, but callbacks probably should also have access to the request context so that context can be used to access things like current user, request logger instances, or whatever else they might need.

[Laurin Quast]

Our main use case for having global (or per plugin) error handler is the original concern raised in #124 (observability). We want to ship graphql-armor to production but are still afraid we might have set our limits too conservatively for some operations that we don't control and thus break them after the rollout. We want to track what kind of operations are rejected via GraphQL armor in order to adjust our limits as required/desired.

Regarding the callback:

onError could write to a remote service and due to the function signature limitations of parse and validate where most of these plugins perform (cannot return a Promise ), onError could only be a non-async function, which makes stuff hard for serverless/edge environments as you would want to write the error to some async storage and wait until that is done. But this is nothing one could workaround, to be honest.

let queue = Promise.resolve();

const errorHandler = (errors: Array<GraphQLError>) => {
  const promise = reportErrors(JSON.stringif(errors));
  queue = queue.then(() => promise);
};

const envelop = getEnveloped({
  plugins: [
    maxTokenPlugin({ onError: error => errorHandler([error]) }),
    maxDepthPlugin({ onErrors: errors => errorhandler(errors) })
  ]
});

// << do graphql logic >>

// << finally >>

await queue;

// << tear down handler >>

Example using cloudflare workers:

addEventListener('fetch', event => {
  event.respondWith(handleRequest(event, event.request))
})

/**
 * Fetch and log a given request object
 * @param {Request} request
 */
async function handleRequest(event, request) {
  let queue = Promise.resolve();

  const errorHandler = (errors: Array<GraphQLError>) => {
    const promise = reportErrors(JSON.stringif(errors));
    queue = queue.then(() => promise);
  };

  const envelop = getEnveloped({
    plugins: [
      maxTokenPlugin({ onError: error => errorHandler([error]) }),
      maxDepthPlugin({ onErrors: errors => errorhandler(errors) })
    ]
  });

  // << Build response etc >>
  const response = await runGraphQLLogic(envelop)
  event.waitUntil(queue)

  return response
}

[Michael Hayes]

One thing I want to call out for the observability side of things is that it will be important to observe non-failure cases as well. For example, I want to be able to write metrics for complexity of queries so we can get max/average/percentiles for complexity, so we can initially set our complexity limits high, and tune them based in actual usage without having to recalculate the scores outside the plugin

[nullswan]

#206 will introduce custom callbacks per plugins along with ability to either throw or report error to the current context (#162)

type GraphQLArmorAcceptCallback = (ctx: ValidationContext, details: any) => void;
type GraphQLArmorRejectCallback = (ctx: ValidationContext, error: GraphQLError) => void;
type GraphQLArmorCallbackConfiguration = {
  onAccept?: GraphQLArmorAcceptCallback[];
  onReject?: GraphQLArmorRejectCallback[];
  throwOnRejection?: boolean;
};

For the moment, specific plugin details will be set to any. However, shared typing might appears very soon.

I just need to figure out how to exit parsing properly for Apollo.
But I think I'll stick with what graphql-js is doing which is throwing.

https://github.com/graphql/graphql-js/blob/main/src/language/parser.ts#L1638
https://github.com/graphql/graphql-js/blob/40ff40a21c710372330e65f0fb58f13c2df92a77/src/error/syntaxError.ts#L9