Proposal: graceful shutdown improvements

[André Eriksson]

Summary

This proposal defines a more full-featured graceful shutdown process for Encore applications, that aims to (a) improve shutdown correctness while (b) reducing graceful shutdown durations, particularly for local development.

Background

Encore already provides graceful shutdown handling (see docs). However, experience has shown it is difficult to use.

For example, a common use case is for a service to coordinate shutting down associated resources (like a client to a third-party API) as soon as (but not earlier than) outstanding requests complete.

Proposed design

This proposal suggests adding a new package shutdown that provides information about
graceful shutdown behavior.

Services can opt in to being notified about shutdown behavior by implementing shutdown.Handler on its service struct (see below).

package shutdown // import "encore.dev/shutdown"

// Progress provides progress information about an ongoing graceful shutdown process.
//
// The process broadly consists of three phases:
//
// 1. Drain active handlers
//
// As soon as the graceful shutdown process is initiated, the service will stop accepting new
// incoming API calls and Pub/Sub messages. It will continue to process already running handlers
// until they complete (or the ForceCloseHandlers deadline is reached).
//
// This phase continues until all active handlers have completed or the ForceCloseHandlers deadline
// is reached, whichever happens first. The ActiveEndpoints, ActivePubSubMessages, and ActiveHandlers
// contexts provide insight into what handlers are still active.
//
// 2. Shut down infrastructure resources
//
// When all active handlers have completed, Encore will begin shutting down infrastructure resources.
// Encore closes all open database connections, cache connections.
//
// This phase continues until all infrastructure resources (and service structs that implement [Handler])
// have been closed or the ForceShutdown deadline is reached, whichever happens first.
//
// 3. Exit
//
// Once the previous phase has completed, the process will exit.
// The exit code is 0 if the graceful shutdown completed successfully (meaning all resources
// returned before the exit deadline), or 1 otherwise.
type Progress struct {
	// OutstandingRequests is canceled when the service is no longer processing any incoming API calls.
	OutstandingRequests context.Context

	// OutstandingPubSubMessages is canceled when the service is no longer processing any Pub/Sub messages.
	OutstandingPubSubMessages context.Context

	// RunningHandlers is canceled when the service is no longer actively processing any handlers,
	// which includes both incoming API calls and Pub/Sub messages.
	//
	// It is canceled as soon as both OutstandingRequests and OutstandingPubSubMessages have been canceled.
	RunningHandlers context.Context

	// ForceCloseHandlers is canceled when the graceful shutdown deadline is reached and it's time to
	// forcibly close active handlers (API endpoint handlers and Pub/Sub subscription handlers).
	//
	// When ForceCloseHandlers is closed, all outstanding request's contexts are canceled.
	//
	// It is canceled early if all active handlers are done.
	ForceCloseHandlers context.Context

	// ForceShutdown is closed when the graceful shutdown window has closed and it's time to
	// forcefully shut down.
	//
	// If the graceful shutdown window lapses before the cooperative shutdown is complete,
	// the ForceShutdown channel may be closed before ActiveHandlers is canceled.
	ForceShutdown context.Context
}

// Handler is the interface for resources that participate in the graceful shutdown process.
type Handler interface {
	// Shutdown is called by Encore when the graceful shutdown process is initiated.
	//
	// The provided Progress struct provides information about the graceful shutdown progress,
	// which can be used to determine at what point in time it's appropriate to close certain resources.
	//
	// For example, a service struct may want to wait for all incoming requests to complete
	// before it closes its client to a third-party service:
	//
	// 	func (s *MyService) Shutdown(p *shutdown.Progress) error {
	//		<-p.OutstandingRequests.Done()
	//		s.client.Close()
	//		return nil
	//	}
	//
	// The shutdown process is cooperative (to the extent it is possible),
	// and Encore will wait for all resources to be closed before exiting
	// unless the ForceShutdown deadline is reached.
	//
	// The return value of Shutdown is used to report shutdown errors only,
	// and has no effect on the shutdown process.
	Shutdown(*Progress) error
}

[Dominic Black]

I wonder if there's a usecase for knowing the "time remaining" from the various contexts, not just "is this cancelled or not".

For instance if you have a shutdown job you know takes a while, you might decide to defer it if you've not go enough time left.

[André Eriksson]

Yeah, but that's already available from ctx.Deadline().