This repository contains the Depaginator. The Depaginator is a tool for traversing all items presented by a paginated API: all pages are retrieved by independent goroutines, then each item in each page is iterated over, calling a HandleItem
method. The index of the item in the list is also passed to HandleItem
for the benefit of ordering-sensitive applications.
For full details, refer to the documentation. The basic concept is for the consuming application to create an object that implements GetPage
, HandleItem
, and Done
methods, conforming to the depaginator.API
interface. The GetPage
method is passed a PageMeta
object and a PageRequest
, which bundles a PageIndex
integer with an application-defined Request
. The GetPage
method must then retrieve the desired page of the results, add any relevant metadata--including requests for subsequent pages--to the PageMeta
, and return a an array of items with the type used to call Depaginate
. The Depaginator
will then call HandleItem
for each element in the Page
. After all pages have been retrieved and all items handled, the consuming application calls Depaginator.Wait
, which will call the Done
method (passing it the final PageMeta
) and return any page retrieval errors that were encountered.
Many server APIs that return lists of objects will "paginate" the response to avoid overwhelming the connection or the client--or the server or database. However, many clients consuming that API need to perform some operation on all the returned objects, such as displaying them to the user or applying additional filters to select specific items. Especially for large lists, this process can be quite slow; this may be fine for user-interactive clients, such as command line clients, but if some other operation is being performed, such as bulk modifications, this can be unacceptably slow. The Depaginator
is intended to simplify the implementation of code that iterates over all the items in a list by allowing their retrieval as fast as the server API will permit.