Feature Request: Add an AsyncSeq overload for ScanAsync

[Nat Elkins]

I have a use case where I may need to scan a table for millions of items. I am unable to allocate an array that large. What I really want is to be able to operate on it like a stream. Would you accept a PR adding an overload that returns an AsyncSeq instead of an array? It would require taking a dependency on https://github.com/fsprojects/FSharp.Control.AsyncSeq.

Thank you for all your hard work!

[Avi Avni]

I think that getting millions of items need to be done with paging so why use AsyncSeq and not create function that get page size and callback function?

[Sam Ritchie]

I made a start on paging support a year or so ago but never finished. It seems a more flexible approach as you’d often want to paginate data via an API or a user interface. I can have another go at this if it seems useful?

You could certainly then wrap a paginated query in AsyncSeq (and I think this is worthwhile for admin-type scripts).

[Sam Ritchie]

See @bartelink’s comment here: #16 (comment) - I think there’s value in implementing scans & queries using TaskSeq, it may need some thought around the API design. I wouldn’t be fundamentally opposed to returning a TaskSeq as standard from all unbounded scans/queries