extend "concurrency limit" functionality to support both batches and "continuous pooling"

[Kyle Simpson]

If you do:

FA.concurrent(5).map(someTask);

This could be interpreted as "I want task batches of maximum size 5 to execute concurrently (in parallel." But the question is, what happens when the first of those tasks completes? Does it immediately spin off another task, or does it wait for the batch to finish and then do another batch of size 5?

I could see either behavior being desirable, depending on the situation. So, we're going to handle it by letting you optionally specify a second number, a minimum-active threshold:

// FA.concurrent(5).map(someTask) --> means:
FA.concurrent(5,5).map(someTask);

This means, always try to keep 5 active tasks (as long as there are more tasks waiting in the pool. This is "continuous pooling".

But this:

FA.concurrent(5,3).map(someTask);

..means: "start a batch of 5, refill the batch once the active tasks count falls below 3."

And this:

FA.concurrent(5,1).map(someTask);

..means: "start a batch of 5, refill the batch once it's completely empty". In other words, this is standard "batch mode", whereas the previous (5,5) would be like continuously drawing in new active tasks from the pool.

The default will be that the minimum-active threshold defaults to the batch size (aka, (5) means (5,5) and (3) means (3,3)).