Thoughts on v3

[Rob Landers]

Hello, I'm not sure the best place to put this... but I've been building a RAFT implementation using this library and the HTTP client/server, just for fun. I originally started with a pure fiber implementation to get to know fibers and then discovered the in-progress v3 branches when I realized just how much I'd have to implement from scratch. There's a few bits of feedback I think I'll leave here:

concurrent awaits

A pattern I've run into is where you want to send a number of HTTP requests in parallel to multiple servers. The code ends up looking something like this:

$requests = [];
foreach($servers as $server) $requests[] = async(doRequest(...), $arg1, $arg2);
foreach(Future::iterate($requests) as $next) $next->await();

I may have missed how to do this better, but it'd be great if this pattern could be simplified, maybe something like:

// asyncParallel makes the array's value the first parameter to the callback, illustrated below:
$results = asyncParallel($servers, fn($server, $arg1, $arg2) => doRequest($server, $arg1, $arg2), $arg1, $arg2)

delay/defer/repeat

There's already a delay() that is quite handy. async() appears to run before the current iteration of the event loop ends. It'd be nice to also have a defer() which starts on the next iteration. For example, when you know there is no point in running the callback during the same iteration, or you intentionally want to wait for a loop to let other things process. ie, semantics of EventLoop::defer()

Additionally, the only way to do something on a timer appears to be something like:

async(function() {
  while(true) {
    delay($time);
    doWork();
  }
});

or use the raw EventLoop. It'd be great to have a repeat() or schedule() function to take advantage of the EventLoop's implementation.

Cancellations

I love this concept you've implemented here and I've created a couple of classes. I'm happy to open a PR to add them to the library:

1. ManualCancellation: allows canceling manually instead of via timeout.
2. TrappedCancellation: traps an OS signal and only cancels on that signal.
3. ScopedCancellation: a bit of a hack, but cancels once the method/function that creates the cancellation returns.

Anything else?

I'd just like to say that I love the work y'all have done so far with fibers, it really changes the feeling of the code and makes it so much more readable. I really don't want to create any scope creep because the entire project is so huge (there's so much to do)! I just wanted to leave some feedback after trying out what I think amounts to a somewhat experimental branch. I really appreciate the work and time you've put into this library!

[Niklas Keller]

Hey @withinboredom, opening an issue is perfectly fine for giving feedback. In fact, that's exactly what we're looking for with the tagged beta releases!

concurrent awaits

Instead of using Future::iterate(), you can use the combinators in combination with array_map:

$requests = [];
foreach($servers as $server) $requests[] = async(doRequest(...), $arg1, $arg2);
foreach(Future::iterate($requests) as $next) $next->await();

$requests = Amp\Future\all(array_map(fn () => async(fn () => doRequest(...)), $servers));

I guess there could be another function to make this a little bit easier.

delay/defer/repeat

I think both of your issues are solved with using Revolt\EventLoop::defer() and Revolt\EventLoop::repeat() directly. I haven't felt the need for defer since we've introduced EventLoop::queue.

Cancellations

1. ManualCancellation sounds similar to our DeferredCancellation, previously CancellationTokenSource on the v2 versions.
2. TrappedCancellation sounds like a useful addition, though I'd probably name it SignalCancellation
3. Could you show an example on how your ScopedCancellation is used?

Don't worry about scope creep, more feedback and feature requests are very welcome, even if we might decide to not implement some ideas or defer them to later releases. Thanks!

[Rob Landers]

I think both of your issues are solved with using Revolt\EventLoop::defer() and Revolt\EventLoop::repeat() directly.

I've been trying to avoid using Revolt directly for a couple of reasons:

1. I'd have to pin Revolt as a direct dependency to my project, which may conflict with Amp later down the line.
2. Seeing that Amp has changed its loop architecture from v2 to v3, it may change again in the future, so I'm preferring to use higher-level abstractions as much as possible.

ManualCancellation sounds similar to our DeferredCancellation, previously CancellationTokenSource on the v2 versions

aha! The first time I looked at this class, I missed the getCancellation method, and saw it didn't extend the Cancellation type, so thought it was for something else!

Could you show an example on how your ScopedCancellation is used?

So, without getting into the weeds too much, RAFT requires several loops working in tandem (election timeouts, handling and sending RPCs, peer liveness, etc). Some of these loops have some fairly strict lifetimes (if the server is already a leader, you don't want it to time out and run an election). More commonly, I've found it most useful when I know there's a possibility for an unhandled exception to throw. You can just wrap the code in a try-finally, but nesting can get pretty intense, so I was looking for a solution that didn't involve nesting and boilerplate. So for example, instead of this:

function() {
  $cancellation = new DeferredCancellation();
  try {
    doWork($cancellation->getCancellation());
  } finally {
    $cancellation->cancel(); // cancel everything because we're leaving the function due to an unhandled exception
  }
}

you can write:

function() {
  $cancellation = new ScopedCancellation($_); // $_ is an out ref var that cancels once it loses scope via destructors
  doWork($cancellation);
}

As I said, it mostly just reduces nesting and boilerplate, but it also feels very hacky, even if useful.

you can use the combinators in combination with array_map

I was avoiding all() because it aborts on any failure, while Future::iterate() seems to not abort. I haven't actually tested this, since I handle errors in the callbacks, but aborting iteration on any error is undesirable in this case.

[Rob Landers]

Example implementations of the two cancellations here: https://github.com/amphp/amp/compare/v3...withinboredom:v3?expand=1

I obviously lack the amazing inline documentation skills seen throughout this project :)

[Bob Weinand]

Revolt is there to stay as long as there is Amp v3. Maybe with amp v4 there's something different, but so will be many amp-specific functions as well. It's recommended to rely on revolt itself.

On topic of the ScopedCancellation, I am wondering whether we should not just simply add a __destruct() to DeferredCancellation which calls cancel() (and provides a function skipCancelOnDestruct() or similar).
I feel like this should be the expected behaviour - It's seldom the case that you only want temporarily control the cancellation; in addition it alleviates manual finally handling which can easily be forgotten.

[Rob Landers]

Revolt is there to stay as long as there is Amp v3

That's good to hear!

I am wondering whether we should not just simply add a __destruct() to DeferredCancellation which calls cancel()

That's actually a pretty good idea and would work just as well; much simpler than my implementation too! I agree that it would make sense as the default as well.

[Aaron Piotrowski]

On topic of the ScopedCancellation, I am wondering whether we should not just simply add a __destruct() to DeferredCancellation which calls cancel()

I was thinking the same and pushed what I had locally in 8b06746. There's some complication because I wanted to avoid creating an exception object on the happy path, since making that backtrace is not without cost and I wanted to avoid it if it was never going to be used.

I guess there could be another function to make this a little bit easier.

What are everyone's thoughts on something like the following:

function Amp\concurrent(\Closure ...$tasks): array
{
    return Amp\Future\all(array_map(Amp\async(...), $tasks));
}

I was avoiding all() because it aborts on any failure

@withinboredom Have a look at Amp\Future\settle(), which does not return until all futures have resolved, regardless of outcome of each future.

Revolt is there to stay as long as there is Amp v3.

There is the potential for Revolt to have another major version which Amp could upgrade to without breaking Amp's API, yet would conflict in dependent projects. Therefore, it may be advantageous to provide a simple API for timers, which was the motivation for Interval which I removed before we tagged the first beta in 0f0d13c. Providing that would leave little reason for app authors to use Revolt directly, or am I forgetting something else we would need to add?

[Bob Weinand]

pushed what I had locally in 8b06746

Delayed exception generation in Cancellation looks pretty sane to me. I'm happy with this commit.

Have a look at Amp\Future\settle()

I feel like the Future functions do not really convey their true meaning in names, especially as we are trying to cover (one, some, all) for both of (successes, resolved).
all -> all success, else throw
settle -> all resolved
race -> first resolved
any -> first success
some -> some successes

maybe it should just be more systematic and obvious:

(first, some, all) -> (success, resolved), mapping to:

first(), firstResolved()
some(), someResolved()
all(), allResolved()

I believe this will massively increase immediate understanding of what they do, without much description, and improve discoverability. (When autocomplete proposes me settle or race I'm definitely not going to intuitively know what to expect.)

function Amp\concurrent

I've observed the pattern

Future\all([
    async(fn() => thing1()),
    async(fn() => thing2()),
    async(fn() => thing3()),
]);

to definitely happen in my code. I'm not opposed to this, I'm just not sure of whether it's worth it. Doesn't save that many chars writing:

Future\concurrent(
    fn() => thing1(),
    fn() => thing2(),
    fn() => thing3(),
);

In addition concurrent() is opinionated in that it decides what combinator to apply (in this case all)...

[Rob Landers]

(first, some, all) -> (success, resolved), mapping to

Maybe something like:

firstSuccess(), firstResolved()
someSuccess(), someResolved()
allSucceed(), allResolved()

or something? I feel like that would be more consistent in saying what they do. Just all() by itself (IMHO) just conveys they'll all complete, not necessarily we want them all to succeed. FWIW, I also feel that all() is a misnomer as it's really just some() in the not-happy path? IOW, I would expect all() to run all of them all the time and it's current implementation isn't intuitive. Now that I know that iterate aborts on a failure, I understand why I was seeing warnings about some http client requests not being processed when firing a few dozen at a time. It made my implementation subtly incorrect -- now fixed -- without it being obviously incorrect.

[Aaron Piotrowski]

The names for these functions are derived from JS Promises, so there is some assumed familiarity, though I would agree these names are not necessarily ideal.

The term "resolve" has largely been dropped in favor of "complete" – A future completes successfully with a value or errors with an exception. This mostly is due to futures not being completable with another future.

Therefore I would suggest the following:

settle() -> allCompleted()
all() -> allSuccessful()
some() -> someSuccessful()
any() -> firstSuccessful()
race() -> firstCompleted()
Add someCompleted()

[Niklas Keller]

I wanted to rename them anyway, to make clear they're awaiting and not returning a new future, so I'd like to add await into the names.

[Bob Weinand]

I was thinking the same, though I don't really want to have Future\awaitAllCompleted() … it's such a long name. :-|

Maybe there's some possible middle ground to find here?

[Aaron Piotrowski]

Verbosity is probably best. You can alias with use Amp\Future\awaitAllCompleted as aac; 😉

[Niklas Keller]

I've opened #383 if you have some feedback on those names.