Naming things - unwrapping Eithers in boundary-break: .value or .?
adamw opened this issue · comments
Recent ox releases include a boundary-break implementation specialised to Eithers, which allows you to write "unwrap" an Either within a specified boundary, for example:
def lookupUser(id1: Int): Either[String, User] = ???
def lookupOrganization(id2: Int): Either[String, Organization] = ???
val result: Either[String, Assignment] = either:
val user = lookupUser(1).value
val org = lookupOrganization(2).value
Assignment(user, org)Here, .value is used for the "unwrapping". However, one alternative would be to use the Rust-inspired .?. On the one hand, @lbialy argues that .value is clash-prone, along with @Ichoran who says .value suggests that it's a safe accessor.
On the other, symbolic operators historically didn't work out that well, plus, as @alexandru points out, it looks awkward and doesn't play well with fewer-braces.
What's your take? Vote with a 👍 on a comment with your pick, or propose your own!
.value
.?
Disclaimer: Bikeshed ahead! (My opinions about this are not that strong)
I'm not a fan of neither .? not .value, as they both seem to imply that the left is "wrong", an this is not necessarily the case.
While Either is right biased, which makes it convenient to have an error channel on the left and a happy path on the right, Either is just a tagged union. It doesn't carry the semantic weight of something like Try, Result or Validated.
I don't have any good suggestions, though... Maybe something like .rightValue/.leftValue?
@JD557 in general, yes, either is just a tagged union of equivalent types; but in terms of boundary-break, and the way it's used for error handling in ox we're taking the right-biased interpretation.
I'd also add .unwrap for your consideration.
Uh, the worst part of Rust api :(
@lbialy So it's the worst part, or is ? ok? ;)
Personally, I've been using .? for a long time in my own code--I do use Rust as well, so that was my inspiration to create the feature (at first as a Scala 2 macro that mostly worked)--and I've found it very pleasant.
I haven't often run into cases where Alexandru's critique would apply because .? works best (both in performance and ability to reason about behavior) when local and in a flat style, not deeply nested monadic stuff. I have lots of code that looks like
def loadTemplate(): Source Or Err = Or.Ret:
"template.qmd".path.slurp.?.tap(lines => parse(lines).?)(The Or stuff is a success-unboxed Either clone; Or.Ret: is my version of either: here.)
The point of using .? or whatever you call it is to get your error cases out of your hair fast and where they occur. And .? is quite visually distinctive. The only way I've made it even more obvious is calling it .GET or somesuch and I don't like the shouting.
If you don't have a favored branch and a disfavored branch but rather two co-equal branches, you should not have a single accessor but something like
extension [L, R](either: Either[L, R]) {
/** Exit to boundary matching the left branch, or keep going with the right branch's value. */
inline def rightOrBreak(using Label[L]): R = either match
case Right(r) => r
case Left(l) => boundary.break(l)
/** Exit to boundary matching the right branch, or keep going with the left branch's value. */
inline def leftOrBreak(using Label[R]): L = either match
case Left(l) => l
case Right(r) => boundary.break(r)
}However, even though I wrote them, I never use these, or variants that have Either[L, R] as the label, in practice. It's confusing. Dropping out early with a disfavored or default value is cool. Jumping helter-skelter whenever computation is finished from multiple points is not cool.
I'm not voting here because I've already made my own choice with my own library, but I wanted to share some experiences.
This is also something Martin seems to be keen to introduce to the language, given his keynote from 2023. I don't really understand @alexandru's comment about .? not fitting fewer braces syntax (which I personally don't like because I like the distinction between "just a separate scope" like class x: or object y: or def fun: Unit = and "lambda-passed-as-argument body" like map { x =>) because that's exactly the context in which Martin first proposed question mark operator publicly 🤔
A weekend though - since we are considering symbolic operators, maybe we're incorrectly fixated on ? which comes from Rust. In Java, this is used for the elvis operator; in Kotlin, for optionally dereferencing nulls. We can imagine a similar operator being used for explicit-nulls in Scala.
So ... maybe e.g. .! would actually be better? This might indicate quite strongly that it's a short-circuiting operation, disrupting the normal control flow.
This is also something Martin seems to be keen to introduce to the language, given his keynote from 2023. I don't really understand @alexandru's comment about .? not fitting fewer braces syntax (which I personally don't like because I like the distinction between "just a separate scope" like class x: or object y: or def fun: Unit = and "lambda-passed-as-argument body" like map { x =>) because that's exactly the context in which Martin first proposed question mark operator publicly 🤔
The fewer braces syntax is here to stay, and, unless it gets deprecated somehow, will probably become the dominant style in Scala 3, once backwards compatibility with Scala 2.x is no longer a concern. This syntax isn't experimental, and there's no in-between. Another outcome would be for Scala to have multiple syntaxes, forever, but that's not very Pythonic, and it's not a very likely outcome, due to conformity nowadays being forced by Scalafmt. Although, a Klingon language option would be fun.
This is valid syntax, showing method chaining in action, and it's less than ideal for the obvious reason that .? is barely visible, looking like someone's cat briefly stepped over the keyboard:
validate(user).flatMap: user =>
fetchRepository(user)
.?
// ...also this...
validate(user)
.flatMap: user =>
fetchRepository(user)
.?I've always disliked special operators. I think all special operators in Scala-land suck, making the language actually harder to read. I'm not fond of operators such as <+>, >>, or *> from Cats, or <*> from Scalaz, either. The reasons would be:
- you don't have a clear name for it that you can communicate verbally, unless you learn it, and then it's something silly like the “tie-fighter operator”;
- it looks like noise;
- I hate infix or postfix method notations, too;
- no matters what the authors think about it, the operation is rarely that special as to warrant the use of a special operator.
The argument that Martin may want to make .? official is a good one if it comes to pass. But then Martin says in Lean Scala that we should drop DSLs and use the language instead. So better make sure that .? will actually become official, and also provide an alternative for those of us that will hate it anyway (which is what Cats does).
Something to think about 🙂
.get ? Like for options, we know already it means something special!
I don't recommend .get because it doesn't really scream "short-circuit". The key for this to work is for it to be obvious. Nonlocal control flow requires attention.
.! is fine too, I suppose, but ! in Scala has been used for messaging actors, which is less like local control flow than is elvis or the nullable-type-as-option operation. I actually used it myself for an orthogonal control flow operation where you can have different attempts; .! is used to try to get something for which a failure means you should move to the next strategy:
val m = Map("a" -> 1, "b" -> 2)
val number = attempt:
safe{ "w".toInt }.!
.attempt:
m.get("c").!
.default:
0It's...okay. I find .? stands out better.
.ok should be fine. Not sure why the parser wouldn't like it? But I'm not sure it stands out enough.
This is valid syntax, showing method chaining in action, and it's less than ideal for the obvious reason that
.?is barely visible, looking like someone's cat briefly stepped over the keyboard:
Honestly, that's super-duper obvious to me. How do you not see it?
I mean, sure, before you're used to it there is the whole, "Hey, kitty, what are you doing??" thing. But it's not invisible--it's the only thing on the line!
It's the in-line usages that might be easier to overlook:
validated(readTable(t).?, readOptions(o).?).? ++ getDefaults().?So better make sure that
.?will actually become official, and also provide an alternative for those of us that will hate it anyway
Having a non-symbolic alternative is good practice anyway. In kse3, I used
extension [X, Y](or: X Or Y)
/** Exit to boundary matching the disfavored branch, or keep going with the favored branch's value. Like `.?` but unwraps Alt */
inline def getOrBreak(using Label[Y]): X = or.fold{ x => x }{ y => boundary.break(y) }which is almost the same save .? does not unwrap the Alt and getOrBreak does, but maybe that's a mistake on my part and the canonical way should just be to use .? as a symbolic alternative to .getOrBreak.
Either way, I think getOrBreak is the natural linguistic parallel to getOrElse, which is the other way to handle the disfavored case.
Oh, that's right, I misunderstood before. It would have to be .ok_?, which is clunky to type.
Another idea is using .please. In addition to your code looking very polite, "please" suggests that there is some doubt about whether it's going to happen, but you asked nicely, so if it isn't going to happen the alternative should be nice too (which jumping out with the disfavored value is).
Maybe ok without a question mark?
validated(readTable(t).ok, readOptions(o).ok).ok ++ getDefaults().ok@odersky - I like the question mark better, but .ok is a reasonable contender. In the adverb form, it conveys the same sense of uncertainty as adverbal .please but it's a heck of a lot shorter.
Since we're talking about the direct style here, I feel that .please may not be direct enough.
How about something more persuasive, e.g. .now, as in "give me that value now!" ?
@ghik - The problem is that .now is useful for time-related methods. For instance, java.time.Instant.now. .ok and .please (and .?) have the advantage of being relatively unencumbered by other "obvious" uses.
I like the question mark better, but .ok is a reasonable contender. In the adverb form, it conveys the same sense of uncertainty as adverbal .please but it's a heck of a lot shorter.
@Ichoran unironically I have defined .pls() (and .pls_ok()) as traits (with Option/Result impls) in Rust to get my own nicer error reporting behavior. I think it looks okay :)
I do think that something like .? stands out more, but is a bit ugly with the dot. I agree that .ok is a good contender, my only gripe with it is that it's a bit hard to discover: .get is the more universal operator, and .ok doesn't really tell you what it's doing from the name (while vaguely being a name...)
Of course it's imo just bikeshedding, once we use it for a while I believe it'll become more familiar no matter what we choose.
On the other hand I would not go with .! because in many other languages (Kotlin with !!, Typescript with !, Swift with ! immediately comes to mind) it has .right.get's behavior.
Thank you for all the input :) In the end I'll go with .ok() and .fail(). As mentioned above, they are short and unique. .please() and .sorry() did receive a lot of love, but I think they would loose their charm quite quickly when used day-to-day :). We'll leave it for some more rarely used function ;)
Note that I think I'll use .ok() with parentheses, as this is not a field/property read, but might have control-flow effects (jumping to the boundary).
ok() opens a possibility of ok(inline f: X => L) as nicely syntactically compatible, because once you use this you will find that left-mapping prior to .ok() is rather a drag (and slows things down), but your types don't always match what you need.
Are these operators going to be implemented with an empty parameter list or without one at all? I think we had a convention that side effecting methods and functions should be called with an empty param list. I dislike this convention immensely personally and control flow change is a bit different from "having side effects" but if there's a consensus that it's a good thing for readability maybe we should retain it?
It's not clear to me what's the convention for the empty parameters list, e.g., foo(). For example, Iterator in Scala is something like this:
trait Iterator[+A]:
def hasNext: Boolean
def next(): A
The hasNext method is obviously side-effecting, and can also throw exceptions. And I once brought up this argument, and the answer at that time was that hasNext does not move the internal cursor, whereas next() does.
I think that beyond any useful conventions we can have, the technical difference between hasNext and next() is that hasNext can be overridden with a val (infinite streams, maybe?)
Just a clarification:
The hasNext method is obviously side-effecting
Have we agreed on precise definition of "side-effecting"? If you said "not referentially transparent" instead, it would be unambiguous.
(I really hope to not start a lengthy, academic-ish discussion about purity, RT and side effects here 😅)
@ghik I did some research on that, and I'm using the following definition:
a function has an effect, if apart from returning a value, changes observable behavior of the system
@alexandru as for the () ... not sure if Iterator follows the current best practices, but I always understood the convention that any method with effects should have (), as otherwise it looks like a simple property accessor. But my understanding might be totally wrong from somebody else's perspective, I don't think it's codified anywhere.
There was a discussion about this on Scala User just recently. There's Seth Tissue gave a link to an earlier discussion: https://users.scala-lang.org/t/paramterless-functions-with-and-without-parentheses-different/9939/12?u=odersky
@odersky Thanks! There's a couple of approaches proposed there, to include () if:
- the target object is mutated
- if there are side-effects
- if the call couldn't have been a mutable field
I guess it's moot to discuss Iterator, but for .ok(), rules (2) and (3) apply, so I still think that it's warranted.
It would be nice to have a more precise definition of a test for "no side effects", similar to how referential transparency is often used as a test for "purity". For example:
function
fhas no side effects iff removing an invocation offwhose result can be discarded does not change observable behavior of the system.
For example:
val it = Iterator(1,2,3)
it.hasNext // removing this invocation does not change observable behavior
println(it.next())This of course includes all pure (RT) functions, but also things like .size, .iterator and .hasNext (assuming they don't throw exceptions).
This is pretty much the same as @adamw definition, but it provides a more tangible method to prove that a function is effect-free.
hasNext is not necessarily just a memory read, as it can do I/O, like querying a database, or reading from a Kafka-powered network socket. hasNext obviously mutates the internal state of the object often and its behavior can be clearly non-deterministic, since the outcome can depend on planetary alignments.
@ghik in your example, doing next() without hasNext is violating the Iterator's protocol. Actually, not checking the returned value of hasNext is violating the iterator's protocol, but what I'm saying is that next() can throw in case you're not calling hasNext, even if hasNext had returned true.
From my POV, it's hasNext that does the heavy lifting, not next() because by the time you call next(), you need to have the next value available already. Consider that an alternative encoding would be the one from .NET, which they define like this:
trait IEnumerator[+A]:
def moveNext(): Boolean
val current: AIn this case, current is indeed just a memory read, which is why they describe it with a getter. But moveNext() serves the same purpose as hasNext in Scala/Java. There are pros and cons to both approaches, but in both cases, it should be pretty clear that some serious side effects can happen in hasNext and moveNext().
I'm just saying that the current convention isn't well established, and the only meaningful difference is that without parens, you can override it with a val.
Having parens in .ok() is fine by me, since that call may trigger some actual side effects, and it may also attract some attention.
It's a murky area, since the only well-defined criterion is referential transparency, but we (collectively) hesitate to enforce it strictly.
Specifically for ok I would prefer to omit parens because they look too noisy to me in this setting. As a justification we can look at the intended meaning of ok. It should just get the underlying value if it is defined, and propagate the error if it is not. Propagating the error is an admittedly effect but it's a bounded.
I realize these are not very strong arguments.
I just wanted to leave an encouragement to try out .ok-style jumps for lightweight JVM 21 virtual thread based futures!
The patterns you get aren't always the most efficient, but boy are they easy to write!
You hardly even need to make futures monadic at that point.
If Fu( f ) runs f as a future (which of course might fail), then Fu( f ).map(x => g(x)) will run f and then g on the result if/when the first one succeeds. But so does Fu( g( Fu(f).ok ) )! You do have to be very careful about jumping boundaries, but "eh whatever, we failed, just let whoever is running us deal with it" is a powerfully simplifying concept.
@Ichoran the question is, is .map at all useful in "direct-style"? Although for futures, we have a similar method: Fork.get. It's simpler there because the error channel is fixed to be an exception.
I'm not sure if it's useful for more than familiarity. I have yet to figure out a substantial value-add to provide with it. Reading left-to-right is an advantage, but you can always Fu( Fu(f).ok pipe g ), and in some ways hiding the concurrency inside a map makes it require more thought.
I guess if you want it exactly the same as map it has to be Fu(f).pipe{x => Fu(g(x.ok))}, which is a little bit of a mouthful. So I guess the advantage of map is that if you really do want sequential operations, and you really do want your thread to queue up the work in the order in which it happens, map makes that easy. Otherwise--even if it's not very consequential in most cases--you might get the thread for the result started before the thread for the inner computation, etc..
@Ichoran I think this example would rather be: fork { f.get.pipe(g) } (or using your naming: Fu(f.get.pipe(g))), if you want to run the transformation asynchronously.
So Fork.get jumps to a boundary that accepts Left[Exception, Any]? Or it just throws an exception? I don't see it in main.
Anyway, what I was advocating for was that .ok on Fork would be .joinEither.ok, and that Fork.apply would have the same boundary as either:'s left case.
In my library I can do things like:
val lastnames = Fu:
val lines = p.slurp.?
lines.map(q => Fu(db.ask(q).?.lastname)).map(_.?)and it all just works: in a thread I slurp a file, terminate early with any error, then take each line as a lookup, etc.
There isn't any distinction between normal error handling and future error handling, not in the least because, in ox notation,
either:
fork:
unsafeOperation.okis a bug.
Sorry, Fork.join. And yes, it throws exceptions in the fork is failed. There's also a .joinEither.
So in what you propose, Fork creates a boundary allowing you to report Throwable errors? Indeed either: fork: unsafeOp.ok is a bug, as that's an illegal capture of the boundary. But if the fork was joined, that would be fine.
In your lastnames example, where do you join the futures/forks?
Maybe the difference comes from the fact that in Ox we don't try to interfere in any way with exceptions - if they're thrown, we just do the appropriate cleanup to make sure there are no leaks. But they are mostly considered "panics". If you want typed errors, you should then use Either & boundaries. I think I'd need a more detailed explanation of your approach to fully understand it, though :)
Well, I'm not sure this is exactly the right place to discuss it, but I'll try to explain my reasoning. (It's also present to an extent in the kse docs here and here.)
I'm going to re-explain things that I think you already know so that I don't wrongly assume some part which ends up leaving examples confusing.
The starting premise is that we have a good means of handling errors in direct style. I wasn't satisfied with Either, so I created an unboxed clone of Either called Or. And I wasn't satisfied by Throwable's performance, so I created an Err type that can wrap String or Throwable. It reads nicely: operations that might fail look like User Or Err.
Furthermore, I favor direct-style jumps to handle error conditions, for which I use .? so I can go back and forth between Scala and Rust with less cognitive burden. It looks like this:
def loadAdmin(path: Path): User Or Err =
Err.Or: // This is the boundary
val lines = path.slurp.? // slurp might fail; jump to boundary
val user = User.parse(lines).? // parse also might fail
user.administrator // Let's suppose that this can't fail because fallback is the user administers themselvesNow, given that, what are the key type signatures of a future? We need to be able to start a computation that in general might return something, so that's (() => A) => Future[A], conceptually. And we need to eventually get an answer, which might fail, which is Future[A] => A Or Err conceptually. (I typically call this kind of non-guaranteed operation ask.)
The first unification of concepts is that with virtual threads, you don't worry about blocking. There's no reason to keep typing { future.join; future.ask() } over and over again. ask() itself should block.
The second unification of concepts is that future.ask().? is usually what you want anyway--in direct style you go after the value and let the error take care of itself as much as possible--so just make that future.?.
And the third unification of concepts is that if something fails within a thread, you probably want to just terminate with that error, so rather than writing
// Use of future is pointless save for being illustrative
def example(): Bar =
Err.Or:
val future = Fu:
Err.Or:
unsafeOp.? + safeOp
val value = future.?.? // Once to unwrap Future, once to unwrap error inside
value.bar
.getOrElse(Bar.default)you should just re-use the future's own I-might-fail-because-I'm-a-concurrent-operation capability with the contents' I-might-fail-because-reasons. If you want to distinguish them, you can pack that information into the error type.
But if you're re-using the error capability anyway, every thread boundary is an error-handling boundary. So why not empower the user to use it that way?
Thus:
def example(): Bar =
Err.Or:
val future = Fu:
unsafeOp.? + safeOp
val value = future.?
value.bar
.getOrElse(Bar.default)This makes everything both highly convenient and highly safe.
Since you seem to be taking a rather similar attitude towards at least some of how ox works, I wanted to suggest that you consider a similar pattern.
The open question I have is whether it is still too easy to accidentally have things cut across thread boundaries.
After several hours of work with Ox (on streams where I build scala.today) I have to say that I concur with many of the remarks above. Either handling being somewhat separate and parallel to ox's concurrency stuff leads to painful pitfalls. A big chunk of said issues come from the fact that Break is just a RuntimeException and stuff gets captured when it shouldn't. forks capturing Breaks is just one of the problems in this area - any try/catch with wide enough catch captures Breaks, scala.util.Try does that also and for things like that capture calculus won't help. Then there are aforementioned cross-overs between ox.either and supervised like fork* being able to capture Label[Either[_, _]]. To my surprise this sometimes work just fine, like with mapPar where the combinator rethrows exceptions and gives great usability because suddenly either { x.mapPar(10)(_.doStuff.ok()) } just works. It would be wonderful to rethink the api of supervised/fork methods to either make captures illegal. What's very nice is that with structured concurrency scoping rules intersect with implicit scopes and therefore a simple NotGiven can help a lot. For example:
I also have a gripe with nested either blocks as they can easily hide issues:
@lbialy - My kse3 library handles those issues by discarding Try and adding a .catchable extension on Throwable. Instead of Try, I just have a safe method that catches exceptions and packs them into an A Or Throwable.
So, yes, all of what you said. kse3 fixes it; ox can too, but probably has to make similar tradeoffs.
Even then, though, there's always a danger of
val outer: Int or Boolean = Or.Ret:
val inner = Fu:
val thing: String Or Boolean = Alt(true)
thing.?.toInt
.ask().getOrElse(_ => false)The thread running Fu intercepts the boundary jump for the Alt(true), generates an error due to the exception, and we get an Alt(false) instead of an Alt(true) like we should.
This isn't extremely hard to avoid if you keep methods small and logic clean, but if you were using a monadic style instead of a direct style, it would be a pile of type errors that need to be fixed with monad transformers and such instead of a runtime error.
Or you can just prevent capture of .? on forks. With #146 this:
val errOrInt: Either[Throwable, Int] = Right(23)
either:
supervised:
fork:
errOrInt.ok()
no longer compiles.
@Ichoran great explanation, thank you! That should almost be a blog, would be a great entry to ScalaTimes :)
One thing I don't understand:
val outer: Int or Boolean = Or.Ret:
val inner = Fu:
val thing: String Or Boolean = Alt(true)
thing.?.toInt
.ask().getOrElse(_ => false)
Where is the boundary interception happening? Is Alt anything more than a constructor?
But otherwise, I'm almost convinced that fork should also create a Either[Throwable, T] boundary. One thing I'm not sure about is whether to keep the two error modes: untypes (exceptions) & typed (logical errors, represented as Eithers), separate, as it is now, or somehow combined, as with fork-creating-a-boundary. Do we want at all to encourage people to work with Either[Throwable, T] types? I can already see the dillemas: when to throw an exception, when to return Either[Throwable, T]. And it's a good one! :)
@adamw - The Alt(true) is just creating the disfavored value; Alt is isomorphic to Left; Is corresponds to Right.
The boundary interaction happens at thing.?. It's supposed to return a String, but it can't, so it jumps with the disfavored value, the Alt(true). This tries to take it across the Fu: thread boundary, which would take an Alt[Err] but not an Alt[Boolean], up to the Or.Ret: boundary. But, of course, the Fu thread is running on its own, so there's nowhere to jump to; the result ends up as an exception. ask() gives that exception as an Alt[Err], and then getOrElse has to take the or-else branch, which sets the value to false.
Which, incidentally, is a bug. The return type would be typed as Int | Boolean as I wrote it. It should have been mapAlt(_ => false).?, or Or.FlatRet: with no .? jump.
I should always paste my code into the REPL 😆
A small change, but one thing that might help when working with typed errors & forks is the ability to join & unwrap in a single call, so given a f: Fork[Either[E, A]], in #155 we now have f.ok() (which must be run in an either: block.
Providing any special handling for exceptions with forks doesn't make that much sense, as any exception thrown in a fork will be end the scope (if it's supervised - which is the default). So in a supervised scope, Fork.join() will either yield a result, or throw an InterruptedException.
For more utilities & feedback (thanks @lbialy & @Ichoran, very valuable!) let's use the other PRs & new issues.
(btw. - if you would be willing to publish some of the scaladocs from kse3 as a blog post, I think many people would be interested and benefit from it)