Pipeline

[ShalokShalom]

I understand your decision, to not use operator overloading.
To frame my following opposition to it, here my background:

I can not use FSharp in my current project, since it is barely implemented.

Python is an alternative. Something like Coconut is awesome, but its editor support is limited and I like good syntax highlighting.

After looking for a good hour through all kinds of Stackoverflow posts, PyPi, and several other sources to find at least a proper pipe, did I finally stumble over your project and was finally in Aww O.O

I finally found it, and then I read this:

Just to be honest: This is how F# code looks like:

I dont see any sense in a FSharp library for Python, that does not allow me to do that.

I can not emphasis enough, how important the pipeline operator is for me, its just fundamental to how FSharp feels and works for me. :)

Especially to place them into a new line, which is how idiomatic FSharp looks and feels.

Its also easier, to apply tutorials.

I understand that |> is probably not possible without recompiling and that overloading an already used operator is confusing, while I strongly suggest to adopt one that is not in broader use, or give the option to use | or >> anyway

I really see the point of avoiding confusion, while I dont work in a team - like most coders, especially in the open-source field - and reading into code based is always a struggle and this makes code at least more unambiguous and it's important to me, to be able to use FSharps traditional method.

It confuses me much more, to do it the way you implemented it.

Thanks a lot!

[Dag Brattli]

In F/ this would currently look like:

def expect_string(expected_string):
    return pipe(
        expected_string,
        string_to_char_list,
        List.map(expect_char),
        sequence_parsers,
        mapParser(char_list_as_string)
    )

So you are free to put each operator in the pipeline on separate lines. IMO not very different from:

let expectString expectedString = 
    expectedString
    |> stringToCharList
    |> List.map expectChar
    |> sequenceParsers
    |> mapParser charListAsString

This aligns well with e.g RxJS: https://rxjs.dev/guide/v6/pipeable-operators

[ShalokShalom]

Thanks a lot

[Gunther Klessinger]

Hi, I had a similar reaction when Dag's RxPython switched from "dotchaining" to the pipe function.

Meanwhile I love it and consider it a game changer.
Because Python has very good list support and those lists (of functions wrapped into operator functions), you can recursively throw into the pipe function (pipe(*my_operator_func_list)) - and that in a recursive process, because an operator function has the same interface as pipe itself.

details from my own working domain...

@ShalokShalom I was using Dag's RxPython (v2) for a few years, before it had the version with the pipe function in it. My first reaction to their version 3 was the same as yours. I was so in love with the "dot chained" operator pipelines, that I hated the idea to have to write all in plain old python functions and put stupid boring commas between the operators. I'm not the brightest guy unfortunately, so I did not immediately see the obvious - found that only after I forced myself to work with it. Finding was:

It's a game changer!
At least for what I wanted it for - and I'm so glad Dag took the work and made it happen in RxPY, when ReactiveX standardized .pipe(...)

Because now I can get my processing chain specs from anywhere, e.g. from json over the network, created by other people.
All you have to do is replace function names from the spec by the functions themselves (in map, or other ops dependent what you want to offer in such specs and possibly turned into partials, when parametrized). And working with lists is nice in python.

Then you can throw the spec pretty much as is into the pipe function .pipe(*my_mapped_function_list) and you just built a working computing pipeline, over a network, running e.g. in a worker.

Yes, would have been possible with meta programming tricks also by dot chaining the functions - but latest when you want to go recursive or let the guys on the other side define if-else or async stuff or data merging operations it gets really nasty. I always ended up with trying to explain the domain expert guys, who defined what should happen when, that they should express their stuff within the (Reactive) framework. But normally they did not pick it up, they could not overcome the initial understanding barrier. Now with pipe I can abstract that away from what they care about, business flows.

Trying to say: With pipe, composability is just out of the box and just fits with what python has to offer: partials, variable length function args, kw args.

So, yeah, I love "pipe" meanwhile. Love on second sight, but those usually last longer anyway ;-)

[Dag Brattli]

Yes, free functions are the most flexible. I think many of us would like to see a pipe operator |> in Python without having to overload __or__. But even in F# you get problems when you take it further with e.g monadic functions where >=> is specific to a given effect e.g Result and you need to be careful about the sequence of importing dependencies if one dependency have an different >=> than the other.

I've btw added .pipe as a method on all classes so you can always start piping on the object itself, e.g:

ys = xs.pipe(
    Seq.map(lambda x: x * 10),
    Seq.filter(lambda x: x > 100),
    Seq.fold(lambda s, x: s + x, 0)
)

For other objects you would need to use the pipe-function. Also note that the above example can be composed directly to create your own custom operators e.g:

from fslash.core import compose

custom = compose(
    Seq.map(lambda x: x * 10), 
    Seq.filter(lambda x: x > 100), 
    Seq.fold(lambda s, x: s + x, 0)
)

ys = custom(xs)

[ShalokShalom]

Wonderful, I didn't know about that 🤭

Do you consider this as something that is worth finding its way into the Readme?

I personally feel the comment above can be copied into it almost 1:1

Also the benefits that AXGKI mentioned, maybe?

Thanks 👍🏻

[Gunther Klessinger]

But even in F# you get problems when you take it further with e.g monadic functions where >=> is specific to a given effect e.g Result and you need to be careful about the sequence of importing dependencies if one dependency have an different >=> than the other.

@dbrattli , in order to give a qualified answer I did invest a little bit of time in this F# thing and must say, yes, looks really fun to read and write...
My brain grew into a state where reactive became natural to read and write and I wonder more where this here could fit into a reactive pipeline, basically, for what do I need it when I have your RxPY?

wild guessing of mine...

Have to invest a little into comparing and combining the two - guess FSlash will be faster and stacktraces will be nicer and it can help reduce "lala land" even further, for use cases, where building the reactive trampoline would be overkill (?) Plus this railway oriented programming could be done I guess, by putting every event through a reactive pipeline into such an maybe(?) option(?) either(?) whatever monad foo thing (sooory;-), you know what I mean....) I'll find out myself, will need a little time though, so I don't put this into a new question.

Would be cool to see a use case one day from you, where F# and RxPY are combined in an example or so. I saw that in F# they from time to time use IObservable, so it seems they really are complementary.
Like you did back then with the reading of some webresource, based on throttled distinct user input from a websocket, that was an eye opener ;-)
Thanks again for all your work for the community!

[ShalokShalom]

@AXGKl My 2 Cents: Reactive programming got popularised due to Elm. It was the first commercially used, production-ready implementation of a reactive pattern and brought a lot more to the table, that complemented it.

React then took one aspect, put it on top of .. Javascript and called it a day. Since people dont like studing new things, was the jump to React easier for them - even if it meant more pain in the long run.

In case you are interested into this kind of programming:
Elm gave FRP up, since it found something even more intriguing: The Elm Architecture. And that one is also, what gets now copied all over the place: FSharp with Elmish and WebSharper MVU and OCaml with Bucklescript TEA, Elm itself and lots of other implementations.

All they have proper type inferrence and a HM type system.
In case you are truly interested into FRP, you might want to look into other languages, and how they implement that concept.

[Gunther Klessinger]

@ShalokShalom, hey. Should clarify (the names are really misleading) that ReactiveX and it's [Python implementation] (https://github.com/ReactiveX/RxPY) from Dag has nothing to do with React the JS Framework (which is not reactive at all) but also does not impose FRP on you. You can as well push mutable objects around.
But they do care about when stuff happens i.e. in which order.

I played around with elm back then, when I was doing a projects using Redux and the creator constantly mentioned elm as his inspiration. React was never an option for me and I have no idea even why they named this thing like they did. Svelte should be called React or nowadays Angular ;-)
And yes, would I do a frontend project again, I'd definetly go again for an immutable state model, where model changes trigger view updates, deterministic at any time, alone from the model, like in elm (and as you say, which made it also into the F# web architecture).
The only deviation: I would always feed the changes into an event stream[1] and not dispatch and reduce updates directly. And that concept of an (better: THE) event stream as first class design concern, I find still, after many years absolutely the right way to think about the problems, at least in my domain, frontend anway but also backend. Far more important than abstraction or encapsulation concerns or data shape and all that stuff... And yes, also more important than purity, i.e. concerning functional or not. Especially in python immutability anyway is pretty expensive, so yes, as you say: When I really want to do FRP I'd look for another language and after this here it would not be clojure anymore but F# - so excited you both seem to be about (thanks for the hints btw)

[1]: To be clear: With "Event Stream" I mean it in the sense of ReactiveX, not to be confused with FRP. You can pump mutable objects through a ReactiveX pipeline and its up to you if you mess up the state of those within the stream or if you do it functional. They don't impose anything on you and you'll find out the hard way that you better deepcopy here and there ;-)
But what they DO care about is: WHEN does stuff happen. Time - with it you get a tight grip on time: The expensive stuff only happens when someone is interested in its results and it happens at a pace you have 100% control over - at any time.

<party talk>
I'm actually a education wise a particle physicist, so the idea that the system behaves differently, dependent on observation was and still is totally resonating with me, plus: Hey, reality outside this IT world is 4 dimensional - but 3 of those dimensions are gone within the software we write, only one is left: time :-)
</party talk>

I'll see if it makes sense to combine this with F# (or FSlash to begin with, guess its the most convenient start into this world, for a python guy)

[ShalokShalom]

Oh, thats cool. I started from the beginning with functional programming and all this stuff, since I watched a lot of videos and read articles about this topic and the experts seemed all to recommend it, while the overall population seemed very fond of what they are used to, despite its flaws.

I found many useful concepts, currently is Godot a great thing: https://medium.com/swlh/what-makes-godot-engine-great-for-advance-gui-applications-b1cfb941df3b

Especially for you being interested into particle physics, could that be cool, we support also Python and F# as community tools and the upcoming version has a new particle system:
https://godotengine.org/article/improvements-gpuparticles-godot-40

[Gunther Klessinger]

Wow this godot looks massive. It's insane what the opensource world is able to achieve.... Many thanks for the pointer! And I'm glad that FP has a renaissance, too much energy went down the drain thinking about insane object hirarchies the last decades...

PS: I've put most of my stuff into details above, sorry for the post-editing. But this is the first issue of a Dag Brattli project - and your question was exactly the perfect one to get answered by Dag himself - don't want to distract people interested in FSlash.

PS2: particles...

The particles I meant are on yet another scale - and *my* explaination of their "crazy" behaviour is, that that the universe seems to be set up ReactiveX style. No traveling back in time like most Physicists think but subscription pipelines built before any message particle is emitted. Even the terminology fits, observable streams everywhere :-)

[ShalokShalom]

Yeah, we can succeed chatting here, the Godot Discord: https://discord.gg/ZxWHsH9

🥳