A uniform interface is
yevhen opened this issue · comments
I'd like to discuss an addition of a uniform communication interface, which has potential for seriously improving Orleans' interoperability, extensibility and ease-of-use for a number of advanced scenarios.
Can you please elaborate a bit what kind of communication interface you are talking about. We have been playing with some ideas here as well.
No worries, I'm writing it. Mind you, it will be rather lengthy one :)
Well, the response was so lengthy that I've decided to split it into 2 parts and don't post the second part, since it will be a huge spoiler of my presentation which I'm preparing for an upcoming Orleans meet-up. Let's keep a bit of mystery :)
P.S. I've also changed the name of the issue back, as I firmly believe that a uniform interface is indeed 42 :) Let's reserve this issue for a future pull request (if there will be general acceptance, of course).
Some food for thought.
I believe that a root of all current Orleans' problems stems from the unfortunate decision to support a non-uniform communication interface.
Consider the following non-uniform interface:
public interface IInventoryItemGrain : IGrain
{
Task Rename(string newName);
Task CheckIn(int count);
Task CheckOut(int count);
Task<int> GetTotal();
}Due to remoting nature (and custom execution scheduling) client-to-grain and grain-to-grain communication via plain .NET object reference is impossible and communication proxy need to be used instead. The communication proxy will hide the complexities of calling remote procedure by automatically handling serialization/deserialization, activation, dispatching, etc.
This is a well known RPC-pattern. With the well-known set of problems. Since the proxy is completely transparent and completely mirrors the interface of the remote object, there only 2 possible ways exist to generate such proxies:
- Dynamic (run-time) generation (AOP Interceptors, ExpressionTrees)
- Static (code) generation (WCF, Web Service Reference)
The Orleans' team decided to use the 2nd approach with all its ramifications, such as:
- Creating and maintaining code generation tool for each possible target language (there are over 30 languages available on .NET platform) is a burden, so only a subset of popular languages was chosen, seriously limiting the use of the framework outside of the selection.
- Requiring unnecessary separation of interface from it's implementation, leading to duplication (mirroring) of declarations across files and proliferation of projects (due to requirement of physically separating interface/implementation assemblies).
Let's imagine, for a moment, that instead of using static code generation we go with the option 1 - dynamic proxy generation. What will be different?
- No need to generate language-specific code - just CodeDom or IL or whatever and then generate and cache a DynamicAssembly at run-time.
- Since there is no language-specific parsing anymore (just reflected .NET type) - the serialization of anything is possible, ie F# tuples, record types, discriminated unions, etc.
That fixes #38 in terms of the language support (the F# interplay of Async and TaskScheduler is a different problem, that need to be tackled on its own). And it's not only F# support, it's supporting all .NET languages out there.
That also partially fixes #37 and makes #40 largely irrelevant. I have doubts about ROI of implementing #40. If it ain’t broke, don’t fix it!
To be continued ...
I'm not sure you've addressed @gabikliot question about the 'common' or 'uniform' interface. I am assuming you're using the term Uniform Interface to refer to rest style uniform interface constraints, but I'd prefer not assume.
I'm also unclear how uniform interfaces relates to dynamic vs. static rpc proxy generation.
Playing the part of the ever critical... :), I'd note that the proposal illustrates the advantages of 2, while exposing the disadvantages of 1. For a clearer differentiation, can you elaborate on the advantages/disadvantages of both approaches?
“That also partially fixes #37” - Add support for Bond/Avro/other serialization
These are serialization technologies which have specific advantages (performance, data versioning, …). Allowing a bond object to be serialized across the framework in some manner other than through bond serialization would not provide the benefits bond users expect.
@jason-bragg Indeed, I haven't addressed it fully. I've started writing an answer but it was too long and will spoil my presentation for an upcoming Orleans meet-up. So I've just posted the first part of the puzzle, about dynamic vs static code generation and what problems it could fix.
To be precise, it was only about language support. Since, with dynamic approach the need to support a language-specific code generation will go away. At run-time, everything is just a some form of .NET type, fields and properties, whether it's an F# tuple, record type or DU. Perhaps, an automatic serialization won't be the most optimal serialization, since there could be some internal, non-serializable fields or properties. But it's a step into right direction, IMHO.
Taking it further. If you want to have a more optimal serialization, you could plug it in. That might be something that recognizes some intrinsic properties of the structure being serialized, such as generic converter that now how to optimally serialize/deserialize an F# discriminated union or record type. In that way the burden of supporting all possible types is shifted away from framework developers. And that's nice, as now they can focus on a more important things, than on nifty-gritty details of some weird data structures being used by their users.
... I'd note that the proposal illustrates the advantages of 2, while exposing the disadvantages of 1.
Did you mean it other way around? :)
@jason-bragg Yes, the REST is the perfect example of uniform interface.
“That also partially fixes #37” - Add support for Bond/Avro/other serialization
These are serialization technologies which have specific advantages (performance, data versioning, …). Allowing a bond object to be serialized across the framework in some manner other than through bond serialization would not provide the benefits bond users expect.
That's why I said partially :) With dynamic proxy generation, the serialization format could be an option. Still, for the full compliance with a schema-based serialization formats, such as Bond/ProtoBuf/MessagePack a uniform interface is required.
BTW, at the moment, you could plug you own serialization very easily. But if you're still using code generator, it won't buy you much in terms of the performance, since codegened serialization will always be faster. That means there is no point to support custom serialization protocol for just performance reasons but rather for the other factors that you've mentioned. IMHO, the most important one is interoperability.
@gabikliot @jason-bragg Don't get me wrong, I don't want this change (static-to-dynamic generation) to be implemented, but it could be a last resort if there will be no agreement on introduction of uniform communication interface.
“an automatic serialization won't be the most optimal serialization, .. But it's a step into right direction, IMHO.”
In the general sense, that serialization should be magical and just happen, I agree, this is a step in the right direction. However, property based encoding of objects structured to be serialized using a specific technology will likely (has for us) introduce very obscure and hard to track down bugs. This means, the ‘magic’, in this case, is more of an illusion, prone to disappointment.
“more optimal serialization, you could plug it in”
Orleans does allow for custom serialization, but I am of the opinion more work is necessary to efficiently support third party serialization protocols.
“burden of supporting all possible types is shifted away from framework developers”
Agree 100%. What is needed, more than support for some specific serialization tech, is improvements upon the extensibility points where third party serialization technologies can be added. Though it occurs to me that I’m writing this in the wrong thread, so I’ll shut up about it now. :)
“Did you mean it other way around? :)”
Doht! face-palm
“Yes, the REST is the perfect example of uniform interface.”
Thanks for the clarification.
@jason-bragg -->
What is needed, more than support for some specific serialization tech, is improvements upon the extensibility points where third party serialization technologies can be added.
Once again, a uniform interface will be of great help here, since it effectively reduces the number of such extensibility points ;)
@yevhen do we have a date set for your presentation yet? Looks like we might want to dedicate a part of it to this interesting philosophical discussion. :-)
I think we can plan for the Friday next week. I'll contact @richorama tomorrow.
I am intrigued to hear more about this idea, not least because of the embedded HHGG reference ;)
It would definitely help the understandability of the proposal if it can chrisply spell out some of the intended scenarios, and in particular exactly which "interface"(s) would be affected.
For context on where i an approaching this from, i see many different "interfaces" involved in the end-to-end message flow in Orleans, including:
- client app code to grain [proxy] factory
- client-side factory to local [typed] grain-proxy
- proxy to client-side invocation pipeline [eg "client-side interceptors"]
- serialization-deserialization to-from wire message format
- grain directory and locator service{(s)
- server-side invocation pipeline to grain state storage provider(s)
- server-side invocation pipeline to grain code [eg "server-side interceptors"]
- and last, but certainly not least, then overarching app specicfic end-to-end client to grain interface
From earlier comments on this thread, i am guessing that you are talking about the interface to the invocation pipeline and run-time injection of AOP facets?
So please take your time and give us a chrisp and clear statement of what you are proposing.
This will help us to understand what you are proposing, and we definitely ARE open to new ideas that can help make Orleans even more useful for cloud-scale development. :)
Sadly, I only had time to explain what a uniform interface is, since there was a lot of confusion around that term, but hadn't explained why I think it is really a 42. So I'll briefly explain here.
The first and foremost, since message is a schema and message passing makes an interface uniform, there is no need to use static codegen anymore (to generate static factories and references for custom interface). From that point, Orleans' native (autogenerated) serialization is simply just another serialization protocol, same as ProtoBuf, JSON, Bond, etc, with some added features and limitations (limited language support and interoperability). It's just an option, not a requirement. You can get it on Nuget as an additional, optional package. This fixes or makes unnecessary:
- GrainFactory uses generated factories via Reflection #141
- Refactor code generation to use Roslyn instead of CodeDOM #40
- Proper support for F# #38 (and just any other .NET platform language)
- Add support for Bond/Avro/other serialization #37
- Mixing in interfaces and classes from other projects
- Grain interface and implementation in same project
Akka has Http protocol, so I don't see why Orleans can't. With uniform interface and support for any serialization format out there, creating custom protocol endpoint is just a piece of cake. That will make for great interoperability story and will give developers options, so they can choose whether they're ok with constraints of native serialization protocol (ie, writing code only with C#\VB) and use it, or choose something more interoperable (F#, calling Orleans cluster from Node.js frontend, etc) and live with its constraints.
Once we got rid of static codegen requirement, we are in a dreamland :). The next thing we can drop is SDK. IMHO, it only creates more confusion, adds maintenance burden and in my view it's simply a workaround. I haven't found anything valuable in this thingy. What that for? It's just few project templates, that won't be needed if we get rid of static codegeneration. It also contains docs and samples, which are already on GitHub and always more recent. Tools? That's just another NuGet package. So, this fixes or makes unnecessary:
- Cannot build grain interfaces project using NuGet's Microsoft.Orleans.Templates.Interfaces package #213
- Orleans in an Azure CI/CD setup? #192
- Orleans Code Generator NuGet #183
- And many others, which I haven't noted but I remember there were few more.
Ah, I forgot the most important part which SDK provides - pre-configured local silo host. Hey, Orleans' users, does anybody actually using that stuff? Why? What problem does it solve? What I know, for sure, is that it only creates more confusion with its WebSphere-wanna-be hosting and deployment model. Applications folder? Magic post-build hook in the grain implementation project, so it will be magically deployed (turns out, it's simply xcopied) to some magic folder, the path to which is defined in another magic environment variable ($OrleansSDK). Err, can someone explain me the rationale behind all of this?
IMHO, it creates wrong perception. Orleans is not WebSphere (god, thanks), it doesn't require dedicated application server (container) exe of some sort, which need to be pre-installed. It could be hosted anywhere. WebRole, WorkerRole, WindowsService, ConsoleApp, whatever. It's tiny. It's light. It's embeddable. You can host the silo anywhere you like. It's incredibly easy to do ... or not?
Perhaps, the sole reason why this thing (local silo host) exists, is to hide the complexity of configuring and hosting the silo. While configuring client/cluster when running them in different processes is more or less ok, the embedded scenario is absolutely horrible experience. Just have a look at Orleans samples. All that fiddling with AppDomains and initialization via AppDomainStart with callback method, arghhh ... This is something that I fixed at first in Orleankka. I want samples to be as lightweight as possible and the whole thing to be as easy as 2+2. So I've added fluent DSL around configuration and hosting to hide all of that complexity behind a discoverable API (facade), which guides end-user and is easy to consume.
From sample here:
var system = ActorSystem.Configure()
.Playground()
.Register(Assembly.GetExecutingAssembly())
.Serializer<JsonSerializer>()
.Done();5 lines of code to run Orleans client and cluster within a same process with some default (playground) configuration. Want to run this on Azure? Just reference Azure library and use extension method ActorSystem.Configure().Azure(). Done.
Simplified configuration and hosting is something everyone would deeply appreciate. Especially for embedded scenarios. It will also make specialized hosts obsolete and will give users that F5 experience without any dirty hacks with post-build events and pre-installed silos.
Another problem here is that programmatic configuration is considered second-class. Which makes me sad. The first thing I usually do, when dealing with libraries\frameworks which relies on file-based (xml, json) configuration is embedding those as resources. I fail to understand the point of deploying them raw. Does someone expect that this will be directly changed on prod server? Personally, I won't be happy if my admin will directly edit an Orleans configuration file. What she can safely change there and for what reasons?
Ok, that all just starting to be too ranty. And I've completely derailed off the subject. Let's get back to uniform interface (still, configuring everything with XML feels so enterprisey 😁 ).
Another interesting property of having a uniform interface is that it doesn't imply that implementation also need to be a uniform. Orleans requires that all grain methods should be async, either return Task or Task<T>. That leads to dreaded TaskDone.Done and Task.FromResult(42) for non-async behaviors. Which is just an additional noise. This is something that we have exploited in Orleankka with automatic handler wiring feature. It allows end-user to define handlers like this:
class TestNonUniformActor : Actor
{
public void On(VoidMessage m)
{}
public string On(ResultMessage m)
{
return "42";
}
}And that will be magically converted (wired) to an async version that will automatically return TaskDone.Done or wrap the result into Task<T>. This is impossible with native Orleans way of declaring custom non-uniform interfaces, due to 1-1 mapping between interface and implementation.
Amusing, isn't it? 😀
P.S. To be continued ...
Sorry for rant, I'm a bit cranky today ...
Continuing on interesting properties of the uniform interface ....
It's ridiculously easy to extend since it's extremely narrow, and this is one of the prerequisites for Decorator pattern. Basically, the one can decorate infinitely both around client communication interface (ActorRef.Tell/Ask) and also its server-side counter-part (Receive(m)).
Good example of client-side decorator can be found in Orleankka itself. I was extremely unhappy with how Orleans is dealing with exceptions. Instead of getting back an actual exception type, the end-user need to deal with AggregateException. Which is clunky and it reduces the benefit of using async\await.
AFAIR, the rationale behind this was, that with AggregateException the stack trace will be preserved across the chain of calls, which might be executed on different nodes. That full stack trace will be lost if an actual exception is re-throwed. Well, there is a well-known trick with preservation of stack trace on rethrow. So I've created an extension method for Task which will unwrap the original exception and re-throw it while preserving the full stack trace.
The problem is that with a non-uniform interface, the end-user will need to add .UnwrapException() to every actor call. Which is tedious and could be easily forgotten. But with uniform interface, I was able to simply add it to an ActorRef, so that any call will automatically have this behavior. Single place, 2 lines of code. Even if I didn't build this directly into Orleankka library, any end-user would be able to do exactly the same by creating a generic and reusable wrapper around ActorRef.
Amusing, isn't it? 😀
I've just lightly touched on power and incredible usefulness of uniform-interface with server-side decoration in my presentation. I did tell you that it is a most important hook that framework could provide and it's basically an AOP for free, and it's the most perfect form of AOP from all. Sadly, I've ran out of spare time to prepare slides with actual examples. So let's see few ...
P.S. To be continued ...
Please don't take our "silence" as a sign of disagreement @yevhen -- you just gave us a lot to think about all at once!
Thank you for the very informative talk last week, which greatly helped to provide more context on your ideas, and for writing up more details about your proposal above.
I am looking forward to your next Grumpy Cat installment on this subject ..... 😉
+1 Need time to think about the story and how best to integrate it without confusing and alienating people who like the existing one. Good stuff!
Yes, we need people like Yevhen (and cohorts). Let's link Codeplex here too. I'd like to have DI, in a way or another. Owin is my role model currently. There are other points of view too! :)