Consider rewriting the core engine in Rust (with Node binding)

[David Bruant]

From what I understood meeting with @CedricArtigue , JS was chosen among other things because of the community and that you could have some predictable performance (by reusing objects which prevents triggering GC) and a more reasonable workflow than what's possible with Scala.
Another idea for predictable performance would be to use Rust. It has C-like performance (and predictable memory characteristics unlike JS, see #52 ), static typing, awesome community, easy parallelism, actual threads (unlike Node which only has processes), safe memory.

It's possible to expose a Node.js API via a module which code is written in Rust. See
https://blog.risingstack.com/how-to-use-rust-with-node-when-performance-matters/
http://calculist.org/blog/2015/12/23/neon-node-rust/

[Marc Vertes]

Thanks for the suggestions regarding rust, we will look at it closely.

Another reason behind the choice of javascript worth to mention is the dynamic nature of the runtime: Javascript is fully interpreted, i.e. parsed, compiled then executed on the fly by a JIT compiler embedded in the runtime, and exposed via simple statements like eval() or Function() (which you noted in #53).

Beyond the confort for the developper, this feature is essential to implement a smart and fast distributed system, where we generate then send then execute pipelined worker code for each stage of the execution graph. Working at source level directly to native machine code rather than at bytecode or object code (as for java hadoop/spark) allows for deep optimizations and large performance gains. For that the javascript combination JSON.serialize/parse, streams and eval is quite good and hard to beat.

Would be happy to reconsider other languages/platforms if/when they bring similar capabilities.

[paullovessearch]

Interesting, I had never heard of RUST before, so it was good to do some research.

My interest in skale is for a JSON-native distributed processing engine, which RUST is not, but which JavaScript (obviously) is. So I vote for sticking with JavaScript.