Lobster's future, performance benchmarks, FFI
dumblob opened this issue · comments
Hi, newbie here.
I wonder what is the foreseeable future of Lobster. Any plans? Any real world use cases? Any production systems/apps/... using it? Any commercial subjects investing into Lobster ("backing it up")?
I'd also like to ask for clarification about performance. I couldn't find any benchmarks, but the http://strlen.com/lobster/ page mentions rather poor performance (it compares it to interpreted Lua). I'd say a language with strict static typing and highly optimized refcounting GC should score much higher - maybe close to Nim or even V. Could you add Lobster to https://github.com/kostya/benchmarks/ to make it crystal clear?
And finally let me ask for an update on FFI in Lobster. I've read somewhere .so loading might be implemented. My question is two fold. I'm interested in both static (compile-time) FFI for speed (and safety as a second and less important reason) and runtime FFI (for convenience).
For plans in terms of features, scanning thru TODO.txt is currently the best indicator. There are currently no major features planned, in the sense of ones that completely overhaul the language.. Lobster has existed for 10 years or so now, and many large changes that I wanted to do have happened recently, so hopefully the language is entering a phase of more "stability" now.
It's mostly me working on it, and even that is a side project next to my day job. I wish there were corporate backing, that would change everything.. but corporate backing is very hard to get, and there are lots of new languages. I don't have the time to promote it to users and corporations to try and make it happen, either.
As far as big users, that is hard to tell, with only small percentage actively engaging. I don't know of a major software product or game having shipped with it. My most serious project that I am building with it is this https://twitter.com/wvo/status/1349404384334028800 which I hope will eventually become a release on Steam, but its early days.. beyond that I have a ton of game prototypes and even another programming language written in it.
Performance has changed over time, as it started out as an interpreter, and only recently has become JIT/AOT only, so now is a lot faster. That said, the early transition to JIT was as an "unrolled interpreter", meaning a lot of the interpreter mechanisms were still in place, and I've slowly been undoing them.. so while the language has the potential to be at least as fast as those languages mentioned, there's still a lot of work to be done for that to actually be the case. I'd rather do benchmarks once that work is further along, but https://github.com/kostya/benchmarks/ seems a nice one to add to, will add to TODO.
If you want a language that is within close range of speed to C, Lobster isn't a contender yet.
It hasn't been an issue for me, since in games I like to lean on all the C/C++ libraries out there for anything speed sensitive rather than re-implementing everything in Lobster, but yes, the end goal for this language is to be able to compete with e.g. Rust for most things.
The .so loading would be mostly to allow certain functionality to be built independently of the main Lobster exe, but as far as FFI is concerned it would identical to how the current FFI works (which must be statically linked in, as explained in http://aardappel.github.io/lobster/implementation.html). I am not sure what you mean by "compile time FFI".
I should probably update that claim of comparing to interpreter Lua, since that is a bit old. This is the only somewhat recent benchmark: https://github.com/stefandd/Tic4
Website updated :) http://strlen.com/lobster/
My most serious project that I am building with it is this https://twitter.com/wvo/status/1349404384334028800 which I hope will eventually become a release on Steam, but its early days..
Interesting. I'm aware of one quite innovative block-based game - https://store.steampowered.com/app/1213450/Craftica/ which is at the same time surprisingly cheap on resources (I know the lead dev quite well - he's the creator of the Dao language and he's an excellent programmer with deep theoretical and practical knowledge).
beyond that I have a ton of game prototypes and even another programming language written in it.
Curious about that. Especially why yet another programming language?
Performance has changed over time, as it started out as an interpreter, and only recently has become JIT/AOT only, so now is a lot faster. That said, the early transition to JIT was as an "unrolled interpreter", meaning a lot of the interpreter mechanisms were still in place, and I've slowly been undoing them.. so while the language has the potential to be at least as fast as those languages mentioned, there's still a lot of work to be done for that to actually be the case.
Will actually the non-WASM interpreter stay in Lobster or the future is only C++ and WASM backends (while going with saying there are good enough WASM interpreters out there)?
If you want a language that is within close range of speed to C, Lobster isn't a contender yet.
Assuming the speed of Lobster development will not diverge much from the last two years, when could it get close to C speed?
The
.soloading would be mostly to allow certain functionality to be built independently of the main Lobster exe, but as far as FFI is concerned it would identical to how the current FFI works (which must be statically linked in, as explained in http://aardappel.github.io/lobster/implementation.html). I am not sure what you mean by "compile time FFI".
With "compile time FFI" I meant statically linked in and thus requiring binding generation (either automatically or manually) & maintenance. With "run time FFI" I meant run time binding to symbols - imagine manually fopen()ing mylib.so and then calling a function whose name will the user give as input string.
Speaking of FFI, the craziest but enormously useful thing would be to allow seamless use of Python modules acquired using pip. It'd use some generic Python C FFI under the hood and provide some nice syntax similar to Python's "from xxx import yyy, zzz" etc. Yes, it's totally crazy, but would actually solve the biggest problem of any non-widespread language. Namely the lack of huge ecosystem. And because Lobster looks very similar to Python experience-wise, this crazy idea could actually work.
I'm aware of one quite innovative block-based game
There's hundreds of block based games being worked on now.. many not particularly impressive though. Craftica looks cool, I'll have to try it out to see how they are doing the slant editing.
I've been working on block based games since ~2000: http://cubeengine.com/
DAO looks impressive.. with an LLVM JIT no less :) I have plans for more direct LLVM integration.. one day.
Especially why yet another programming language?
Um.. your asking that on a repo for yet another programming language? :)
I've designed and implemented several dozens of languages.. typically novel design ideas don't all fit inside a single language, so sometimes you have to start a new one. The one I am tinkering with doesn't have a lot of overlap with Lobster, it is very low level by design and with much more explicit typing. It serves a different space.
Will actually the non-WASM interpreter stay in Lobster or the future is only C++ and WASM backends (while going with saying there are good enough WASM interpreters out there)?
I am not sure what this is asking.. currently Lobster has pretty much one backend that spits out either C or C++ that gets fed to libtcc or an external C++ compiler respectively. There is no interpreter anymore. There used to be a direct to Wasm backend, but I scrapped that for the C++ backend, since it was easier to maintain and generates more optimal code. There's no need for a Wasm interpreter.. if I wanted an interpreter, directly interpreting Lobster bytecode would still be faster, since converting to Wasm generates relatively a lot of Wasm instructions for each Lobster bytecode.
Assuming the speed of Lobster development will not diverge much from the last two years, when could it get close to C speed?
Hard to tell.. on the C++ path getting to closer to C speed relies on the C++ compiler being able to collapse the relatively clumsy output I feed it down to nothing, which entails a) making the output less clumsy or more direct (less abstraction) and b) carefully directing the compiler what to inline etc. That's a repetitive process of staring at the output assembly, guessing why it wasn't able to optimize it, improving the C++ output, repeat. Some of these improvements are not easy, since there are a lot of layers of abstraction in between, for example how all builtin functions are treated equally, how structs and variable are effectively on a little local register stack, etc.
There may be some cases where it is already close to C. The problem is, it is fragile, one use of a language construct or built-in function that blocks optimization and suddenly you're looking at a huge slowdown compared to C. So making it more predictable is the hard part.
I could give an estimate, but it is rather meaningless.. as this is a very non-linear optimization process, and it also depends on the time I have to dedicate to it (which may be worse or better than the last 2 years, and depend on how much myself and others need the extra speed). And I don't like making promises.
With "compile time FFI" I meant statically linked in and thus requiring binding generation.
Yup that is pretty much what we have now.
the craziest but enormously useful thing would be to allow seamless use of Python module
I actually did look into what would be required to load Python C modules exactly for that reason (using existing libraries easily), but the problem is that the APIs to these modules heavily depend on CPython data structures and specifics.. its the same reason why its hard for optimized Python implementations to gain traction, since you still need half a CPython implementation just to access all these modules. And while Lobster may look like Python, it is super different when it comes to data structures, so not actually that easy to bind to.
An easier FFI method is on my list, probably something based on being able to declare any binding specifics in Lobster itself, such that it could bind to C code without further C/C++ glue.
The current FFI system requires a lot of manual code writing, but I usually use that opportunity to simplify/lobsterify the API. At least I am getting something for my effort. Most C library APIs would feel very clumsy when ported 1:1 to Lobster.
the craziest but enormously useful thing would be to allow seamless use of Python module
I actually did look into what would be required to load Python C modules exactly for that reason (using existing libraries easily), but the problem is that the APIs to these modules heavily depend on CPython data structures and specifics.. its the same reason why its hard for optimized Python implementations to gain traction, since you still need half a CPython implementation just to access all these modules. And while Lobster may look like Python, it is super different when it comes to data structures, so not actually that easy to bind to.
Maybe not trying to make it that much efficient but rather "just avoid copying if easily achievable".
http://blog.leahhanson.us/post/julia/julia-calling-python.html
That'd still work for the use cases I had in mind (and would be orders of magnitude faster than writing it on your own or migrating the whole project to a different library used through FFI).
An easier FFI method is on my list, probably something based on being able to declare any binding specifics in Lobster itself, such that it could bind to C code without further C/C++ glue.
Yep. I'm looking forward to that
The current FFI system requires a lot of manual code writing, but I usually use that opportunity to simplify/lobsterify the API. At least I am getting something for my effort. Most C library APIs would feel very clumsy when ported 1:1 to Lobster.
Sure, they would. Though I still perceive them having at least 3 rather major advantages:
- they would require 0 maintenance incl. no need to maintain separate documentation (that's the main motivation behind my question about FFI)
- they would cause 0 cognitive load (no need to learn anything new, no need to think about "yet another abstraction" nor "up to which degree this particular lib was actually lobsterified")
- they would have higher satisfaction potential for the programmer who would have broader opportunities to efficiently create needed (and only those really needed but no more) own abstractions instead of "fighting" with abstractions (often without an efficient way to escape them) someone else made just for the sake to have them (i.e. not tailored to the context you want to use the library in)
Maybe not trying to make it that much efficient but rather "just avoid copying if easily achievable".
http://blog.leahhanson.us/post/julia/julia-calling-python.html
That'd still work for the use cases I had in mind (and would be orders of magnitude faster than writing it on your own or migrating the whole project to a different library used through FFI).
I can see the convenience, and it would be fun to do a prototype to see what it would take to directly load Python C libs.
That said, in the Julia case it also has pyeval, which means it must link or rely on the whole CPython implementation. I would not want such a dependency, keeping Lobster a self-contained thing is pretty important to me. Though I guess having functionality that will simply fail if it can't find an already installed Python could be ok, I don't find it particularly elegant.
Also, Julia is at its core a dynamically typed language, interfacing with another dynamically typed system, which is a lot easier. On top of that it has macros to paper over some other differences. I suspect that in Lobster it be either less elegant and/or a lot more work.
And I don't see how to access all these Python data structures without actually including a bunch of Python code. That would be the first thing to look into. Looking into e.g. the numpy C API, it takes all these heap allocated Python structures as arguments, which means it will in the very least need a small "Python heap" to store all these things. Unless it has an explicit way to delete these, that may drag the Python GC along with it, etc etc...
Sure, they would. Though I still perceive them having at least 3 rather major advantages:
- they would require 0 maintenance incl. no need to maintain separate documentation (that's the main motivation behind my question about FFI)
Yup, definitely an advantage.
- they would cause 0 cognitive load (no need to learn anything new, no need to think about "yet another abstraction" nor "up to which degree this particular lib was actually lobsterified")
That depends on whether you are someone familiar with the lib looking to use it with Lobster, or someone who is using Lobster that may want to use the lib. The latter category is definitely better served with a "lobsterified" API.
Have a browse thru http://aardappel.github.io/lobster/builtin_functions_reference.html and imagine how a lot of these things would look like if they were direct mappings to C.
C APIs use a lot of pointers and handles that need to be maintained by the calling code that I tend to do away with. Or errors that need to be checked because things spread out over multiple API calls that I do away with because it all fits in one call. Or defaults I can apply to a more higher level API that C programmers would not be happy with. etc.
For extreme examples, check out functions like circles_within_range (or wave_function_collapse) which are entire algorithms in a single function call.. they are extremely convenient to call from Lobster, and I've already used them several times in game prototypes to get quick collision detection going or whatever. Yes, they're a bit specific, and maybe even strange to have these in a standard lib. But a version of this functionality that was more generic would likely be spread out over many API calls, require many data types etc. and be slower.
- they would have higher satisfaction potential for the programmer who would have broader opportunities to efficiently create needed (and only those really needed but no more) own abstractions instead of "fighting" with abstractions (often without an efficient way to escape them) someone else made just for the sake to have them (i.e. not tailored to the context you want to use the library in)
Sure, I see that too. As Lobster gets faster / more low level, this makes a lot of sense. For example, the calling style of the functions in https://github.com/aardappel/lobster/blob/master/modules/gl.lobster (allowing these calls to take a body, i.e. calling a function from within a builtin, which is kinda complicated) used to be directly part of the builtin implementation. Now it makes use of much simpler underlying functions and does all the complicated stuff in Lobster itself. There may be other builtins that can be simplified this way.
Another factor in "should this API be a direct C mapping or lobsterified" is to what extend they are part of the standard lib. I will continue to want the standard lib to be pretty elegant and fitting to the language. But if at some point it is easier to pull in entirely optional libraries, I don't mind as much if they have a somewhat clumsy C derived API. Maybe there are parts of the current standard lib that are better off as optional (the Box2D bindings would be a good start).
And I don't see how to access all these Python data structures without actually including a bunch of Python code. That would be the first thing to look into. Looking into e.g. the
numpyC API, it takes all these heap allocated Python structures as arguments, which means it will in the very least need a small "Python heap" to store all these things. Unless it has an explicit way to delete these, that may drag the Python GC along with it, etc etc...
Admittedly my vision was really to kind of "embed CPython". But as a module and distributed separately. So basically fully opt-in. The effort on making "Python interpreters" finally "first-class" (especially with regards to true multithreading without GIL) could make this less convoluted if one would accept some (de)serialization costs. IDK.
Also, just to clarify, I'd slightly prefer the "dynamic" way of Lobster<->Python interaction as PythonCall.jl does over PyCall.jl's attitude mentioned above.
- they would cause 0 cognitive load (no need to learn anything new, no need to think about "yet another abstraction" nor "up to which degree this particular lib was actually lobsterified")
That depends on whether you are someone familiar with the lib looking to use it with Lobster, or someone who is using Lobster that may want to use the lib. The latter category is definitely better served with a "lobsterified" API.
Yep. My inference was: if I don't know the lib yet, I'll scroll through the lib's doc, read some genuine tutorials etc. and I don't want to do the same second time for the lobsterified bindings. But if you are already familiar with the lib and "want to get things done", then you won't probably mind learning the lobsterified API.
For extreme examples, check out functions like
circles_within_range(orwave_function_collapse) which are entire algorithms in a single function call.. they are extremely convenient to call from Lobster, and I've already used them several times in game prototypes to get quick collision detection going or whatever. Yes, they're a bit specific, and maybe even strange to have these in a standard lib. But a version of this functionality that was more generic would likely be spread out over many API calls, require many data types etc. and be slower.
I fully understand what you mean. I'd actually love to have both for each lib - the "original" ugly API and then a lobsterified one. I'd just prefer to have the "original" ones first and then if there is enough time add also the lobsterified ones
- they would have higher satisfaction potential for the programmer who would have broader opportunities to efficiently create needed (and only those really needed but no more) own abstractions instead of "fighting" with abstractions (often without an efficient way to escape them) someone else made just for the sake to have them (i.e. not tailored to the context you want to use the library in)
Sure, I see that too. As Lobster gets faster / more low level, this makes a lot of sense. For example, the calling style of the functions in https://github.com/aardappel/lobster/blob/master/modules/gl.lobster (allowing these calls to take a
body, i.e. calling a function from within a builtin, which is kinda complicated) used to be directly part of the builtin implementation. Now it makes use of much simpler underlying functions and does all the complicated stuff in Lobster itself. There may be other builtins that can be simplified this way.
Another factor in "should this API be a direct C mapping or lobsterified" is to what extend they are part of the standard lib. I will continue to want the standard lib to be pretty elegant and fitting to the language. But if at some point it is easier to pull in entirely optional libraries, I don't mind as much if they have a somewhat clumsy C derived API. Maybe there are parts of the current standard lib that are better off as optional (the Box2D bindings would be a good start).
Oh, I didn't even think about having any FFI lib as part of the standard library (and those few exceptions I'd rather rewrite in Lobster as they probably won't change much over time - e.g. some hash function or PRNG etc.). Of course I wouldn't want to have any non-lobsterified APIs in the standard library. But maybe it's just my different view of what a standard library shall look like
Admittedly my vision was really to kind of "embed CPython"
The funny thing is that the CPython code base is like >10x bigger than Lobster.. so it's more like embedding Lobster in CPython, if that matters :)
But maybe it's just my different view of what a standard library shall look like
Well, this is all mostly historical, from the start I've been implementing standard functions in C++ and adopting quite a few C/C++ libraries for standard functionality.. it was convenient at the time, and part of the very incremental way Lobster has developed. Probably if you'd plan a language more long term, having as much code in the language itself is definitely superior.
Ok, I think my questions got answered. Feel free to close this (for me rather interesting) discussion if you don't see any good use of it.
Thanks for your patience - I'll follow Lobster's development and consider giving it a try for something smaller
@dumblob it's very interesting, and always good to hear someone's perspective on what the language / "eco system" needs.
To summarize I see a couple of things you're suggesting / would like to see:
- Continued work on performance.
- Ability to extend in .so/.dll
- Easier FFI
- More of current functionality moved out of standard lib, and into optional FFI, more into Lobster code based on lower level FFI, once the above works out.
- Possibly a way to pull in Python libs easily.
Not only are these features most logical roughly in that order, they are also most likely to be worked on by me in that order of importance, I think. But anyone else wanting to start pushing any of these also welcome :)
Probably if anyone wants to work on something specific, they can open a new issue referencing this one.
Yep, that summarizes it, thanks!
(though I myself wouldn't probably put it this "pushy way" as I currently don't have any time to spend on Lobster as can be seen from the delays in my answers