`walt-sugar` package
piranna opened this issue · comments
Feature Request
Overview
walt started as a Javascript-like "assembler" language for WebAssembly, in the way that it would only expose the WebAssembly bytecode as Javascript functions, allowing to program low-level code using a high-level language (sort-of).
Over the time, it has started to get some discussions about sugar syntax and high-level features, like Strings or literal objects. These are nice additions to be able to use, but are abstraction layers that don't directly map 1-to-1 to WebAssembly bytecodes. Since we are using here a lerna monorepo, I propose to split this sugar syntax and high-level features to a separate package walt-sugar to have both concepts isolated, implementing the high-level sugar using the low-level functions as building blocks. Alternatively, instead of separate the high level sugar, we could move out the low-level to a walt-assembler package that does only the 1-to-1 mapping of WebAssembly bytecode... or maybe both things, being walt-compiler just only a translator :-)
Impact
Medium. Need to create new packages and split code, but later concepts would be isolated and layered and easier to understand what's WebAssembly core and what not.
Due Date
2018-04-03
but are abstraction layers that don't directly map 1-to-1 to WebAssembly bytecodes
Strings and Objects do map 1:1 to bytecodes though. Maybe not how folks traditionally think about objects and strings(in JavaScript) but still. For example, strings literals are i32 static numbers in the binary and data sections. I don't know if there would be a better way of encoding a data section otherwise.
On the topic of Objects, they are more like C-structs, not JavaScript objects. The same idea for objects as for strings, they are literal i32 offsets with load & store instructions for object lookup and assignment.
I don't know, I don't think that nicer syntax means it's not low level enough.
I do think that a modular way of adding additional syntax on top of the core compiler would be nice. Like a plugin system of sorts, which could operate on the AST. Closures would be a good candidate for this. While the initial implementation is cool, I'm not sure I'm overly happy with it being the core of the compiler tbh.
To wrap it up, string and object syntax does not bother me as I don't plan on adding a dynamic string or object runtime. But allowing others a way of doing this would be valuable!
strings literals are i32 static numbers in the binary and data sections
Yeah, you've got the point, strings are just a list of integers, only that their value is interpreted as characters :-D What I'm talking about is that this data format interpretation can be move out of the core.
On the topic of Objects, they are more like C-structs, not JavaScript objects
So, they are not objects, but structs :-) My concern is about adding some semantics in the core that doesn't fit in a sub-set of Javascript but anything else. Objects in Javascript are more like a hashmap where keys are strings, if entries can't be added then they are not like Javascript objects but instead structs or like the __slots__ field in Python objects. Not bad, but not Javascript :-) But they could be implemented in a separated library, and also someone could implement real dynamic Javascript objects using a hashmap algorythm implemented using the core in a different library (and being a hashmap, it would allow to create Map objects too! :-D )
I don't know, I don't think that nicer syntax means it's not low level enough.
Me too, just only want to make clear and separate concepts of what's raw WebAssembly and what are walt sugar additions :-)
I do think that a modular way of adding additional syntax on top of the core compiler would be nice. Like a plugin system of sorts, which could operate on the AST
Great, I had the same idea :-D Mine was more about using a precompiler stage that translate from Javascript to walt using this sugar libraries, just like how Babel works to compile to previous versions of Javascript. In fact, don't know how much dificult is to develop Babel targets, but the same way it can generate code for different EcmaScript versions, it could be done to generate walt code :-) Obviously here we totally lost the concept of what's low level here, but it's an idea...
To wrap it up, string and object syntax does not bother me as I don't plan on adding a dynamic string or object runtime. But allowing others a way of doing this would be valuable!
Yeah, maybe the problem here is to decide how to allow to do that integration...
Yeah, maybe the problem here is to decide how to allow to do that integration...
Yup, the how is the difficult part :)
By the way, in my search through "languages that tried to be C alternatives", I somehow ended up in a paper for Scala which definitely is not one of those languages, but which does seem relevant to the current topic:
Miniphases: Compilation using Modular and Efficient Tree Transformations
Production compilers commonly perform dozens of transformations on an intermediate representation. Running those transformations in separate passes harms performance. One
approach to recover performance is to combine transformations by hand in order to reduce number of passes. Such an approach harms modularity, and thus makes it hard to maintain and evolve a compiler over the long term, and makes reasoning about performance harder.This paper describes a methodology that allows a compiler writer to define multiple transformations separately, but fuse them into a single traversal of the intermediate representation when the compiler runs. This approach has been implemented in the Dotty compiler for the Scala language.
Our performance evaluation indicates that this approach reduces the running time of tree transformations by 35% and shows that this is due to improved cache friendliness. At the same time, the approach improves total memory consumption by reducing the object tenuring rate by 50%. This approach enables compiler writers to write transformations that are both modular and fast at the same time.
https://infoscience.epfl.ch/record/228518/files/paper.pdf
Only skimmed it myself, haven't had time to dig in deeply, so I don't know if it's specific to JVM languages or if the techniques described are generalisable/translatable to Walt.
For anyone still following along, a major part of this effort has just been completed. Which is nice. It was an interesting challenge to modularize the compiler(at least the semantics parser for now) by feature. The core semantic parser is now completely split by feature and the compiler retained full feature parity.
All in all, this is/was a very good idea. This type of feature separation should pay off in the long term. Core features plus syntax-sugar(like closures) are all implemented as composable "plugins". The nice benefit of this approach is that it makes the syntax infinitely extendable, very similar to how JS is extendable through Babel.
For this to be "complete" it'll take a bunch of refactoring and moving of shared "parser utility" logic, plus ways to test syntax-sugar features in isolation etc. Not to mention the documenting all of the utils which used to be private to the compiler implementation + how to write a plugin.
Oh, and the changes would also require a major version bump to 1.0 (even though still alpha) as it'll change the public APIs without being backwards compatible. This might be avoidable but unlikely.
Anywyas, good stuff.
I'm glad my suggestion has been useful :-)
Oh, and the changes would also require a major version bump to 1.0 (even though still alpha) as it'll change the public APIs without being backwards compatible.
For the record: the SemVer rule you refer to only applies to v1 and up; the semver spec makes an explicit exception for v0 of any software product:
Major version zero (
0.y.z) is for initial development. Anything may change at any time. The public API should not be considered stable.
If you still consider Walt to be in alpha stage, it's better avoid going to v1 just yet. Besides, the releases page doesn't make it clear which of these pages is the Walt release. walt-compiler? walt-syntax?
Thanks for pointing this out. Makes things a bit easier