6.0
sebmck opened this issue · comments
Foreword: None of this is really set in stone. Feel free to argue and discuss. This is kinda a long issue so I added some gifs, I hear developers love that kind of stuff. This post is also a bit rough, figured it's better to get it out there ASAP, I'll improve it as more feedback is collected. Thanks everyone for your (potential) participation!
What is 6.0 going to look like?
No internal transformers. None. Zero. Zilch.
This will be coupled with the removal of the following options:
whitelistblacklistoptionalloosestage
All of these options are redundant when you're manually specifying transformers.
This dramatically reduces complexity in the transformer options. Right now there's way too many ways to change the state of a transformer. This will reduce friction and make it easier to reason about what's actually enabled.
There's also a major problem with including very experimental syntax/transformers in core. If a big change was made to their semantics it would be a breaking change and would require a major version bump of Babel. In the interest of stability and being reactive to changes in the standardisation process, it makes sense to version them independently.
When?
idk whenever
Where will all the current builtin transformers go?
Into userland plugins.
Wont this make it harder to use 6.x like we use 5.x?
Nope! 6.0 will also introduce the concept of presets. These are modules distributed via npm that specify a set of options (plugins etc) that will be merged into your main configuration.
Recommended
This means that the current ES6 to ES5 configuration would be distributed on npm via the babel-preset-es2015 package. It would have the index.js of:
module.exports = {
plugins: [
require("babel-plugin-arrow-functions"),
require("babel-plugin-classes"),
...
]
};To use this you'd just specify the following in your .babelrc:
{
"plugins": ["preset-es2015"]
}Community-specific
These can be even more specific. For example, the React community may have an official preset called babel-preset-react that will have the following index.js:
module.exports = {
plugins: [
require("babel-plugin-jsx"), // enables jsx syntax
require("babel-plugin-flow"), // enables flow syntax
require("babel-plugin-react-jsx"), // transforms jsx to flow
require("babel-plugin-flow-comments"), // transforms type annotations to flow
]
};Just like the es2015 preset, this could be enabled with the following .babelrc:
{
"plugins": ["preset-es2015", "preset-react"]
}Isn't this a dramatic shift in focus?
Nope, this is a part of the long term vision for Babel and everything has been leading up to this (cue dramatic music). I bought this up in #568 back in January where I said:
Ideally I'd like for 6to5 to be a more general JavaScript of the future transpiler. 6to5 already supports Flow/JSX as well as experimental ES7 features so the name isn't even representative of the project in it's current state although it is in how it's widely used.
A JavaScript transpiler based on future standards and specifications, whether ECMAScript or not, I think is an extremely noble goal. I think there's a large use case for 6to5 beyond just ES6. Having an interopable platform to base code transformations on I think is a very powerful idea. This may mean exposing a very feature-rich core that contains transforms or even exposing a public API.
What about syntax extensions?
I'll be holding out on allowing arbitrary syntax extensions for the timebeing. If we're going to do them at all, we're going to do them right. No point half assing a very critical and fragile piece such as syntax extension.
But if there's no syntax extensions then how would existing transformer that enable special syntax work?
So, the syntax would be implemented in the Babel parser Babylon, the "syntax plugins" would just toggle some parser options via an API. So the actual parser code would still live in Babylon. It's a neat solution until we support syntax extensions properly.
What else is coming?
Plugin options (#1833)
Plugin options are something that sorely needed.
Specifying options
These can be specified via a multidimensional array. For example, say you have a babel-plugin-pancakes plugin, you could specify options to it like so:
.babelrc
{
"plugins": [["pancakes", { "syrup": "maple" }]]
}But how would you specify options for plugins defined in a preset? One possible way is to specify them in the options to a preset, so if you had a preset like:
node_modules/babel-preset-breakfast/index.js
module.exports = {
plugins: [require("babel-plugin-pancakes")]
};Then you could specify the options for babel-plugin-pancakes like so:
.babelrc
{
"plugins": [["preset-breakfast", {
"pancakes": { "syrup": true }
}]]
}Accessing them from a plugin
export default function ({ Plugin }) {
return new Plugin("pancakes", {
visitor: {
ObjectExpression(node, parent, scope, state) {
if (state.options.syrup !== "maple") {
throw this.errorWithNode("No objects allowed unless I get maple syrup >:(");
}
}
}
});
}Swappable parser
Add a parser option to allow use of another Babel AST compatible parser.
Plugin hooks (#1485)
There's currently no good way to perform initialisation of a plugin. Currently you can use the Program visitor to kinda simulate this, but those callbacks are only called once the traversal has already started. The introduce of a init and post method to a Plugin will make this make nicer:
export default function ({ Plugin }) {
return new Plugin("pancakes", {
init(state) {
// initialise
},
post(state) {
// teardown
}
});
}Way to provide the entire dependency graph to Babel
Another thing that I want to provide is a way to give Babel your entire dependency graph. This will allow cross file transformations. It'll allow us to catch more bugs (such as importing something from another module that wasn't exported).
This also allows us to simplify code (thus making it more performant) with access to other files. For example, we can drop the module interop output if we know that a file imported is ES6.
Minification plugins
I believe we can do minification better (if not equivalent) as UglifyJS. With the potential type information available this has the potential to lead to huge wins. I've already written quite a few plugins already and I believe with some time investment and the help of community plugins we can have a kickass minification pipeline.
Optimisation plugins
Babel plugins put a lot of potential for optimisations on the table. These optimisations fall into two categories:
Unreliable transformations
These transformations modify JavaScript semantics, they're pretty dangerous and fairly irresponsible. If you're a small devshop or independent developer it's probably way too risky to do this but if you're a largish tech company then you can weigh the risks and it may come out in favor of doing these sorts of transforms. The engine can't infer these kind of changes, you have to tell it that you don't care.
Want to assume all function expressions are constant value types and hoist them? Do it via a plugin.
Want to turn all function expressions that don't use this or arguments into arrow functions? Do it via a plugin. (Arrow functions are currently faster than normal functions in Firefox and likely all other browsers in the future)
Want to manually inline functions, removing the overhead of adding a frame to the call stack, potentially improving perf even if the function has been JIT inlined? Do it via a plugin.
Want to lazy load modules, assuming that the order they're imported doesn't matter as they're side effect free? Do it via a plugin.
You can kinda see how these assumptions can lead to fairly large wins. You just have to decide whether or not it's a good idea to do them.
Reliable transformations
These are transformations that can be done with 100% confidence. These can be performed without possibly breaking any code. With Flow integration and access to the dependency graph a lot of code can be statically inferrable.
But don't engines already do these? Different engines do different things. Some optimisations are too heavy for JITs to do but we have a bit more leeway since we're preprocessing offline. We also potentially have access to more information than engines (type annotations) and so can do some smarter things.
Debugging
Better debugging information will be absolutley necessary as Babel grows in complexity. A particular useful feature for userland would the ability to tell exactly what transforms are breaking or slowing down your build will be critical in this modulrisation.
Performance
5.x performance has improved drastically since 4.x. It could be better though. Performance will never stop being an issue.
If there's one part of Babel that is quite slow due to the overhead of JavaScript, there's the potential to provide two implementations of it. A fast version implemented via a native module that's used in Node and a pure JavaScript one that can be used in the browser. If parts of Babel get implemented as native modules there will always be a pure JavaScript version available.
See issue #1486 for related performance discussion.
Async API
Why?
This will allow an RPC with a long running server such as Flow. This will allow access to a large amount of type info. This will also allow IO to be done.
What about integrations that only allow a synchronous API?
A synchronous API will still be available for those limited integrations. If you attempt to use the synchronous API with plugins that have async visitors then an error will be thrown. Most of the systems that Babel has to integrate with are already async so this shouldn't be a concern for 90% of developers.
Anything else?
Probably forgetting something, I'll add more to this issue as I remember and feedback is collected.
cc everyone
This all sounds ace! it would be cool to bake-in per-plugin perf profiling from the start, so slow plugins can be identified & optimised/disabled.
I am sad to see this direction (no more easy transformations by default), probably because I was doing the same with cssnext, but at the same time I totally understand the "Why".
It's nice to see that Babel will more look like PostCSS. These 2 projects didn't start with the same goal but are more and more similar (parsers/transformers).
Your approach looks good to me!
@MoOx I've thought about how to make it the easiest and the current solution is basically what I landed on. All that will be required to get 5.x behaviour is having the following .babelrc:
{
"plugins": ["preset-es2015"]
}(Preset might be called something different but you get the point)
@glenjamin Oh that'd definently be cool.
You shouldn't require extra configuration just to get Babel's bread and butter (ES transpilation). I understand your vision for Babel, and it's great, but please keep it simple for the common use-case.
All the future-future stuff sounds super exciting!
@sebmck Yeah that's why said I like the approach. Thanks for your awesome work. Good direction.
@brianblakely Trying to keep it as simple as possible but keeping any transforms enabled by default isn't very proactive.
Forgot to mention that I'm also putting a feature freeze on 5.x
You could have a generator/cli as the primary method for installation, so that doing
babel-cli es2015Would npm install --save-dev babel babel-preset-es2015 and generate the .babelrc
Having a preset for every subsequent version of ES (2015, 2016, etc) could get confusing to manage in a couple years. Perhaps it would be more enabling to relegate that stuff to an "esnext" preset instead, which makes Babel work for users the way it does today. Then you just update the preset on NPM and never need to touch your RC again.
@brianblakely In the next couple of years people are going to be targetting completely different things depending on their requirements. Having a blanket version doesn't really work, especially when ES versions aren't going to mean anything anymore.
I think the majority are still going to want to work with the latest ES features. Installing language features a la carte or looking up which preset to install is not a great experience for those users or their teams. Babel is just one tool of many, and should strive to be as zero-config as possible.
Do you not think making Babel much much more powerful is a worthy tradeoff for literally 1 line of config?
I think the majority are still going to want to work with the latest ES features.
I don't believe this is true at all.
The current babel users I would expect are largely made up of early adopters and those who happen to be in the same teams / on the same projects as early adopters.
Using experimental babel features is not dissimilar to using coffee-script, it's a reasonable choice, but not one that the mainstream will definitely latch on to.
I was thinking like you @brianblakely (I speak as the creator of "cssnext"), but I have to admit that handling the evolution of specs can be a pain, and can easily become a dangerous game (it's even harder for CSS because specs are exploded as different modules...).
I added some gifs
gif* :(
In all seriousness, future looks great! Looking forward to these changes.
As another data-point, another popular JS tool with a large set of configuration options recently switched to off-by-default - and survived.
Nailed it well done 👍👍👍👍
@glenjamin everything off !== everything out ;)
Yeah, JSCS has always had no options set by default; some wanted a certain preset (airbnb) to be default for 2.0.0 but didn't get enough support.
The current babel users I would expect are largely made up of early adopters and those who happen to be in the same teams / on the same projects as early adopters.
In the last year I've been the early adopter teaching ES6 to non-early teams via Babel.
ESLint's "extends": "eslint:recommended" option is a really decent solution. My "preset-esnext" idea was kind of similar--Babel curates/updates a list of new ES language features as time goes on for the benefit of its users. Perhaps a lot to ask, as per @MoOx's comment.
I want to cut off this discussion about what should be included in Babel by default as it's not been productive so far.
Babel is a compiler. At it's heart it is a system for building transforms that take JavaScript in and spit JavaScript back out, and in between it can run all sorts of really complex operations on JavaScript. Babel has a lot built into it for making this easy.
There are many applications for a compiler: transpiling, minifying, linting, type checking, etc. it's a very useful tool to have in many contexts.
So to assert that it should have some default behavior as the "common use case" is misguided. Yes, transpiling of ES-next features will always be a supported by the Babel team, it's not necessarily the one it will be used for.
Babel is meant for building an ES2015 transpiler (not being one), as well as many other tools. That's the use case Babel is going for and that's why this decision is being made.
This sounds like a great direction. I agree with the sentiment of making the common case as easy as possible, but also strongly agree with documenting exactly what transforms a library requires. With the ability to put babel config into package.json, we end up neatly describing important characteristics of the source.
but also strongly agree with documenting exactly what transforms a library requires. With the ability to put babel config into package.json
This touches on something that I've been wanting to discuss. (Movie trailer voice) In a world where developers put non-compiled code on npm, do we want the ability to compile all of that down in Babel?* In other words, should a module bundler using Babel be able to require files that need different transforms based on the module being loaded?
My understanding is that Browserify can do this via package.json (via the browser and browserify fields) although I'm not sure how it handles loading transforms listed in devDependencies if they don't get installed.
Should .babelrc's be somehow required for distribution?
* Example use case: I wrote my library with a bunch of ES2015 features, rather than compile it forever I'm going to distribute the ES2015 version and have a .babelrc file for people who want to import it with a Babel module loader.
Babel is meant for building an ES2015 transpiler (not being one), as well as many other tools. That's the use case Babel is going for and that's why this decision is being made.
+1000 for that, a really succinct way of putting it that explains this new direction. To me this sounds like a fantastic approach and one I'm really looking forward to working with.
@thejameskyle I'd be really interested in being able to publish "raw" code rather than code that's been generated by Babel.
I wrote my library with a bunch of ES2015 features, rather than compile it forever I'm going to distribute the ES2015 version and have a .babelrc file for people who want to import it with a Babel module loader.
Not sure if .babelrc is better than options in a package.json but that's definitely a latter discussion, I'd be really keen to see and explore how this approach might work. Running some Babel task in an npm postinstall kills me a bit inside ;)
Edit: you can use package.json, as @sebmck pointed out. I stand very much corrected :)
@jackfranklin You can specify your .babelrc file via package.json already which is what I believe @leebyron was referring to.
@sebmck well, please ignore everything I say, ever. Misunderstood, thanks for clarifying 👍
This sounds great! My first thought about making Babel aware of the entire dependency graph was the same as @dashed's: I wonder how this overlaps with concord? It's essentially going to mean a custom module format (even if it does piggy-back on package.json) and he's been doing a lot of work on that. As I understand it, Babel is still going to transform code written in a superset of JS into a subset of that language (with transforms operating on "JS+" ASTs) whereas webpack transforms anything to anything (with transforms necessarily operating on strings), but I wonder if there could be some common ground/interop in these formats so that "Babel modules" would Just Work with concord-aware systems.
@matthewwithanm You'll need to preprocess all the files with Babel's API and then read all the dependencies from the processed file and then rinse and repeat. I can't speak to how this would interact with other systems. This would just be a feature that would be integrated into something like Ember or React Native's bundler.
Module bundlers would integrate with Babel, not the other way around. FWIW here is probably what the API would look like:
import * as babel from "babel-core";
var dependencyGraph = {};
var file = babel.process("code", opts);
file.metadata.modules; // read all imports and babel.process them into dependencyGraph
file.transform(dependencyGraph).code;Just so I'm clear, this is not a module format at all. This is just an extra API that module bundlers can use.
These are modules distributed via npm that specify a set of options (plugins etc) that will be merged into your main configuration.
So presets are modules, that export a configuration object, that may or may not specify plugins. If it does export plugins, what does that array contain? Is the idea that you'd list out strings of plugins for babel to resolve on its own (standard plugins like babel-plugin-arrow-functions), and also functions of plugins that the preset itself has?
I guess what I'm getting at is, if there was a hypothetical "future-react" preset for me to use in my project would it look like this?
// /.babelrc
{
"plugins": ["preset-future-react"]
}
// /node_modules/babel-preset-future-react/index.js
module.exports = {
plugins: [
// stuff that lives in:
// /node_modules/babel-preset-future-react/syntax/*
require('./syntax/future-jsx'),
// stuff that comes with babel
// that probably lives in:
// /node_modules/babel/node_modules/babel-plugin-*
'babel-plugin-arrow-functions',
'babel-plugin-classes',
// stuff that lives in:
// /node_modules/babel-preset-future-react/syntax/*
require('./transforms/future-react-jsx')
]
}Plugin options are something that sorely needed.
Plugin hooks
Awesome! What about a sanctioned way for plugins to pass data to other plugins?
@sebmck sounds really awesome! Have you considered using npm's "scoped packages" feature to be able to properly namespace and differentiate "officially" maintained plugins/presets from packages which just happen to begin with babel-?
@babel/plugin-arrow-functionsvs.
babel-plugin-arrow-functions
Will plugins be able to specify the version(s) of babel they are compatible with, a la peerDependencies?
Awesome! What about a sanctioned way for plugins to pass data to other plugins?
So presets are modules, that export a configuration object, that may or may not specify plugins. If it does export
plugins, what does that array contain? Is the idea that you'd list out strings of plugins for babel to resolve on its own (standard plugins likebabel-plugin-arrow-functions), and also functions of plugins that the preset itself has?I guess what I'm getting at is, if there was a hypothetical "future-react" preset for me to use in my project would it look like this?
Nope, it'd look like this:
// /node_modules/babel-preset-future-react/index.js
module.exports = {
plugins: [
// stuff that lives in:
// /node_modules/babel-preset-future-react/syntax/*
require('./syntax/future-jsx'),
require('babel-plugin-arrow-functions'),
require('babel-plugin-classes'),
// stuff that lives in:
// /node_modules/babel-preset-future-react/syntax/*
require('./transforms/future-react-jsx')
]
}There'll be nothing bundled in core.
sounds really awesome! Have you considered using npm's "scoped packages" feature to be able to properly namespace and differentiate "officially" maintained plugins/presets from packages which just happen to begin with
babel-?
That'd be pretty cool, haven't really thought about it. Some people have squatted really generic package names like babel-plugin-jsx which is EXTREMELY annoying and really limits us a lot. I'd afraid that it'd be confusing as you might have @babel/plugin-whatever and babel-plugin-whatever.
Will plugins be able to specify the version(s) of babel they are compatible with, a la peerDependencies?
Haven't really thought about that. Any ideas?
I would caution against making preset modules which are nothing but a collection of dependencies.[1] The convenience they provide is illusory. Invariably consumers will need some combination of the dependencies and that you didn't account for and will wind up manually specifying them or nagging you to change what is included in a preset. Additionally, look forward to the ecosystem being filled with forgotten third-party "preset" modules that which are nothing but a bunch of outdated dependencies. Also, semantic versioning is pretty nonsensical for this type of module. I believe the correct solution to this problem is high-quality documentation.
Pinging @jugglinmike if he has some time to weigh in on how presets have worked out for jshint.
[1] I have a lot of experience maintaining plugins and "collection" plugins (I started grunt-contrib some years ago).
Invariably consumers will need some combination of the dependencies that you didn't account for
Then you'll manually specify each plugin you want.
and will wind up manually specifying them or
Why is this a bad thing?
nagging you to change what is included in a preset.
This shouldn't be an issue. What a preset constitutes is very clear.
Additionally, look forward to the ecosystem being filled with forgotten third-party "preset" modules that which are nothing but a bunch of outdated dependencies.
Isn't this just a consequence of having a plugin ecosystem of any kind? I'm not
Also, semantic versioning is pretty nonsensical for this type of module.
Why? Semantics and transforms change, it can break code that uses the old semantics.
Scoped packages for official babel packages sounds like a good distinction, not sure if there are any downsides to this however (apart from the @ convention thing)
Modularisation surely has to be the right road for a transpilation tool, with everything off by default. Encourage devs to consider which features they are allowing/disallowing, a number of presets will enable the quick way into using babel.
@sebmck No, but I know the npm people (and other package manager folks) have. I do think it will be a very important implementation detail. Without this, people will run into many incompatibility issues with outdated plugins. Worst case, it can be checked at compile time by babel, but installation time would be preferred.
In addition to changing the architecture, this approach will change the Babel's
versioining semantics. Breaking changes/new features/bug fixes will no longer
be directly appreciable by application developers; instead, the version number
of the babel module will primarily concern plugin authors.
I think this reduces the benefit of the Babel/Babylon separation (from a
versioning standpoint, at lease). Babylon's releases will largely dictate
Babel's version. I don't have the context to say if merging makes sense,
though. Probably it doesn't, since there are benefits to decoupling beyond code
publication. But I did want to point that out as it's related to thoughts I've
been having about JSHint and it's integration with parsers.
As for application developers, their focus will shift from Babel version to
plugin versions. That is, until Babel 7 comes out. At that point, application
developers may feel some pain as their plugin dependencies switch to the new
API in staggered fashion; some plugins will depend on 6, others on 7, so they
will have to gate on the lowest common denominator until all the plugins they
rely on have release support for 7. During this time, they'll have to accept
that an increasing number of their dependencies are no longer receiving bug
fixes/new features (especially important for plugins concerning evolving
proposals). That last concern could be mitigated by responsible plugin
maintainers that backport changes to previous version branches, but despite the
general consensus for the "many small modules" approach, I haven't witnessed
many Node.js developers honoring this contract.
Major version releases are ideally few and far between, and it's careful, open
discussion like this that keeps them that way. The situation I've described
above is not ideal, but it should be weighed against the expected frequency of
major versions and the expected number of commonly-used plugins.
@tkellen JSCS has had presets pretty early on and it seems to work alright - users do need to know how to modify/extend a preset to include/exclude transformers/plugins they do/don't want but presets should be set and not change often if at all. Yeah it's true people do make issues to change what's in a preset.
I think this reduces the benefit of the Babel/Babylon separation (from a versioning standpoint, at lease). Babylon's releases will largely dictate Babel's version.
Babel is already a collection of packages. They all operate on one version line that's specified in VERSION. When a new release is done, all the packages are bumped to that version and published, only if they've had changes since their last release. They're already tightly coupled. Babylon exists for Babel usage so a breaking change in Babylon (and thus a major) can't be done since it always has to be compatible with the present Babel release.
At that point, application developers may feel some pain as their plugin dependencies switch to the new API in staggered fashion; some plugins will depend on 6, others on 7, so they will have to gate on the lowest common denominator until all the plugins they rely on have release support for 7.
It's possible that this sort of major version pain can be solved (or at least alleviated) by adding in shims for older (or deprecated) APIs. This is similar to how Ember operates, although these compatibility layers are removed in Ember majors. This however comes with the additional problem of people not upgrading their plugins because they work in latest versions.
That last concern could be mitigated by responsible plugin maintainers that backport changes to previous version branches, but despite the general consensus for the "many small modules" approach, I haven't witnessed many Node.js developers honoring this contract.
I can't particularly speak for other developers but I know that all the official plugins will be kept up to date and backported if necessary (I've done this before for babel-eslint although the plugins would be on a much larger scale). There's already an issue in the current small Babel plugin ecosystem where a lot of the plugins being released are of very poor quality. I'm not quite sure how to tackle that though other than only encouraging the use of "blessed" plugins.
You do bring up a very good point about version and dependency spaghetti though. It's not something that I've considered nearly enough. Not entirely sure how that should be handled but I'm confident that it doesn't (or shouldn't) hinder the modularisation. Thanks for your thoughts @jugglinmike!
Invariably consumers will need some combination of the dependencies that you didn't account for
Then you'll manually specify each plugin you want.
👍 agreed.
and will wind up manually specifying them or
Why is this a bad thing?
This in and of itself isn't a bad thing. The bad thing is the maintenance overhead you will create for yourself (you're already starting down the rabbit hole trying to come up with a way to specify options for preset modules) and the confusion you will create for consumers as they try to navigate the waters of when to use a "preset" plugin vs when to use a plugin directly, and how to configure it properly.
nagging you to change what is included in a preset.
This shouldn't be an issue. What a preset constitutes is very clear.
I would expect it to be clear to you. But it is highly likely that will be unclear and subjective for consumers when they suddenly can't get everything they need from a preset module.
Additionally, look forward to the ecosystem being filled with forgotten third-party "preset" modules that which are nothing but a bunch of outdated dependencies.
Isn't this just a consequence of having a plugin ecosystem of any kind? I'm not
Yes, having outdated plugins is a part of having a plugin ecosystem. Baking in support for "preset" modules which are nothing more than a list of dependencies will compound the problem and confuse users who are searching for just the right preset to fit their needs rather than installing plugins directly.
Also, semantic versioning is pretty nonsensical for this type of module.
Why? Semantics and transforms change, it can break code that uses the old semantics.
With individual plugins it is relatively easy for users to reason about versions. With preset plugins you create a new/meaningless version abstraction on top of multiple plugins, and you'll be bumping it continuously.
Major bump on a single plugin? Major bump on the preset, even though only one part of the preset changed. You might have clear visibility into why that happened but I promise a large swath of your consumers won't. Maybe you'll decide to avoid the constant version bumping and coordinate releases of the preset. Now you'll get users who want the latest version and will complain that the preset isn't up to date. Of course you can tell them to use the plugins directly, but you can avoid this mess if you just have them do that in the first place.
I could go on at length about the pitfalls of this approach if you're not convinced/want to know more.
@tkellen JSCS has had presets pretty early on and it seems to work alright - users do need to know how to modify/extend a preset to include/exclude transformers/plugins they do/don't want but presets should be set and not change often if at all. Yeah it's true people do make issues to change what's in a preset.
What JSCS does is quite different than what @sebmck is suggesting. JSCS has all of the needed code baked in to use a preset.
One other point I'll make is that you're essentially dipping your toes into creating a package manager with the idea of preset modules. Can you use a preset module inside another preset module? If no, why not? If yes, compound all the versioning issues I've mentioned even further. Also, how would you configure options for that?
PS: I am wildly appreciative of the work you do on babel, I use it extensively!
One other point I'll make is that you're essentially dipping your toes into creating a package manager with the idea of preset modules.
My understanding is that a preset module is just a plugin that is a collection of other plugins - this doesn't seem to be materially different to an end-user as any other plugin. The author takes on some responsibility to provide a bunch of functionality as needed.
I would expect that the most popular presets would change fairly infrequently, with most of the churn/experimental plugins not being part of the most common presets.
@tkellen Would most of your concerns still exist if presets were generators for babel config + added dependencies rather than first-class plugins?
This in and of itself isn't a bad thing. The bad thing is the maintenance overhead you will create for yourself (you're already starting down the rabbit hole trying to come up with a way to specify options for preset modules) and the confusion you will create for users as they try to navigate the waters of when to use a "preset" plugin vs when to use a plugin directly, and how to configure it properly.
We're having this discussion to iron out any potential issues. It may eventually come down to it that you just can't specify options for presets, or that presets aren't a very good idea at all (however an alternative HAS to be sought).
it is highly likely that will be unclear and subjective for consumers when they suddenly can't get everything they need from a preset package.
To the end user a preset looks and behaves just like a normal plugin.
Yes, having outdated plugins is a part of having a plugin ecosystem. Baking in support for "preset" plugins which are nothing more than a list of dependencies will compound the problem and confuse users who are searching for just the right preset to fit their needs.
With individual plugins it is relatively easy for users to reason about versions. With preset plugins you create a new/meaningless version abstraction on top of multiple plugins, and you'll be bumping it continuously.
Presets will just be a conglomerate of stable plugins. It's not as if the major version will need to be bumped every week.
One other point I'll make is that you're essentially dipping your toes into creating a package manager with the idea of preset modules.
Not really? Presets literally just merge two sets of options together. A preset can literally specify any of the Babel options.
Can you use a preset module inside another preset module? If no, why not? If yes, compound all the versioning issues I've mentioned even further. Also, how would you configure options for that?
Sure, I don't see why not. Just because you can doesn't mean you should.
My question to everyone against presets is this. What do you suggest as an alternative? Manually specifying each individual plugin is NOT a proactive solution. Let's say I want to use Babel for minification... okay, I have to now find a bunch of plugins. Alright so I need merge-sibling-variables, simplify-comparison-operators, dead-code-elimination, minify-booleans and ugh... what else am I missing? Oh right, constant-folding. Oh wait, I'm forgetting... This isn't a very good experience to go through, spending hours configuring your tooling is crap.
Other tools suffer from configuration hell. A big criticism (whether valid or not) about Webpack for example is the amount of config you typically have. DIY tools aren't very easy to use, offering these preconfigured configs is a good solution to this problem IMO. I agree that it's not flawless but it's better than any of the alternatives (at least the ones that I can think of!)
@tkellen Would most of your concerns
still existgo away if presets were generators for babel config + added dependencies rather than first-class plugins?
Having presets be sugar for babel config rather than first class modules would alleviate nearly all of the pitfalls I have outlined.
Having presets be sugar for babel config rather than first class modules would alleviate nearly all of the pitfalls I have outlined.
I'm not sure what you mean by this, can you elaborate? AFAIK this is already the case? (Taken from the OP)
These [presets] are modules distributed via npm that specify a set of options (plugins etc) that will be merged into your main configuration.
I'm not sure what you mean by this, can you elaborate? AFAIK this is already the case? (Taken from the OP)
Sure!
What @glenjamin suggested is this:
{
"plugins": ["cookies"]
}
...expands to:
{
"plugins": [
"flour",
"baking-soda",
"salt",
"butter",
"sugar",
"brown-sugar",
"eggs",
"chocolate chips"
]
}
Say you've installed only salt and butter. If you try to run babel with that preset, an error is thrown that tells you to run npm install flour baking-soda sugar brown-sugar eggs chocolate-chips to satisfy the preset.
So if someone wants to setup an ES2015 environment (AKA use Babel 6.x like they did 5.x) they'd have to do:
$ npm install babel # install babel
$ npm install babel-preset-es2015 # install the preset
$ npm install
babel-plugin-es2015-arrow-functions babel-plugin-es2015-block-scoping babel-plugin-es2015-classes babel-plugin-es2015-constants babel-plugin-es2015-destructuring babel-plugin-es2015-for-of babel-plugin-es2015-literals babel-plugin-es2015-modules babel-plugin-es2015-object-super babel-plugin-es2015-parameters babel-plugin-es2015-properties-computed babel-plugin-es2015-properties-shorthand babel-plugin-es2015-regex-sticky babel-plugin-es2015-regex-unicode babel-plugin-es2015-spread babel-plugin-es2015-tail-call babel-plugin-es2015-template-literals # install the plugins the preset uses...Not the nicest setup. This would also mean adding new plugins to a preset would be a breaking change.
Not quite what I had in mind.
The original suggestion is that presets are plugins, and thus get added to babel config & package.json
My suggestion is that presets are cli tools (or managed via a single cli tool), and "using a preset" does a one-off task which makes the relevant changes to .babelrc and package.json.
The workflow would be something like:
npm install babel
./node_modules/.bin/babel --preset es2015
# babel off goes and finds out how to expand this somehow
# .babelrc is generated / modified to include list of plugins
# package.json is modified to include plugins as dependencies
npm install
# installs all the new stuffI'm not sure if I like this, but it seems like it could be a viable option.
@sebmck I'm suggesting that you bake preset-expansion into babel itself and not have preset packages at all. If you're really concerned about one large npm install command, you could add babel install <preset> and have it run npm for the user.
@tkellen Oh cool, that's an interesting alternative. I'd worry a bit about things not being very DRY but maybe it's worth it.
What about plugin's dependencies? For example regenerator depends on let/const transformer, cause if i will use regenerator with native let/const - they will have broken inside of generator function. For now i should turn ALL transformers on, or ALL off. Tight coupling of transformers is the worst side of babel for now. Yesterday was released iojs 3.o with full native support of classes, with better support than babel, but i can turn off es6.classes transformer cause class decorators will have broken and may be some another transformers. I think removing of tight coupling (if its possible) should be in priority.
@PinkaminaDianePie Easier said than done. Some transformers just need to do complete sugaring to retain correct semantics. Partial transforms are not easy.
Sorry to be annoying with PostCSS, but the exact same issue is happening so I will just throw out my experience just to let you know (I am not really smart like most of you here - so I don't know how to make the situation better, but I guess the resolution of this discussion will help).
PostCSS is already what Babel will become: a parser+transformer engine (for css). It consume plugins to make transformation to the AST.
Currently there is a lot of plugins in the ecosytem and there is also what they call (shitty name) "plugin packs" (= preset).
I created cssnext as an tool on top of PostCSS, but the way PostCSS has evolved allow user to consume cssnext as a plugin (internally, postcss can consume another postcss instance to collect registered plugins).
This allows user to consume a collection of plugins as a plugin. For example for minification, there is the tool "cssnano" (which have it's own interface, like cssnext - that have it's own cli, grunt, gulp and * plugins to run cssnano). The way postcss is made makes that cssnano can be user directly has a plugin if you play directly with PostCSS.
Like @tkellen is explaining, this situation is weird. The ecosystem is confusing and I really hate that.
Some people thing cssnext is a tool (that's what I wanted in the first place), but some use it as a plugin (and most of the time, they do not use it the way they should, because cssnext should be run before others plugins (as it's inlining @import to be able to apply transformation on every files - yeah css don't have npm or require()), but some people don't think and/or read the doc (yeah that's another problem) so after that they ask question or have unexpected behavior as their plugins list is not working as expected.
We have the some duplicates (cssnext use autoprefixer, as it was designed to be a final tool) but some people add autoprefixer too in there list (guess what, they didn't read first line of the doc -- it's even written in the first screen of the homepage...) and this is stupid.
cssnext use cssnano for minification (a plugin pack use another plugin pack), so we have a sort of pluginception (but since postcss flatten plugins list, it just create a "duplicate" issue). In order to optimize that, cssnano for example have used a technique that detect plugin that have already been used to not reuse them again. It's like bad design to me.
So maybe preset should be tools with they own interface. And preset should not be able to be consumed directly by Babel. This will prevent bump/versions issues.
In order to do that, I recently created some modules:
- one to a .rc file (so any tool can easily load a .rc (YAML/JSON/JS) file with a line of code) https://github.com/MoOx/rc-loader
- another (that is not ready but almost) that allow to very easily create a cli tool to run a transformer (I just extracted cssnext cli (that have a watcher based on chokidar) and it will help (exactly like babel-cli) to transform one or multiple files.
These two modules might be easily used with a very small boilerplate to create preset as tool. Not sure it's the solution but it's one to think about. Even if I am not a fan (I didn't find, even while reading this all thread, the holy graal)...
So maybe preset should be tools with they own interface. And preset should not be able to be consumed directly by Babel.
I'm not sure what you mean? Do you mean making an entirely different project that's just a wrapper around Babel?
Yeah. That's how some PostCSS plugin packs are made atm. Reminder: I am not saying it's THE solution.
You're just forking your own ecosystem and it leads to a lot of duplicated work, you'd need additional integrations (you'll need a gulp, browserify, grunt, webpack etc plugins) and marketing (which is already extremely difficult).
I know, that why I said I am not a fan of this solution and following carefully this thread.
But atm, PostCSS doesn't have an official cli tool or a way to declare configuration easily (like a .babelrc), but that's another story I hope to fix soon before deprecating official interface of cssnext, since postcss is becoming popular these days.
I see how presets are useful for the basic scenarios. But I'm totally missing the point how I can customize presets once I need a small step away. Here is the real life scenario: I'm using all es6 transformers for the server code in my project, meanwhile for the client part of my project I've blacklisted transformers which requires runtime and regenerator (since it will be a huge size overhead for the client lib). Currently I just use .eslintrc in corresponding dirs which is clear and beatiful. How I will be able to achieve this behavior with presets? Create packages for the client and server presets and publish them to NPM?
@inikulin I think the point would be that a preset is a starting point for your own config - similar to .eslintrc
// .babelrc
{
"plugins": [
"preset-es2015",
require("babel-plugin-flow"), // add a transform
// and a way to blacklist/remove a transform that's included in a preset
]
}and if you have a lot of projects that you want the same transforms for you can publish a package and use that instead.
@sebmck but you want to build full dependency graph! You can use it to decrease transformers coupling if you will tell each transformer about which other transformers are in use. For example if es7 decorators transformer know what native classes are in use - it will transpile decorators to one code, but if transpiled classes are in use - to another. This metod can not prevent all problems with tight coupling, but it can help with many transformers anyway. For now iojs supports about 44% of es6 features, but i can not turn off no one transformer in babel.
@PinkaminaDianePie You still need to do full desugaring to retain correct semantics and to ensure that each piece of evaluated correctly. You can't move a decorator to before a computed property etc as it may have side effects. This is harder than it seems and it's what I meant by partial transforms (that do the whole desugaring).
// and a way to blacklist/remove a transform that's included in a preset
Which, if I get it right, will not be present in the new version.
@sebmck but what will you do with that? Nothing? It will be sad. It will be more sad if we still need to use full list of transformers to es5, in enviroment with 75%+ es6 support. I hope that splitting babel's core to independent transformers in separate repos will be first part to decrease tight coupling. And it will be much easier to understand single transformer so it should help with pull requests from community. Im waiting this for finally fix my old issue with native generators myself =D
but what will you do with that? Nothing? It will be sad.
I haven't said anything is final.
I hope that splitting babel's core to independent transformers in separate repos will be first part to decrease tight coupling.
There's no tight coupling between most of the transformers, it's only where this desugaring is necessary where you encounter it.
And it will be much easier to understand single transformer so it should help with pull requests from community.
This isn't really the primary motivation and I'm skeptical that this will help much. The same complexity will still exist, it will just be spread out more, which makes it even more complex in and of itself.
Im waiting this for finally fix my old issue with native generators myself =D
If you can't figure out how to make changes in the existing codebase, then splitting everything up will make it even harder.
I'm starting to think the CLI to manage config is the correct approach but I can't help but feel it adds a lot more complexity to using Babel. Anyone care to weigh in?
@sebmck What if you'll make presets configurable and each package within preset will have alias, e.g.:
{
"plugins": [
[
"preset-es2015",
{
"whitelist": [
"es6.arrowFunctions",
"es6.classes",
"es6.properties.computed",
"es6.properties.shorthand",
"es6.constants",
"es6.blockScoping",
"es6.destructuring",
"es6.modules",
"es6.parameters",
"es6.templateLiterals",
"strict"
],
"blacklist": [
"regenerator",
"runtime"
]
}
]
]
}You will be able to proceed with the modular approach, but keep familiar user experience. (Config looks ugly, but it's a subject for debate anyway). And you will not need any kind of CLI tool to deal with it.
@sebmck Well OK, you can remove blacklist and whitelist then. Disable everything in the preset by default and let people specify transformers that they want to use (and you can get rid of prefix, since preset name is already some kind of prefx:
{
"plugins": [
[
"preset-es2015",
[
"arrowFunctions",
"classes",
"properties.computed",
"properties.shorthand",
"constants",
"blockScoping",
"destructuring",
"modules",
"parameters",
"templateLiterals",
"strict"
]
]
]
}In this case you can get rid of the preset- prefix. Just es2015 plugin:
{
"plugins": [
[
"es2015",
[
"arrowFunctions",
"classes",
"properties.computed",
"properties.shorthand",
"constants",
"blockScoping",
"destructuring",
"modules",
"parameters",
"templateLiterals",
"strict"
]
]
]
}Enable all transformers in the preset:
{
"plugins": [
["es2015", "*"]
]
}Alternatively:
{
"plugins": ["es2015"]
}If you have to specify all the transformers you need then you might as well just specify them without the preset concept at all right?
I'm starting to think the CLI to manage config is the correct approach ....
This approach, babel install <preset>, reminds me of what docpad does: http://docpad.org/docs/plugins
After some digging, the reasons for this seem to stem on npm and peer dependencies: docpad/docpad#927 (comment)
I'm hoping the CLI would still be optional.
@hzoo Nope. At first you will need to npm install all of them, 11 installs vs 1 for my example. Then you will need to jump over the packages to find out documentation for each of them, instead of using one hub package with all the stuff you need.
Oh ok you're saying the preset npm installs all of the transforms but you specify the ones you want
Right. Wasn't it the purpose of the presets and I'm missing something?
Why don't we use the following naming for packages?
require('@babel/core')require('@babel/preset-es2015')require('@babel/plugin-arrow-functions')
It's easier to understand what's official and what's not.
@teugen That was already bought up by @tgriesser here:
Have you considered using npm's "scoped packages" feature to be able to properly namespace and differentiate "officially" maintained plugins/presets from packages which just happen to begin with
babel-?
@sebmck peerDependencies will be deprecated in npm 3.0, so no point in taking that into consideration.
I would really like to see more use of scoped packages in projects like babel which target a very large audience.
Really like the roadmap, would be a lot easier to get involved in fixing bugs for specific plugins. First time I tried to wrap my head around babel's source was very overwhelming.
peerDependencies will be deprecated in npm 3.0, so no point in taking that into consideration
I'm pretty sure this is no longer the case, peerDeps are staying, but will no longer auto-install, they'll just tell you what to add to your normal dependencies.
My understanding is that Browserify can do this via
package.json(via thebrowserandbrowserifyfields) although I'm not sure how it handles loading transforms listed in devDependencies if they don't get installed.
It doesn't handle that. Per the README:
Make sure to add transforms to your package.json dependencies field.
That last concern could be mitigated by responsible plugin
maintainers that backport changes to previous version branches, but despite the
general consensus for the "many small modules" approach, I haven't witnessed
many Node.js developers honoring this contract.
That's a contract?
I don't have a solution to propose right now for presets, but I'm not enthused about a solution that requires a CLI.
@sebmck @thejameskyle Since the subject of presets has so far dominated the discussion here, and there still seems to be a lot more to discuss, do you think it makes sense to try to direct future discussion to #1827?
As long as the CLI and require hook both stay useful and easy to use I think these changes are all great.
Keeping the code better abstracted for long term use is great but it shouldn't compromise the interface to the code for users.
I like the idea behind rollup (Next-generation ES6 module bundler), would this be considered going forward?
peerDependencies will be deprecated in npm 3.0, so no point in taking that into consideration
I'm pretty sure this is no longer the case, peerDeps are staying, but will no longer auto-install, they'll just tell you what to add to your normal dependencies.
correct
Way to provide the entire dependency graph to Babel
If this is what I think it is, I'm very excited for it. I'd love to be able to have the bundler / module formatters be able to use custom resolution algorithms(i.e. npm, bower, etc) and it sounds like this would enable that?
No internal transformers. None. Zero. Zilch.
awesome. 👍 😄
+1000. I ❤️ it! 👍 😄
peerDependencies have never been deprecated, it's always been just a case of the behaviour changing slightly. All it means is that if your plugins peer depend on babel, you will still need to install babel yourself. npm will warn that you are missing the dependency rather than installing it for you.
Please refrain from 👍 it's hard enough already to filter through the babel/babel repo noise.