A sublime-syntax language definition for Ecmascript / Javascript / ES6 / ES2015 / Babel or what have you.
Although a fair amount of work remains before I’m ready to call this ‘1.0.0’, the definition and first theme (Excelsior) are now robust enough to consider this ‘released.’
Sublime syntax is only currently only available in Sublime Text’s dev channel.
New in 0.3
- JSX support
- Adds source.js at root to increase compatibility with other packages
New in 0.2
- ‘Villa-Lobos’ theme
- Async functions/methods/arrows are fully scoped the way generators are
- Promise and its methods are uniquely scoped
- Added ‘todo’ scope per http://www.sublimetext.com/forum/viewtopic.php?f=2&t=18882
- Various minor fixes, like allowing multiple spreads in an array literal
- Remaining Work
- Why Bother
- Feature Coverage
- SublimeText Symbols
- Themes
- Scopes
- About the Scope Conventions
- More Infobarf for Someone Hypothetically Curious About the Definition Itself
- I’d like to provide more themes with the definition package itself. The first of these is ‘Excelsior,’ a kind of tacky attempt to demonstrate what can be done that I’ve grown very fond of as it matured. It’s (mostly) complete now. Still, it’s probably a bit much for many users.
- The chunk of comment-delimited code starting at
assignmentExpression_NO_IN
is actually possible to generate as a build step. Doing so will reduce the chances of accidentally editingassignmentExpression
without making the corresponding change inassignmentExpression_NO_IN
. I’m not sure if this is worthwhile yet but it’s rolling around in my head. - A lot of the redundant-looking contexts exist for good reasons -- but not all of them. There should be a clean-up and consolidation phase for contexts, but more importantly, for patterns (or more often, pattern components).
There are at least three tmLanguage syntax definitions available for Javascript right now, and two support ES6+: JSNext and Babel Sublime (which I think is a fork of the former). Since they’re actually great already, I should explain what the point of this fourth one is.
Sublime syntax is a brand new YAML syntax definition system that was introduced in Sublime Text dev build 3084 in April of 2015. Previously the only definition system Sublime supported was tmLanguage, originally from TextMate. In addition to cutting out the common build step of YAML -> heinous XML, Sublime syntax introduced a ‘context’ system for controlling what kinds of matches could take place at a given juncture.
Contexts exist in a FILO stack. They can be pushed, popped, or ‘set’ (pop-push). As in tmLanguage, matching cannot look past the current line, but using contexts bypasses some of the old consequences of that constraint, allowing more advanced and accurate scope assignment.
The number one reason why I made this is that it bothered me that many disparate
elements of the grammar that just happened to look similar had to share generic
scopes like meta.brace.curly
. Thanks to Sublime syntax, Ecmascript Sublime
doesn’t have to have scopes that describe what something looks like. In the
case of braces, instead you will find scopes for block statements, object
literals, object binding patterns, class and function bodies, namespace imports,
etc -- the things that the braces mean.
It goes a bit beyond that, too. If you like, you can color if statements differently from loop statements for example. Generators, methods, and accessors are also individuated. In fact there are far more very specific scopes available than anyone would ever reasonably use, but the point is to allow the theme designer to choose exactly which elements share colors (most good themes seem to have relatively small palettes, really).
However there are good reasons one might prefer one of the other choices for syntax highlighting. For one, maybe you like a theme that plays better with one of the others. There are also differences with the approaches taken that may or may not suit your style. And finally, Ecmascript Sublime is a lot more rigid by nature, so invalid token sequences will not often pass unnoticed -- you may find this useful, or you may find it bothersome.
Here’s an early shot of Excelsior. I should add an updated one soon:
And here’s an example I’m fond of, using background colors to indicate the depth of matching groups in a regular expression:
In addition to everything in the final draft of ES6, I’ve included support for
certain ES7 strawmen that have already been implemented in Babel or Firefox. The
depth is not always as great as with established language features, though.
For example, async/await keywords are supported, but I didn’t go whole hog by
making the whole async function targettable. Once async is formalized -- the
grammar is still up in the air -- then it will be implemented thoroughly like
generators. Done.
The following candidate ES7 features, as they stand today anyway, are covered:
- async / await (including methods, expressions, and arrows)
- generator comprehensions
- array comprehensions, but only as implemented by Babel and Mozilla. (Which differs from the current spec, as I understand it.)
- exponentiation and mallet operators
- class and method decorators
- trailing commas where they don’t belong at all ugh
- object literal spread and object binding pattern rest
- bind operator
- do expression
There are two optional ES7 Babel transforms I haven’t done yet:
- class properties
- export extensions
To be honest I don’t think class properties has a very great chance of entering
ES7 in its current form, so I’m hesitant to implement it yet. Class syntax is
a subset of object literal syntax + allowance of the static
keyword; the
equals-sign syntax (as opposed to a more obvious colon syntax) seems like an
unlikely departure. Plus, they’re totally useless language cruft. If someone
wants it though let me know and I’ll add it; barring that, this will wait for a
more mature spec if it emerges.
Export extensions is more immediately deserving but the spec has not yet added a grammar section. Waiting for that.
I didn’t include JSX. If you use JSX, you should use Babel Sublime, which handles that in depth. I’m not opposed to adding it, but since I don’t use it myself it wasn’t a priority. (Pull reqs welcome.)
A Sublime syntax definition involves more than the definition proper. These other items use the tmPreferences format to describe how automatic indentation works, as well as how the "symbol list" is populated.
If you haven’t used Sublime symbol navigation before, I recommend checking it out. It’s a quick way to navigate the local file or all open files by keyboard shortcut (local is ctrl+r, global is shift+ctrl+r).
In Ecmascript Sublime, function, generator and class declarations will show up in Sublime’s symbol list, as well as default exports. It will also pick up function, generator, and class expressions that either have names or are directly assigned as part of a var, let or const statement. In this last case it will display the name of the variable the expression was bound to, and this can include arrow functions.
Note that only expressions that appear as the initializer in a variable or
constant declaration will be added to the list, so const x = () => {}
is added
as ‘arrow x,’ but x = () => {}
is not.
All of these are local-only except classes. If people want the others to be global (except default exports obviously), we can change that. This is what made sense to me, but I pretty much only ever use the local list myself so I wasn’t sure what habitual global users would like to see.
The first theme, Excelsior, is finished but for continued tweaking. At present there are three total:
- Excelsior
- Villa-Lobos
- Carthage
The third of those is particularly weak, and none is as clean as someone with better self-control would have it.
As of version 0.1.2, I’ve also added many additional scopes and tweaked a number of things to increase interoperability with existing themes. Monokai, Cobalt, and Brogrammer were my reference themes; the goal was to make them match as closely as is reasonable to how they look when using Babel Sublime. There’s a handful of things that cannot be made to match because of what I guess could be called philosophical differences, but it should now be possible to use Ecmascript Sublime with just about any theme without cringing.
I should note, there is another theme included with the package. It isn’t a real theme though -- it assigns random colors to hundreds of scopes. It is useful for referencing when designing themes but you’d be insane to actually use it.
If you’re interested in adding support for Ecmascript Sublime to your theme, or are developing a new theme with this in mind, you’ll probably want a list of the targettable scopes...
It goes without saying that only a minority of these will be truly useful to most themers; on the other hand, you get to decide which those are.
These are the scopes generally intended for targeting. For readability, I’ve omitted the ‘.es’ suffix as well as ‘.begin.es’ and ‘.end.es’ for various punctuation, as these are only useful for debugging. Note that the scopes ending in ‘.regexp’ do not also have ‘.es’.
- Comments & Directives
comment
comment.block
comment.line
comment.line.shebang
(e.g.#!/usr/bin/env node
)meta.comment.body
(content that is not part of the delimiter(s) or border)meta.comment.border
(includes anything that seems like a border, and//
)meta.comment.box-drawing
(box drawing characters can be targeted)meta.directive.use-strict
(the'use strict'
directive)punctuation.definition.comment
(the delimiting//
,/*
or*/
)
- Comprehensions
keyword.control.loop.conditional.if.comprehension.array
keyword.control.loop.conditional.if.comprehension.generator
keyword.control.loop.for.comprehension.array
keyword.control.loop.for.comprehension.generator
keyword.control.loop.of.comprehension.array
keyword.control.loop.of.comprehension.generator
punctuation.definition.comprehension.array
punctuation.definition.comprehension.generator
punctuation.definition.expression.conditional.comprehension.array
punctuation.definition.expression.conditional.comprehension.generator
punctuation.definition.expression.loop.comprehension.array
punctuation.definition.expression.loop.comprehension.generator
- Constants & Literals
- General
constant
constant.language
constant.language.boolean
constant.language.boolean.false
constant.language.boolean.true
constant.language.null
constant.language.undefined
- Arrays
punctuation.definition.array
punctuation.separator.array-element
- Numbers
constant.language.infinity
constant.language.nan
constant.numeric
constant.numeric.binary
constant.numeric.decimal
constant.numeric.hexadecimal
constant.numeric.octal
meta.numeric.exponent.digit
meta.numeric.exponent.e
meta.numeric.exponent.sign
meta.numeric.prefix
punctuation.decimal
- Objects
punctuation.definition.object
(the braces inlet x = { a: 1 };
)punctuation.separator.key-value
(the colon inlet x = { a: 1 };
and the equals sign in the class properties proposal)punctuation.separator.object-member
(commas between properties)variable.other.readwrite.property.object-literal
variable.other.readwrite.property.object-literal.allCap
variable.other.readwrite.property.object-literal.initCap
variable.other.readwrite.property.shorthand
variable.other.readwrite.property.shorthand.allCap
variable.other.readwrite.property.shorthand.initCap
variable.other.readwrite.property.shorthand.rest
variable.other.readwrite.property.shorthand.rest.allCap
variable.other.readwrite.property.shorthand.rest.initCap
- see also Functions for accessors and methods
- Regexp
constant.character.escape.control-char.regexp
constant.character.escape.hexadecimal.regexp
constant.character.escape.null.regexp
constant.character.escape.pointless.regexp
constant.character.escape.regexp
constant.character.escape.unicode.regexp
constant.other.character-class.predefined.regexp
constant.other.character-class.set.regexp
keyword.control.anchor.regexp
keyword.operator.negation.regexp
keyword.operator.or.regexp
keyword.operator.quantifier.regexp
keyword.other.back-reference.regexp
meta.group.assertion.negative.regexp
meta.group.assertion.positive.regexp
meta.group.capturing.regexp
meta.group.non-capturing.regexp
punctuation.definition.assertion.negative.regexp
punctuation.definition.assertion.positive.regexp
punctuation.definition.character-class.dash.regexp
punctuation.definition.character-class.regexp
punctuation.definition.string.regexp
punctuation.definition.group.capturing.regexp
punctuation.definition.group.non-capturing.regexp
string.regexp
string.regexp.flags
- Strings
constant.character
constant.character.escape
constant.character.escape.hexadecimal
(e.g.'\\x41'
)constant.character.escape.newline
(a terminal backslash)constant.character.escape.null
(i.e.'\\0'
)constant.character.escape.pointless
(an escape that is not needed)constant.character.escape.unicode
(e.g.'\\u0041'
or'\\u{41}'
)punctuation.definition.string
punctuation.definition.string.interpolated
punctuation.definition.string.interpolated.element
punctuation.definition.string.quoted
punctuation.definition.string.quoted.double
punctuation.definition.string.quoted.double.parameter
punctuation.definition.string.quoted.single
punctuation.definition.string.quoted.single.parameter
string
string.interpolated
string.quoted
string.quoted.double
string.quoted.single
- General
- Functions & Function-Related
- General
- Parameters
entity.other.property-binding.parameter
keyword.other.rest.parameter
keyword.other.rest
punctuation.definition.binding.array.parameter
punctuation.definition.binding.object.parameter
punctuation.separator.array-element.binding.parameter
punctuation.separator.object-member.binding.parameter
punctuation.separator.parameter
punctuation.separator.property-binding.parameter
variable.parameter
variable.parameter.rest
- Execution & Do Expressions
keyword.control.do-expression.do
(the keyword from ES7, not the loop)meta.instantiation
(applied to identifier being instantiated)meta.invocation
(applied to identifier being invoked)punctuation.definition.arguments
(parens in invocation/instantiation)punctuation.definition.block.do-expression
(braces indo
expression)punctuation.separator.argument
(comma in arguments)variable.other.readwrite.tag
(foo infoo`str${ exp }`
)
- Parameters
- Types
- Accessors
entity.name.accessor.get
(name of accessor)entity.name.accessor.set
(name of accessor)keyword.control.flow.return.accessor
(not sure why I gave it its own)punctuation.definition.accessor
punctuation.definition.accessor.body
(braces)punctuation.definition.accessor.parameter
(the param id in aset
)punctuation.definition.parameters.accessor
(parens)storage.modifier.accessor.get
(keywordget
)storage.modifier.accessor.set
(keywordset
)
- Async Functions
entity.name.method.async
(name of async function)keyword.control.flow.await
(keywordawait
)punctuation.definition.function.async
punctuation.definition.function.async.arrow
punctuation.definition.function.async.arrow.body
punctuation.definition.function.async.body
punctuation.definition.parameters.function.async
punctuation.definition.parameters.function.async.arrow
storage.modifier.async
(keywordasync
, general)storage.modifier.async.expression
(keywordasync
, in expression)storage.modifier.async.method
(keywordasync
, in method declaration)
- Classes
entity.name.class
entity.name.constructor
meta.super-expression
punctuation.definition.class.body
punctuation.definition.constructor.body
punctuation.definition.decorator
punctuation.definition.parameters.constructor
punctuation.terminator.property
(es7? class property semicolon)storage.modifier.extends
storage.modifier.static
storage.type.class
storage.type.class.expression
variable.other.readwrite.property.class.es
(es7? class property key)
- Functions
entity.name.function
entity.name.function.allCap
entity.name.function.initCap
entity.name.method
keyword.control.flow.return
punctuation.definition.function
punctuation.definition.function.arrow.body
punctuation.definition.function.body
punctuation.definition.method
punctuation.definition.method.body
punctuation.definition.parameters
punctuation.definition.parameters.function
punctuation.definition.parameters.function.arrow
punctuation.definition.parameters.method
storage.type.function.arrow
storage.type.function
storage.type.function.expression
- Generators
entity.name.function.generator
entity.name.method.generator
keyword.control.flow.yield.iterate
keyword.control.flow.yield
punctuation.definition.generator
punctuation.definition.generator.body
punctuation.definition.method.generator
punctuation.definition.method.generator.body
punctuation.definition.parameters.generator
punctuation.definition.parameters.method.generator
storage.modifier.generator.asterisk
storage.modifier.generator.asterisk.expression
storage.type.function.generator
storage.type.function.generator.expression
- Accessors
- General
- Operators
- Assignment
- General
keyword.operator.assignment
keyword.operator.assignment.conditional
keyword.operator.assignment.conditional.default
(default initializer)keyword.operator.assignment.conditional.mallet
(||=
-- used to be in Babel)keyword.operator.unary.delete
- Augmented
keyword.operator.assignment.augmented
keyword.operator.assignment.augmented.arithmetic
keyword.operator.assignment.augmented.arithmetic.addition
keyword.operator.assignment.augmented.arithmetic.division
keyword.operator.assignment.augmented.arithmetic.exponentiation
keyword.operator.assignment.augmented.arithmetic.modulo
keyword.operator.assignment.augmented.arithmetic.multiplication
keyword.operator.assignment.augmented.arithmetic.subtraction
keyword.operator.assignment.augmented.bitwise
keyword.operator.assignment.augmented.bitwise.logical
keyword.operator.assignment.augmented.bitwise.logical.and
keyword.operator.assignment.augmented.bitwise.logical.or
keyword.operator.assignment.augmented.bitwise.logical.xor
keyword.operator.assignment.augmented.bitwise.shift
keyword.operator.assignment.augmented.bitwise.shift.left
keyword.operator.assignment.augmented.bitwise.shift.right
keyword.operator.assignment.augmented.bitwise.shift.right.unsigned
- General
- Bitwise
keyword.operator.bitwise
keyword.operator.bitwise.logical
keyword.operator.bitwise.logical.and
keyword.operator.bitwise.logical.not
keyword.operator.bitwise.logical.or
keyword.operator.bitwise.logical.xor
keyword.operator.bitwise.shift
keyword.operator.bitwise.shift.left
keyword.operator.bitwise.shift.right
keyword.operator.bitwise.shift.right.unsigned
- Comparison
keyword.operator.comparison
keyword.operator.comparison.equality
keyword.operator.comparison.equality.coercive
keyword.operator.comparison.equality.strict
keyword.operator.comparison.non-equality
keyword.operator.comparison.non-equality.coercive
keyword.operator.comparison.non-equality.strict
keyword.operator.relational
keyword.operator.relational.gt
keyword.operator.relational.gte
keyword.operator.relational.in
keyword.operator.relational.instanceof
keyword.operator.relational.lt
keyword.operator.relational.lte
- Evaluative
keyword.operator.accessor
keyword.operator.bind
keyword.operator.comma
keyword.operator.new
keyword.operator.spread
keyword.operator.ternary
keyword.operator.ternary.else
keyword.operator.ternary.if
keyword.operator.unary
keyword.operator.unary.typeof
keyword.operator.unary.void
- Logical
keyword.operator.logical
keyword.operator.logical.and
keyword.operator.logical.not
keyword.operator.logical.or
meta.idiomatic-cast.boolean
(i.e.!!val
)
- Mathematic
keyword.operator.arithmetic
keyword.operator.arithmetic.addition
keyword.operator.arithmetic.decrement
keyword.operator.arithmetic.decrement.postfix
keyword.operator.arithmetic.decrement.prefix
keyword.operator.arithmetic.division
keyword.operator.arithmetic.exponentiation
keyword.operator.arithmetic.increment
keyword.operator.arithmetic.increment.postfix
keyword.operator.arithmetic.increment.prefix
keyword.operator.arithmetic.modulo
keyword.operator.arithmetic.multiplication
keyword.operator.arithmetic.sign
keyword.operator.arithmetic.sign.negative
keyword.operator.arithmetic.sign.positive
keyword.operator.arithmetic.subtraction
- Assignment
- Statements
- General
entity.name.statement
(statement label)keyword.control.flow.break
keyword.control.flow.throw
punctuation.definition.block
(braces of any block statement)punctuation.separator.label-statement
(statement label colon)punctuation.terminator.statement
(terminal semicolon or empty statement)
- Conditional Statements
keyword.control.conditional
keyword.control.conditional.else
keyword.control.conditional.if
keyword.control.switch
(the switch keyword)keyword.control.switch.case
(the case keyword)keyword.control.switch.case.default
(the default keyword, in a switch)punctuation.definition.block.conditional
(braces forif
orelse
)punctuation.definition.block.switch
(braces forswitch
)punctuation.definition.expression.conditional
(parens forif
)punctuation.definition.expression.switch
(parens forswitch
)punctuation.separator.case-statements
(colon aftercase
ordefault
)
- Loop Statements
keyword.control.flow.continue
keyword.control.loop.do
keyword.control.loop.each
(deprecated)keyword.control.loop.for
keyword.control.loop.in
keyword.control.loop.of
keyword.control.loop.while
punctuation.definition.block.loop
(braces inwhile (x) {...}
)punctuation.definition.expression.loop
(parens inwhile (x) {...}
)punctuation.separator.loop-expression
(semicolons in a C-stylefor
)
- Try Statements
keyword.control.trycatch
keyword.control.trycatch.catch
keyword.control.trycatch.finally
keyword.control.trycatch.try
punctuation.definition.block.trycatch
(braces in all three)punctuation.definition.parameters.catch
(parens incatch (err)
)variable.parameter.catch
(err incatch (err)
)
- Module Statements
entity.name.module.export
entity.name.module.import
punctuation.definition.module-binding
(braces inimport { x, y } from 'z'
)punctuation.separator.module-binding
(comma inimport { x, y } from 'z'
)storage.modifier.module.as
storage.modifier.module.default
storage.modifier.module.from
storage.modifier.module.namespace
(the asterisk inimport * from z
)storage.type.module.export
storage.type.module.import
variable.other.readwrite.export
variable.other.readwrite.import
- Nonsense
keyword.control.with
(deprecated)keyword.other.debugger
(deprecated)punctuation.definition.block.with
(deprecated)punctuation.definition.expression.with
(deprecated)
- General
- Variables & Constants
- Declarations
storage.type.constant
storage.type.variable.let
storage.type.variable.var
- Binding Patterns (Destructuring)
entity.other.property-binding
(y inlet { y: z } = x
)punctuation.definition.binding
punctuation.definition.binding.array
punctuation.definition.binding.object
punctuation.separator.array-element.binding
(comma in array binding)punctuation.separator.binding-binding
(the comma here:let x, y
)punctuation.separator.object-member.binding
(commas in object patterns)punctuation.separator.property-binding
(colon inlet { y: z } = x
)- see also Functions for binding patterns in parameter declarations
- Declarations
- Identifiers
- General
variable.other.readwrite
variable.other.readwrite.allCap
variable.other.readwrite.initCap
variable.other.readwrite.property
variable.other.readwrite.property.allCap
variable.other.readwrite.property.initCap
- Contextual References & Pseudo-References
variable.language.arguments
variable.language.new-target.fake-accessor
(the dot innew.target
)variable.language.new-target.fake-object
(the new innew.target
)variable.language.new-target.fake-property
(the target innew.target
)variable.language.super
variable.language.this
- Special Properties
variable.other.readwrite.property.proto
(i.e.,x.__proto__
; a certified Bad Part)variable.other.readwrite.property.prototype
(i.e.X.prototype
)
- Native (& Nearly Native) Objects
support.class.builtin
(e.g.Array
)support.function.builtin
(e.g.parseInt
)support.variable.builtin
(e.g.Math
)
- Domain-Specific Objects
support.class.node
(that is,Buffer
)support.variable.node.module
(module & exports environmental vars)support.function.node.require
support.variable.dom
(e.g.document
; kept this list minimal though)support.variable.dom-library
(e.g.$
)support.variable.functional-library
(e.g._
)support.variable.node
(e.g.process
)
- General
- JSX
entity.name.tag.jsx
(element name)keyword.operator.accessor.jsx
(access dot in ‘namespaced’ element name)keyword.operator.spread.jsx
(spread operator in attribute interpolation)meta.interpolation.jsx
(covers interpolated sequences)meta.namespace.jsx
(html/xml namespace prefixes -- not same as what jsx calls namespace)punctuation.definition.attribute.begin.jsx
(single or double quotes)punctuation.definition.attribute.end.jsx
punctuation.definition.interpolation.begin.jsx
(curly braces)punctuation.definition.interpolation.end.jsx
punctuation.definition.tag.begin.jsx
(element tag delimiter)punctuation.definition.tag.end.jsx
punctuation.separator.attribute-value.jsx
(equals sign)punctuation.separator.namespace.jsx
(colon)string.attribute.jsx
(literal attribute value)string.text.jsx
(literal chardata)variable.other.entity-reference.jsx
(html/xml entity refs)variable.other.attribute.jsx
(attribute name)
- Other
invalid
invalid.deprecated
(e.g.with
)invalid.illegal.newline
(e.g., inside a single-quote string)invalid.illegal.octal-escape
(e.g.'\\101'
; not valid since ES3)invalid.illegal.token
(syntax error)meta.whitespace
punctuation.definition.arguments.promise
(included so promises and async functions could share colors)punctuation.definition.expression
(parentheses of a parenthetic expression)support.class.builtin.promise
(singled out for reason stated above)variable.other.readwrite.decorator
(if a decorator expression begins with an identifier, as is typical, it will have this scope applied)
These scopes are used to facilitate proper population of the symbol list.
meta.symbol-helper.arrow.es
meta.symbol-helper.class.es
meta.symbol-helper.function.es
meta.symbol-helper.generator.es
These scopes exist alongside others above to maximize interoperability with existing themes, especially those targetting JSNext and Babel Sublime.
constant.other.object.key.js
entity.name.class.js
entity.name.function.js
entity.name.method.js
entity.name.tag.js
entity.name.type.new
entity.quasi.element.js
entity.quasi.tag.name.js
keyword.generator.asterisk.js
keyword.operator.module.js
keyword.other.js
meta.brace.curly.js
meta.brace.round.js
meta.brace.square.js
meta.delimiter.comma.js
meta.function-call
meta.function.arrow.js
meta.function.js
meta.instance.constructor
meta.separator.comma.js
punctuation.definition.tag.js
punctuation.quasi.element.begin.js
punctuation.quasi.element.end.js
storage.type.accessor.js
storage.type.extends.js
storage.type.function.js
storage.type.js
string.regexp.js
string.unquoted.label.js
variable.language.proto
variable.language.prototype
I’ve used existing tmLanguage conventions, plus JSNext and Babel Sublime, as guides for scope naming. Nonetheless there’s a fair amount of divergence. Some of this is just the consequence of disambiguating previously conflated elements.
In a few cases, the original Sublime JS tmLanguage had errors, and Babel and JSNext preserved them. I chose to correct these at the risk of decreasing compatibility with existing themes. For example, ‘with’ is not an operator. But there aren’t many of these and they are usually minor things that won’t affect most themes.
Sometimes I opted to use a pre-existing tmLanguage convention over a domain-
specific choice. For example, Babel uses quasi
but many tmLanguages and themes
already target string.interpolated
. In cases like this I typically ‘double
scope’ the tokens so that they can be targetted either way.
Other divergences stem from the objective of the definition, which may differ a
bit. I wanted the scopes to be very reflective of the language’s grammar. For
example, entity.name.function
will appear in a function declaration, but it
won’t appear in function invocations; to Ecmascript Sublime, in that context
what you’re looking at is an identifier (which has a scope) and an invocation
(which also has a scope), but not an entity name.
Many of the new scopes concern punctuation. The punctuation.definition
scope
namespace is the existing convention for these things. So for if statements, for
example, you have the following to work with:
keyword.control.conditional.else
keyword.control.conditional.if
punctuation.definition.block.conditional.begin
punctuation.definition.block.conditional.end
punctuation.definition.expression.conditional.begin
punctuation.definition.expression.conditional.end
One would probably never have a reason to include ‘begin’ or ‘end’ but this kind of trailing specificity helps sometimes with debugging and in any case is an existing convention. Scopes are hierarchical selectors, so if you wanted to have one color for the whole statement, the definition in your YAML-tmTheme could look like this:
- name: If / else statements
scope: >-
keyword.control.conditional,
punctuation.definition.block.conditional,
punctuation.definition.expression.conditional
settings:
foreground: '#FF0000'
If you’re new to theming, make sure you grab AAAPackageDev and ScopeAlways from Package Control. The former will let you translate human-readable YAML into tmTheme XML, and the latter will show, in the status bar, what scopes are being applied where your cursor is.
If you’re poking around in there you might think it’s insane to have so many redundant components. In many cases, similar results could be achieved using meta_scopes on a deeper context stack, since multiple scopes can appear in a selector.
To keep it brief...ish: this very un-DRY approach was the arrived at after a lot of experimenting. Sublime syntax is powerful, but I might be trying to do some stuff that wasn’t anticipated. Taking a very ‘grammatical’ approach (in fact I worked right off the ES6 final draft) is what lets us have all the new & disambiguated scopes to work with, but the price paid for syntactic precision is the need to pay special attention to failing as gracefully as possible when facing a bad token.
There are a few techniques I’ve found to prevent or mitigate ‘bad input cascade,’ and the most important is keeping the context stack as shallow as is reasonable at any given time -- that is, preferring linear over vertical context transitions. This is what necessitates the repetition.
That said there certainly are places where we could merge things, especially by making tertiary contexts that exist only as includes for building others which are actually visited.
If it ever becomes possible to pop multiple contexts, like you can set multiple contexts, we could probably cut the complexity and repetitiveness of the definition by a ton.
The other main technique for preventing cascade effects is to handle some of the illegal cases explicitly. There’s quite a bit of this ‘handmade’ correction actually. Since we know what kind of things are likely to be temporary artifacts of someone being midway through typing something that will soon be valid, we can mark a single bad token invalid but then still transition to what we are pretty sure should be next. With time I plan to add more of this sort of thing, but I only discover good places for it with use.
I was absolutely determined to disambiguate ‘in’ (operator) from ‘in’ (control word). As far as I can tell, this is the only way we can achieve that. This idea is actually present in the formal grammar definition, too; I didn’t make it up myself.
Surprisingly, no. Well, I haven’t noticed a problem anyway. I don’t know about the inner workings of Sublime Text, but if you think about it, if regex matches are the most expensive part, then sublime syntax with atomic contexts is probably efficient: each context will only try matching things that are actually expected to appear there.
Wacky linebreaks! Actually it can correctly scope weird line break situations a lot more often than is possible in tmLanguage, because in linear grammar sequences, like those that define most statements, the context stack is all we need to get 100% perfect matches. But there are cases in ES expressions -- more than I initially thought, too -- where a token is ambiguous until accounting for a later token that is allowed to be on the next line (where we can’t look).
Fortunately, save one, all of these linebreaks are, while legal, totally asinine. That is, if a person really wrote this:
label
: {
[ a,
b=4 ]
= [ (c)
=> { c * 2, d } ]
}
... they don’t deserve syntax highlighting. So consider it a feature.
However even in these cases, Ecmascript Sublime will make a noble attempt at
recovering. In the example above. The binding pattern would be an array, but
when it hit the =4
it would correct the remainder. The (c)
would be an
expression, but at the arrow it would figure out where it really is, etc. But
not everything can be salvaged: the label
would be an identifier, but the
colon would seem to be invalid because at that point we have to assume we’re in
a context expecting this series of tokens to resolve to an expression.
These are some of the cases where a technically-legal-but-obnoxious linebreaks can cause mismatching, where the pipe represents a problematic linebreak:
- Distinguishing between ‘in’ and ‘in’ in a for loop depends on lookaheads for sequences that could cross lines.
- Statement label |
:
- Parameter(s) |
=>
function
|*
- [ or ( of a comprehension | for
async
|function
(but only in expressions -- this is because async could also be an identifier, and accomodating that is important because of the popular Node library)- identifier |
(
(identifier will not be recognized as an invocation) - identifier | ``` (identifier will not be recognized as a tag)
- There are various other situations similar to the last two, mainly concerning ‘secondary’ scopes like invocation rather than core scopes.
One more is a sore spot for me:
- binding pattern | assignment operator
Unlike the others, this one could reasonably show up in code written by a sane person. But even so, it’s probably quite rare in practice.