kddnewton / parsing-ruby

Parsing Ruby

This project is an attempt to distill the history of efforts relating to parsing the Ruby programming language. That includes various implementations of Ruby (MRI, YARV, Rubinius, JRuby, TruffleRuby, etc.), gems that parse Ruby (parser, ruby_parser, ripper, etc.), as well as other projects that parse Ruby source for various reasons (sorbet, tree-sitter, RIL, etc.).

Over time the community has developed a massive amount of projects whose goal is to understand Ruby source at a deeper level. This has resulted in myriad approaches to parsing, with various tradeoffs. They have all had to evolve with the language as new features have been proposed, created, and merged. Below is a timeline of this archaeological dig through the history of parsing Ruby.

Ruby versions

1994-01-07 - Ruby 0.06

This is the earliest changelog entry that I could find that had an explicit version on it. At this point Ruby was still a very early idea and the syntax was changing rapidly.

ChangeLog

Fri Jan 7 15:23:20 1994 Yukihiro Matsumoto (matz at nws119)

* baseline - version 0.06.

1995-05-19 - Ruby 0.76

The changelog here is still entirely Yukihiro Matsumoto. It's been a year since the last entry on this timeline and a ton has changed. The choice of including 0.76 is somewhat arbitrary here. It's just that https://cache.ruby-lang.org happens to have a tar file containing that particular release.

There are some interesting things happening that are mentioned in the changelog below. include used to be a keyword, but there are a couple of keywords that at the moment are going through "methodization" which extracts them from the grammar and moves them into standard-library space. We can see here as well that hashes used to be called Dict. This was the version that included syntax for hash literals and array literals (array literals used to be initialized with braces, but that change here).

The ToDo file used to live in the root of the repository in these older versions. It's an interesting artifact, as it offers a window into Matz' thinking at the time. We can see in this version and many future versions he mentions heredocs (which do eventually get added before 1.0) as well as a byte code interpreter (what would eventually become YARV).

Grammar

• EBNF
• Diagram

ChangeLog

Wed Apr 26 09:50:56 1995 Yukihiro Matsumoto (matz@ix-02)

* parse.y: イテレータブロックの変数宣言を`|'で括るようにした．これ
  でイテレータ変数がない時は宣言そのものを省略できる．文法の変更は
  久しぶりだ．

Mon Mar 6 19:34:32 1995 Yukihiro Matsumoto (matz@ix-02)

* eval.c(inlcude): メソッド化．動的にモジュールをインクルードでき
  るように．さらに任意のオブジェクトにもモジュールをインクルードで
  きるメソッド `extend'も用意した．

* parse.y: 文法からincludeを削除．メソッド化．

Tue Feb 28 15:35:10 1995 Yukihiro Matsumoto (matz@ix-02)

* parse.y: 配列，連想配列の最後に`,'をおけるように．

Fri Oct 14 13:22:18 1994 Yukihiro Matsumoto (matz@ix-02)

* version 0.52 released: ……なんてこったい.

Thu Jul 14 11:18:07 1994 Yukihiro Matsumoto (matz@ix-02)

* parse.y: Dictを生成する構文を追加. こちらを{..}にした.

* parse.y: 配列を生成する構文を[..]に変更した. 過去のRubyスクリプ
  トとの互換性が保てないが, Dictを生成する構文を導入するに当たり,
  perl5に合わせて(意識して), 変更する時期は今しかないと考えた. 
  *BACKWARD INCOMPATIBILITY*

ToDo

• format機能
• here document
• re-write regex code for speed
• byte code interpretor

1995-12-21 - Ruby 0.95

This is also somewhat arbitrary, but the cache has an entry for it so I'm including it here. Technically the last pre-1.0 release was 0.99.4-961224. Between this release and 1.0-961225, it's mostly cleanup and preparation. At this point there are some other contributors, including Jun Kuroda and Hirofumi Watanabe.

The changelog mentions some interesting syntax changes, including changing the defined keyword to include a ? since this release now supports ? and ! at the end of method names. There's also an amusing anecdote about how the rescue keyword used to be misspelled but that this release fixed the typo. Also included are the not operator, dynamic superclasses, and optional parentheses on method definition arguments, to just name a few.

At this point in the ToDo file we get the addition of an item that says hand written parser (recursive descent). While this one didn't end up being executed at any point (Ruby has stuck with the Bison parser generator the entire way through to today) it's a very interesting thought even with today's codebase.

Grammar

• EBNF
• Diagram

ChangeLog

Thu Nov 9 23:26:01 1995 Yukihiro Matsumoto [e-mail]

* parse.y (f_arglist): メソッド定義の引数を括弧で括らなくても良い
  ようにした．

Wed Nov 8 00:17:51 1995 Yukihiro Matsumoto [e-mail]

* parse.y: class文のsuperclass部を定数から式に拡張した．

Fri Sep 8 14:18:51 1995 Yukihiro Matsumoto [e-mail]

* ruby.texi: `!', `?'に対応してアップデート．

* parse.y: defined -> defined?

* parse.y (yylex): 変数名の後ろに`?'も許す．述語メソッドの後ろに
  `?'を追加する．

Thu Sep 7 20:01:33 1995 Yukihiro Matsumoto [e-mail]

* parse.y: 変数名の後ろに`!'を許す．

Mon Aug 7 12:47:41 1995 Yukihiro Matsumoto [e-mail]

* parse.y: resque -> rescue．恥ずかしいがtypoを残しておくわけには
  いかないよなあ．なんで今まで気がつかなかったのか…．

Wed Jun 7 11:58:12 1995 Yukihiro Matsumoto matz@ix-02

* parse.y: not演算子の追加．優先順位の低い`!'演算子．

ToDo

• Hand written parser(recursive decent)
• format機能
• here document
• re-write regex code for speed
• byte code interpretor

1996-12-25 - Ruby 1.0.961225

This is the first public 1.0 release, released on Christmas of 1996. A couple of things change here since our last entry, most of which include changes that take it from looking like C++ to looking more like the Ruby we know today. For example, the operator that indicates a superclass in a class declaration changes from : to <. Also, the continue keyword gets renamed to next. rescue also gets a boost in utility, as it can now capture the exception as a variable and can rescue multiple exceptions at once. There's also a very short-lived triple-quoted string that exists for about 2 weeks in the codebase before Matz removed it.

The todo file contains some interesting entries at this point. Beyond the format function that has been on the list for a while, the regex item now gains the "and copyright" suffix. There's also an entry for access control/packages. This is stil being discussed today, and various efforts have been made to accomplish this in Ruby-space outside of the runtime like packwerk.

Grammar

• EBNF
• Diagram

ChangeLog

Thu Dec 12 00:41:17 1996 Yukihiro Matsumoto [e-mail]

* parse.y (parse_string): """..."""はやはり無くすことにした

Thu Nov 28 00:59:54 1996 Yukihiro Matsumoto [e-mail]

* version 0.99.3-961128

* parse.y (parse_string): 3-quote styleの文字列(例:"""abc"d"e""")

Thu Aug 29 10:49:40 1996 Yukihiro Matsumoto [e-mail]

* parse.y (expr): イテレータの新形式に「method do .. end」形式を採
  用した．もちろん昔の形式も有効．

Tue Aug 20 13:37:16 1996 Yukihiro Matsumoto [e-mail]

* parse.y (primary): rescueの構文を変更(同定引数の追加，複数rescue)

Thu Jul 25 12:15:04 1996 Yukihiro Matsumoto [e-mail]

* parse.y: break/next/redo/retryのメソッド化．

* parse.y (primary): unless/untilの復活

Wed May 22 19:48:42 1996 Yukihiro Matsumoto [e-mail]

* parse.y (superclass): スーパークラスの指定子を`:'から`<'に変更．

Wed Mar 27 10:02:44 1996 Yukihiro Matsumoto [e-mail]

* parse.y: 予約語の変更 continue -> next

ToDo

• パッケージまたは大域変数のアクセス制御
• format機能
• re-write regex code for speed and copyright

1997-12-25 - Ruby 1.0.971225

Following in what would eventually become quite a transition, Matz released a new version of Ruby on Christmas 1997. It's still 1.0 (this is not yet following semantic versioning) but the suffix has been updated to include the date.

There are a couple of interesting syntactical additions to the language during this time. Regular expressions get a couple of special flags that indicate the character encoding. Perhaps this is one of the best advantages of having this language be created outside of an English-speaking country: encoding was always at the forefront. While we wouldn't get great encoding support until 1.9, even having the option at this point is a win.

ChangeLog

Mon Apr 7 11:36:16 1997 Yukihiro Matsumoto [e-mail]

* parse.y (primary): syntax to access singleton class.

Thu Apr 3 02:12:31 1997 Yukihiro Matsumoto [e-mail]

* parse.y (parse_regx): new option //[nes] to specify character
  code for regexp literals.  Last specified code option is valid.

Tue Mar 25 14:08:43 1997 Yukihiro Matsumoto [e-mail]

* parse.y (expr): alias $var1 $var2 makes alias of the global
  variable. 

Fri Mar 14 14:36:28 1997 Yukihiro Matsumoto [e-mail]

* parse.y (yylex): enables negative hex/octal numbers and `_' in
  non-decimal numbers.

1998-12-24 - Ruby 1.3.0

This is considered a "development" release, and is meant to be used as a branch for the core developers and not used in production. It is released one day before the "stable" 1.2.0 version. It includes a couple of syntactic additions that are then used for development over the course of the next year before they are included in the 1.4.0 stable version. They are:

• begin..rescue..else..end clauses
  rescue already existed, but this provided support for the else keyword in this chain in the case that no exception was raised
• <<- indentable heredocs
  heredocs already existed, but you always had to put the ending at the beginning of the line; this change allowed the endings to be indented to the same indentation as the code
• :: method calls
  effectively an alias for the . operator, there was a convention for a while to call class-level methods with ::

Also interesting to note in this version are 2 new entries in the todo file. The first is "named" arguments (what would eventually become keyword arguments). Though the syntax isn't what ended up landing := versus just :, it's still interesting to see the entry all of the way back here.

Worth noting is the objectify interpreters entry. While I can't speak to what this would actually look like, what it could be in reference to would be an object that could be accessed from Ruby-space that would accept an AST as an argument and be able to execute the code on demand. Extending this idea to include a reentrant parser, and Matz is effectively referring to the Rubinius project which would be started a little less than a decade later.

ChangeLog

Thu Dec 24 00:17:00 1998 Yukihiro Matsumoto [e-mail]

* parse.y (primary): enable expr::identifier as method
  invocation.

Mon Dec 7 22:08:22 1998 Yukihiro Matsumoto [e-mail]

* parse.y (primary): allows `def obj::foo; .. end'.

Sat Dec 5 23:27:23 1998 Yukihiro Matsumoto [e-mail]

* parse.y (here_document): indentable here-doc delimiter by
  `<<-'.  Proposed by Clemens <[e-mail]>.  Thanks.

Mon Nov 16 23:26:29 1998 Yukihiro Matsumoto [e-mail]

* parse.y (primary): exec else clause if no exception raised.

Wed Oct 14 00:18:33 1998 Yukihiro Matsumoto [e-mail]

* parse.y (when_args): `when a, *b' style new syntax for array
  expansion in `case'.

ToDo

• package or access control for global variables
• named arguments like foo(nation:="german").
• multiple return values, yield values. maybe imcompatible
• objectify interpreters
• syntax tree -> bytecode ???

1998-12-25 - Ruby 1.2.0

This is a "stable" release, and is meant for production. It's the first stable release since 1.0 was released exactly 2 years prior. A lot of new syntax was introduced in those two years, many of which form the foundation of the kind of Ruby that was see today. Here are a couple of the standout entries:

• heredocs
  easily one of the most difficult-to-parse syntax constructs gets introduced
• =begin to =end
  multi-line comments now were easier to manage
• true and false
  these keywords didn't actually exist before this point
• BEGIN and END
  borrowing from awk, this syntax was very useful for scripts
• %w word lists
  here we start to see the beginning of the %-lists syntax
• top-level constant access
  you could now prefix a constant with :: from within a nested scope to access a top-level constant
• block arguments
  arguments that were themselves blocks would now resolve
• ||= and &&= operators
  this further extended the assignment operators with || and && support

You can see in the changelog that __END__ is no longer a keyword with this release. Over time a lot of the keywords that we have today have flipped back and forth between being methods and being keywords, including next and break (for example).

Grammar

• EBNF
• Diagram

ChangeLog

Wed Jun 24 02:18:57 1998 Yukihiro Matsumoto [e-mail]

* parse.y (mlhs): `((a,b)),c = [[1,2]],3' assigns a=1,b=2,c=3.

Tue Jun 23 11:46:16 1998 Yukihiro Matsumoto [e-mail]

* parse.y (yylex): `&&=' and `||=' added.

Fri Jun 19 14:34:49 1998 Yukihiro Matsumoto [e-mail]

* parse.y (mlhs): nested multiple assignment.

Wed Apr 15 01:22:56 1998 Yukihiro Matsumoto [e-mail]

* parse.y (yylex): allow nested parenthesises.

Wed Mar 4 01:39:52 1998 Yukihiro Matsumoto [e-mail]

* parse.y (block_arg): new syntax - block argument in the
  calling arglist.

Thu Feb 26 17:22:13 1998 Yukihiro Matsumoto [e-mail]

* parse.y (fname): convert reswords into symbols.

* parse.y (reswords): reserved words are now embedded in the
  syntax (sigh).

* parse.y: now reserved words can be method names safely.

Fri Feb 20 10:17:51 1998 Yukihiro Matsumoto [e-mail]

* version 1.1b8 released.

* parse.y (stmt): if/unless modifiers returns nil, if condition is 
  not established.

Tue Feb 17 00:04:32 1998 Yukihiro Matsumoto [e-mail]

* parse.y (yylex): new form `::Const' to see toplevel constants.

Tue Jan 20 15:19:59 1998 Yukihiro Matsumoto [e-mail]

* parse.y (terms): quoted word list by %w(a b c).

Fri Dec 12 00:50:25 1997 Yukihiro Matsumoto [e-mail]

* parse.y (expr): BEGIN/END built in the syntax.

Thu Oct 9 11:17:50 1997 Yukihiro Matsumoto [e-mail]

* parse.y (nextc): script parsing will be terminated by __END__ at
  beginning of line.

* eval.c (compile_error): `__END__' is no longer a keyword.

Tue Sep 30 10:27:39 1997 Yukihiro Matsumoto [e-mail]

* parse.y: new keywords `true' and `false' added.

Wed Aug 27 11:32:42 1997 Yukihiro Matsumoto [e-mail]

* version 1.1 alpha3 released.

* parse.y (here_document): finally here document available now.

ToDo

• package or access control for global variables
• format

1999-08-13 - Ruby 1.4.0

This is another stable release that follows a little less than a year after 1.2.0. Not a huge amount changes with regard to syntax between the two versions, though there are lots of changes elsewhere in the codebase. The changes that were included were:

• binary number literals
  you can now write number literals with the 0b prefix
• anonymous * in method definitions
  you now don't need to put an explicit name on splat arguments in method definitions
• nested string interpolation
  you used to not be able to do string interpolation within string interplation, but that was fixed here
• multibyte character identifiers
  there are now explicit multibyte character identifiers - this is more work toward supporting different encodings, but we won't truly get there until 1.9

The todo file got a lot longer in this version. Below is just a snippet that relates to parsing, but in reality there were multiple sections added. You can see entries that we already had like the packaging system, named arguments, and objectify interpreters. There are also some new entries like class variable (what ended up becoming @@variable), method to retrieve argument information (this could be referring to method calls, but I think it's referring to method parameters on the declaration, which would eventually become Method#parameters), and compile time string concatenation (which we eventually get where "foo" "bar" becomes "foobar").

Grammar

• EBNF
• Diagram

ChangeLog

Thu Jun 24 19:11:29 1999 Yoshida Masato [e-mail]

* parse.y (yylex): support multi-byte char identifiers.

Wed Jun 23 15:10:11 1999 Inaba Hiroto [e-mail]

* parse.y (parse_regx): nested braces within #{} available.

Wed May 19 12:27:07 1999 Yukihiro Matsumoto [e-mail]

* parse.y (f_rest_arg): allow just * for rest arg.

* parse.y (mlhs_basic): allow * without formal argument.

Fri Feb 5 22:11:08 1999 EGUCHI Osamu [e-mail]

* parse.y (yylex): binary literal support, like 0b01001.

* parse.y (yylex): octal numbers can contain `_'s.

* parse.y (yylex): now need at least one digit after prefix such
  as 0x, or 0b.

* bignum.c (rb_str2inum): recognize binary numbers like 0b0101.

ToDo

• compile time string concatenation, "hello" "world" => "helloworld"
• ../... outside condition invokes operator method too.
• %w(a\ b\ c abc) => ["a b c", "abc"]
• package or access control for global variables
• class variable (prefix?)
• named arguments like foo(nation:="german") or foo(nation: "german").
• method to retrieve argument information (need new C API)
• multiple return values, yield values. maybe incompatible ???
• cascading method invocation ???
• def Class#method .. end ??
• class Foo::Bar<Baz .. end, module Boo::Bar .. end
• def Foo::Bar::baz() .. end ??
• objectify interpreters
• syntax tree -> bytecode ???
• format like perl's

1999-12-07 - Ruby 1.5.0

Quickly following the release of 1.4.2 is the release of 1.5.0. Not a ton of time has passed so there aren't too many syntactical changes to mention. A couple of quick highlights though include rescue getting the additional modifier syntax like the conditionals and loops. We also get compile-time string concatenation (something that was listed in the todo from the previous version).

For the first time, the idea of rescue having a special !! operator is mentioned. This lives on in the todo file for quite a while. I'm not aware of if it was ever seriously considered.

Grammar

• EBNF
• Diagram

ChangeLog

Mon Nov 8 14:28:18 1999 Yukihiro Matsumoto [e-mail]

* parse.y (stmt): rescue modifier added to the syntax.

Thu Oct 14 02:00:10 1999 Yukihiro Matsumoto [e-mail]

* parse.y (string): compile time string concatenation.

ToDo

• def foo; .. rescue .. end
• compile time string concatenation, "hello" "world" => "helloworld"
• assignable constant, which now should be called shared variable.
• class variable (prefix?) -- done by shared variable
• rescue modifier; a rescue b => begin a rescue; b end

• operator !! for rescue.
• objectify symbols
• objectify characters
• ../... outside condition invokes operator method too.
• ... inside condition turns off just before right condition.???
• %w(a\ b\ c abc) => ["a b c", "abc"]
• package or access control for global variables??
• named arguments like foo(nation:="german") or foo(nation: "german").
• method to retrieve argument information (need new C API)
• multiple return values, yield values. maybe incompatible ???
• cascading method invocation ???
• def Class#method .. end ??
• class Foo::Bar<Baz .. end, module Boo::Bar .. end
• def Foo::Bar::baz() .. end ??

• RUBYOPT environment variable
• alias $defout $&gt;

• objectify interpreters
• syntax tree -> bytecode ???

2000-09-19 - Ruby 1.6.0

A little over a year has passed since 1.4.0, which means it's time for another stable release. Only one big thing really changed with the syntax between the two versions, which is that rescue can now be used in the modifier form like the conditionals and loops.

Interestingly there are a couple of references to the flip-flop operator in the todo file. This has got to be one of the most controversial Ruby features. Later in 2.6 it will be deprecated, and then un-deprecated in 2.7. Either way it's definitely one of the more interesting syntactical constructs.

There's also a mention of 0 being evaluated as falsy. Fortunately this one did not make it in, as that would have somewhat drastically changed the semantics of Ruby as we know it.

Grammar

• EBNF
• Diagram

ChangeLog

Mon Sep 11 14:24:47 2000 Yukihiro Matsumoto [e-mail]

* parse.y (command_call): kYIELD moved to this rule to allow
  'a = yield b'. (ruby-bugs-ja:#PR15) 

Fri Sep 1 10:36:29 2000 Yukihiro Matsumoto [e-mail]

* parse.y (aref_args,opt_call_args): add block_call to allow a
  method without parentheses and with block as a last argument.

Tue Jul 11 16:54:17 2000 Yukihiro Matsumoto [e-mail]

* parse.y (yylex): `@<digit>' is no longer a valid instance
  variable name.

Sat May 6 23:35:47 2000 Yukihiro Matsumoto [e-mail]

* parse.y (lhs): should allow `obj.Attr = 5' type expression.

Mon Jan 24 02:56:44 2000 Yukihiro Matsumoto [e-mail]

* parse.y (yylex): -2.abs should be `(-2).abs' to accomplish the
  principle of less surprise.  `+2' too.

ToDo

• def foo; .. rescue .. end
• rescue modifier; a rescue b => begin a rescue; b end
• %w(a\ b\ c abc) => ["a b c", "abc"]
• class variable (prefix @@)
• rescue RuntimeError => err

• operator !! for rescue. ???
• objectify characters
• ../... outside condition invokes operator method too.
• ... inside condition turns off just before right condition.???
• package or access control for global variables??
• named arguments like foo(nation:="german") or foo(nation: "german").
• multiple return values, yield values. maybe incompatible ???
• cascading method invocation ???
• def Class#method .. end ??
• class Foo::Bar<Baz .. end, module Boo::Bar .. end
• def Foo::Bar::baz() .. end ??
• Fixnum 0 as false ????
• non confusing in-block local variable (is it possible?)
  • remove scope by block
  • variables appears within block may have independent values.

• alias $defout $&gt;

• objectify interpreters ???
• syntax tree -> bytecode ???

2001-06-01 - Ruby 1.7.1

About a year has passed since 1.6 was released, and Ruby is starting to pick up a little steam. It's also starting to look more and more like the Ruby that folks use today. You probably won't see any 1.7 code the wild any more, but it's possible you might get a glimse of 1.8. Syntactically, a couple of things have been merged:

The break and next keywords now accept values. This becomes useful for methods like detect/find to controls the output of the overall loop. %w also gains the ability to escape spaces within the bounds, which was previously on the todo list. Finally, rescue can be added to singleton method bodies.

Grammar

• EBNF
• Diagram

ChangeLog

Tue May 22 02:37:45 2001 Yukihiro Matsumoto [e-mail]

* parse.y (expr): "break" and "next" to take optional expression,
  which is used as a value for termination. [new, experimental]

Tue Mar 6 10:50:29 2001 Yukihiro Matsumoto [e-mail]

* parse.y (primary): rescue and ensure clauses should be allowed
  to appear in singleton method body.

Wed Feb 7 16:05:22 2001 Nobuyoshi Nakada [e-mail]

* parse.y (parse_quotedwords): %w should allow parenthesis escape.

Sat Dec 2 22:32:43 2000 Yukihiro Matsumoto [e-mail]

* parse.y (stmt): multiple right hand side for single assignment
  (e.g. a = 1,2) is allowed.

ToDo

• operator !! for rescue. ???
• objectify characters
• ../... outside condition invokes operator method too.
• ... inside condition turns off just before right condition.???
• package or access control for global variables??
• named arguments like foo(nation:="german") or foo(nation: "german").
• method to retrieve argument information (needs new C API)
• multiple return values, yield values. maybe incompatible ???
• cascading method invocation ???
• def Class#method .. end ??
• class Foo::Bar<Baz .. end, module Boo::Bar .. end
• def Foo::Bar::baz() .. end ??
• I18N (or M17N) script/string/regexp
• Fixnum 0 as false ????
• discourage use of symbol variables (e.g. $/, etc.) in manual
• discourage use of Perlish features by giving warnings.
• non confusing in-block local variable (is it possible?)
  • remove scope by block
  • variables appears within block may have independent values.
• decide whether begin with rescue or ensure make do..while loop.
• a +1 to be a+1, not a(+1).
• raise exception by `` error
• jar like combined library package.
• objectify interpreters ???
• syntax tree -> bytecode ???
• Built-in Interactive Ruby.

2003-08-04 - Ruby 1.8.0

A lot happens between 1.7 and 1.8. It's been 2 years since the 1.7 release, and Ruby has started to pick up in popularity. The popular "pickaxe" book Programming Ruby (Andy Hunt, Chad Fowler, and Dave Thomas) was released in 2001, which helped spread Ruby even further outside of Japan. Later that year in October, the first international Ruby conference was held in Tampa, Florida. From there, Ruby Central was founded by Chad Fowler and David Black. All of this momentum helped push a lot of companies to start to try out Ruby for the first time, including 37Signals.

Syntactically, there are a couple of notable changes, including:

• %W word lists with interpolation
  much like %q and %Q, %W is created as the word-list version that supports interpolation
• dynamic symbols
  symbols can now be created with interpolation
• break and next take values
  this becomes useful for methods like detect/find to controls the output of the overall loop
• nested constant assignment
  you can now assign the constants multiple levels deep using the :: operator

%w also gains the ability to escape spaces within the bounds, which was previously on the todo list. rescue can additionally be added to class and module bodies, where previously it was only on method definitions.

The todo list includes a couple of interesting new additions. There's mention of in-block local variables and the scoping they acquire. This would be addressed in 1.9. There's also mention of attempting to discourage folks from calling methods without parentheses. To my knowledge no linter currently exists for Ruby, but this would probably be the first rule if one did.

Finally, there's the first explicit mention in the todo file of a parser API. Other tools have already started to crop up outside of core Ruby (like ripper) that give direct access to the NODE syntax tree structs, and with the increase in popularity the demand for these kinds of tools has only grown. This is the last minor version that did not ship with a first-class parser API, as ripper would be merged in 1.9.

Grammar

• EBNF
• Diagram

ChangeLog

Thu Feb 20 10:11:30 2003 Yukihiro Matsumoto [e-mail]

* parse.y (clhs): allow "Foo::Bar = x".

Wed Feb 5 17:11:02 2003 Yukihiro Matsumoto [e-mail]

* parse.y (yylex): no .<digit> float literal anymore.

Mon Nov 4 16:49:14 2002 Yukihiro Matsumoto [e-mail]

* parse.y (primary): allow 'when'-less case statement; persuaded
  by Sean Chittenden.

Fri Oct 18 23:11:21 2002 Nobuyoshi Nakada [e-mail]

* parse.y (value_expr0): allow return/break/next/redo/retry in rhs
  of logical operator.  [ruby-dev:18534]

Fri Oct 11 15:58:06 2002 Yukihiro Matsumoto [e-mail]

* parse.y (arg): rescue modifier is now an operator with
  precedence right below assignments. i.e. "a = b rescue c" now
  parsed as "a = (b rescue c)", not as "(a = b) rescue c". [new]
  [experimental]

Fri Jul 19 10:52:32 2002 Yukihiro Matsumoto [e-mail]

* parse.y (yylex): new decimal notation '0d4567'.

Sat Jun 15 22:56:37 2002 Yukihiro Matsumoto [e-mail]

* parse.y (yylex): obsolete '?<whitespace>'; use '?\s', '?\n',
  etc, instead.

Tue May 28 14:07:00 2002 Yukihiro Matsumoto [e-mail]

* parse.y (arg): no more ugly hack for "**", so that "-2**2" to be
  parsed as "(-2)**2", whereas "- 2**2" or "-(2)**2" to be parsed
  as "-(2**2)".

* parse.y (yylex): '-2' to be literal fixnum. [new]

Tue Mar 26 01:56:33 2002 Yukihiro Matsumoto [e-mail]

* parse.y (primary): while/until statement modifiers to "begin"
  statement now work as "do .. while" even when begin statement
  has "rescue" or "ensure" [new].

* parse.y (bodystmt): rescue/ensure is allowed at every bodies,
  i.e. method bodies, begin bodies, class bodies[new], and module
  bodies[new].

Wed Aug 29 02:18:53 2001 Yukihiro Matsumoto [e-mail]

* parse.y (yylex): ternary ? can be followed by newline.

ToDo

• class Foo::Bar<Baz .. end, module Boo::Bar .. end

• operator !! for rescue. ???
• objectify characters
• ../... outside condition invokes operator method too.
• ... inside condition turns off just before right condition.???
• package or access control for global variables??
• named arguments like foo(nation:="german") or foo(nation: "german").
• method to retrieve argument information (needs new C API)
• multiple return values, yield values. maybe incompatible ???
• cascading method invocation ???
• def Class#method .. end ??
• def Foo::Bar::baz() .. end ??
• Fixnum 0 as false ????
• non confusing in-block local variable (is it possible?)
  • remove scope by block
  • variables appears within block may have independent values.
• method combination, e.g. before, after, around, etc.
• "in" modifier, to annotate, or to encourage assertion.
• private instance variable (as in Python?) @_foo in class Foo => @_Foo_foo
• warn/error "bare word" method, like "foo", you should type "foo()"
• :symbol => value hash in the form of {symbol: value, ...} ??
• objectify interpreters ???
• syntax tree -> bytecode ???
• Parser API
• trap every method invocation, which can be enabled by e.g. trap_call :method.

2007-12-25 - Ruby 1.9.0

Ruby 1.9.0 was a very large change because it integrated Koichi Sasada's graduate thesis YARV which stands for yet another RubyVM. It switched from being a tree-walk algorithm that kept an abstract syntax tree around to manipulate to being a bytecode interpreter that kept around instruction sequences.

In addition, while this transition was being made, ripper was merged into trunk. Now instead of forking parse.y and maintaining its own grammar, it instead was integrated into the main Ruby parse.y. In introduced a special comment format that lived inside the grammar actions that functioned as an alternate action that should be taken in the case that ripper was being built. Mostly it "dispatched" events when those tree nodes were being reduced and then continued building the tree.

There were a couple of additional (somewhat controversial) syntactical additions to the language with this version as well. Those include lambda literals (as in -> (foo) { foo * 2 }) and symbol hash keys (as in { foo: "bar" }). To this day there are folks that will avoid both of those syntax constructs.

Grammar

• EBNF
• Diagram

ToDo

• class Foo::Bar<Baz .. end, module Boo::Bar .. end
• raise exception by `` error
• clarify evaluation order of operator argument (=~, .., ...)
• :symbol => value hash in the form of {symbol: value, ...} ??

• operator !! for rescue. ???
• objectify characters
• ../... outside condition invokes operator method too.
• ... inside condition turns off just before right condition.???
• package or access control for global variables??
• named arguments like foo(nation:="german") or foo(nation: "german").
• method to retrieve argument information (needs new C API)
• multiple return values, yield values. maybe incompatible ???
• cascading method invocation ???
• def Class#method .. end ??
• def Foo::Bar::baz() .. end ??
• discourage use of symbol variables (e.g. $/, etc.) in manual
• discourage use of Perlish features by giving warnings.
• decide whether begin with rescue or ensure make do..while loop.
• method combination, e.g. before, after, around, etc.
• .. or something like defadvice in Emacs.
• property - for methods, or for objects in general.
• "in" modifier, to annotate, or to encourage assertion.
• selector namespace - something like generic-flet in CLOS, to help RubyBehavior
• private instance variable (as in Python?) @_foo in class Foo => @_Foo_foo
• warn/error "bare word" method, like "foo", you should type "foo()"
• objectify interpreters ???
• MicroRuby
• Built-in Interactive Ruby.
• Parser API
• Ruby module -- Ruby::Version, Ruby::Interpreter

Projects

Although it was meant for testing, various projects have cropped up that have used RubyVM::AbstractSyntaxTree to access the Ruby parse tree. The biggest one worth mentioning is solargraph, a Ruby language server.

2009-01-30 - Ruby 1.9.1

It took a little over a year for Ruby to reach 1.9.1 from 1.9.0. This was considered the first "stable" release of the 1.9 series, as most of the kinks with YARV had been worked out at this point.

There was a lot of work at this time in Ruby around encoding. Before, everything was assumed to be ASCII-ish. The 1.9 series made encoding into a first class citizen, which including creating the encoding pragma. This ended up becoming a pattern as other pragma were intoduced later (notably frozen_string_literal).

There were a couple of other interesting additions at this time as well. This includes the foo.() operator alias which resolves to the call method. Also introduced were positional arguments that followed a splat argument. Finally we also got block parameters as block variables.

Grammar

• EBNF
• Diagram

2011-10-31 - Ruby 1.9.3

Two full years later, and Ruby is still in the 1.9 series. The language itself has completely taken off, due in some part to the popularity of Ruby on Rails, which is now definitively in vogue. For the language itself, not a ton of syntax has changed in this time (a lot of work is being done on the standard library and the shiny new underlying bytecode interpreter). The only thing truly of note at this point is that you can now put trailing commas on method invocations.

This version of Ruby is particularly special for a couple of reasons. The grammar for this version of Ruby ended up being included in a couple of international standards, as Ruby became officially recognized first by the Japanese Standards Association in JIS X 3017, and second by the International Organization for Standardization in ISO/IEC 30170:2012.

This time period in Ruby's history is significant in that the influx of new users and the popularity of the language had gotten the attention of some larger companies that were now interested in investing in its future. This resulted in myriad implementations of Ruby being attempted on every platform imaginable, including the Java Virtual Machine, the Parrot Virtual Machine, the Common Language Runtime, and others.

Grammar

• EBNF
• Diagram

2013-02-24 - Ruby 2.0.0

After another year and a half of development since 1.9.3, 2.0.0 was released. Around this time bugs.ruby-lang.org was introduced as the official bug tracker and so the remaining features on this list can include links to the original discussion. These include:

• Module#prepend
  The ability to inject a module into the ancestor chain before the current class so that you could call super to access the original method. This drastically simplified a common construct at the time called alias_method_chain.
• Refinements
  Another controversial feature that created the using and refine keywords that allowed lexically-scoped monkey-patches.
• %i symbol lists
  A small but useful addition that expanded the %w pattern to %i to make lists of symbols.
• Keyword arguments
  Otherwise known as "named" arguments, these were arguments on method calls and definitions that had explicit names. The syntax looked/looks very similar to bare hashes without the braces. So much so that it ended up being implemented using a hash as the final argument to the method call, which caused a fair amount of compatibility problems. This ended up being remedied in 3.0 a full 8 years later.

Grammar

• EBNF
• Diagram

2013-12-25 - Ruby 2.1.0

Starting with Ruby 2.1.0, Ruby starts releasing on a very consistent schedule of every Christmas. The changes in this version include:

• Required keyword arguments
  Previously you always had to specify a default value, but this change allowed those values to be omitted.
• Rational and complex literals
  Rational and Complex already existed as classes, but this allowed syntax like 1/2r for creating them without explicitly referencing the class.
• Frozen string literal suffix
  This is an interesting addition that was temporarily merged and then replaced before this version was released. The proposal was to add an f suffix to strings to make them frozen by default without having to call a method. This is a theme that we'll see a lot over time as folks continually propose more ways to remove string allocations. This eventually gets obviated by a proposal to optimize .freeze on strings within the compiler.

Grammar

• EBNF
• Diagram

2014-12-25 - Ruby 2.2.0

In Ruby 2.2.0, the only large syntax change is that you can now use dynamic symbol hash keys. This means you can do something like { "foo #{bar}": baz }.

Grammar

• EBNF
• Diagram

2015-12-25 - Ruby 2.3.0

As with 2.1.0, there are still ongoing discussions of frozen strings, and we end up settling on the frozen_string_literal pragma (following in the tradition of the encoding pragma). This allows users to specify that all strings in the file should be frozen by default. This was considered a temporary measure to prepare for an eventual future where all strings in Ruby would be frozen by default (this was also added as a command-line switch). This was promised for Ruby 3, but didn't end up making it in.

In addition to the pragma, we also got <<~ heredocs that stripped common leading whitespace, and the &. operator (otherwise known as the "lonely" operator), which only called the method if the receiver was not nil.

Grammar

• EBNF
• Diagram

2016-12-25 - Ruby 2.4.0

Since its addition three years ago, refinements have not quite been adopted in many places. Nevertheless, work continues on them in this release where you can now use refined methods with the Symbol#to_proc syntax. For example, if you were to refine Integer#to_s, you could now write [1, 2, 3].map(&:to_s) and it would function as you would expect.

In addition, we got top-level return, which was useful in scripts where you wanted to bail from the entire program if some precondition was not met (like a dependency not beig available or not being on the right platform). Finally, you could now use multiple assignment in a conditional.

Grammar

• EBNF
• Diagram

2017-12-25 - Ruby 2.5.0

Work continues on refinements, as in this release we get refinements in string interpolations. Also released in this version were rescue and ensure at the block level, which had been referenced in the ToDo file for quite a few years at this point.

Grammar

• EBNF
• Diagram

2018-12-25 - Ruby 2.6.0

Keeping with their release schedule of every Christmas, Ruby 2.6.0 was released one year to the day following 2.5.0. One of the most interesting additions to history of parsing Ruby came in this version, which was the addition of RubyVM::AbstractSyntaxTree. This addition was accidental, it was added as a means of testing another feature that was being added. Unfortunately for the maintainer, people noticed.

RubyVM::AbstractSyntaxTree uses the internal NODE structs to build out an abstract syntax tree, much like many of the projects from the 1.6 to 1.8 days. Because it's written into core Ruby, it necessarily has access to things that other projects don't, which makes it particular faithful to the syntax. It is both praised and criticized on release, as it means another module that other implementations have to support if it becomes the blessed path for accessing the Ruby parse tree. The discussion continues to this day.

Other syntax additions include:

• Flip-flop (deprecated)
  At this point the flip-flop operator is deprecated, much to the chagrin of the commenters on the linked issue. This deprecation ends up getting reverted in Ruby 2.7.
• Endless range
  Ranges without endings get added to the syntax, which is a nice pairing with the lazy enumerable methods that have recently been merged. This allows things like (0..).lazy.map { |number| number * 2 }.take(10), for example, to get the first 10 even numbers.
• Non-ASCII constant names
  Constants in this version can now use non-ASCII characters in their names, which makes for all kinds of fun obfuscated code involving emojis.
• Escape keywords from class/module scope removed
  An extremely rarely (hopefully) used feature of Ruby was that you could call break within a class and module body. This was effectively a bug, but it's interesting and included here because syntax is extremely rarely removed from Ruby. This is one of the few exceptions.

Grammar

• EBNF
• Diagram

2019-12-25 - Ruby 2.7.0

Ruby 2.7.0 brought the biggest amount of new syntax to any version of Ruby. It boasts all kinds of interesting changes, including:

• Flip-flop (undeprecated)
  To the cheers of fans of the flip-flop operator, the deprecation was removed.
• Method reference operator (added)
  The method reference operator .: was added, as a shortcut for accessing a method object like foo.method(:bar).
• Keyword arguments (warning about hash-based)
  Since the addition of keyword arguments back in Ruby 2.0, they've always been backed by an internal hash. That resulted in all kinds of compatibility issues between implementations of Ruby. To simplify things, in 3.0 they decided to make them "real" arguments instead of using an internal hash structure. To ease the transition, Ruby 2.7.0 detected when code would break in 3.0 and warned about that usage.
• No other keywords syntax
  To define that no other keywords would be allowed on a method call, you could now use the special **nil syntax.
• Beginless range
  To complement the endless ranges from the previous minor version, ranges could now omit the beginning as well.
• Pattern matching
  A massive influx of syntax comes in 2.7 with the introduction of pattern matching. It's similar to a case..when statement, but instead uses case..in and comes with a whole host of new syntactical constructs. It merits its own documentation page being added to trunk.
• Numbered parameters
  An interesting addition to this version is numbered parameters. Take a look at the linked issue to see all of the proposed variants. This addition takes inspiration from languages like Scala that support a default block variable.
• Rightward assignment
  Even before 2.7 lands, pattern matching gets another addition which is rightward assignment. You can now assign variables using the special => syntax, as in foo => bar which assigns to the bar local variable. This adds to the pattern matching because you can now pattern match in rightward assignment.
• Argument forwarding
  Because keyword arguments are changing so much, its necessary to make a blessed path for forwarding named and unnamed arguments to another method. You can now use the ellipsis operator (...) to send every kind of argument (positional, keyword, and block) over to another method call. This helped eliminate some of the more problematic single-splat usage that was previously being used to accomplish this.
• Method reference operator (removed)
  Unfortunately, the method reference operator that had been added earlier in the year ended up being reverted. Check the linked issue for the discussion of why.

Grammar

• EBNF
• Diagram

2020-12-25 - Ruby 3.0.0

Seven years since the release of Ruby 2.0, Ruby goes 3.0. A lot of changes that don't have to do with syntax are included in this release. Syntactically, there are a couple of interesting additions:

• Keyword arguments (non-hash-based)
  The 3.x version of keyword arguments (as opposed to the hash-based 2.x version). This is the feature that Ruby 2.7 warned about constantly because folks were using the final hash argument to support keyword arguments before.
• Single-line methods
  Methods can now be written as def foo = bar for more concise syntax.
• Find pattern matching
  Pattern matching gets another type of pattern called the "find" pattern, which allows you to specify two * operators on each side of the value that you're searching for.
• shareable_constant_value pragma
  To better support shared values across ractors, this pragma was added to make constant values shareable without having to freeze them. It's a somewhat complicated option that involves a lot of explanation in the linked doc.
• in pattern matching
  Pattern matching gets one more boost with the ability to use the in operator for a single line. This is similar to rightward assignment, but returns the value of the match.

Grammar

• EBNF
• Diagram

Projects

2000-10-01 - nodeDump 0.1.0

Following the release of Ruby 1.6, Dave Thomas created the nodeDump project. It walked the AST that Ruby generated after parsing and dumped out human-readable documentation about what Ruby would be evaluating. This ended up inspiring a bunch of other projects, and to my knowledge is the first real attempt at accessing and manipulating the AST outside of the core Ruby developers.

When this is released there is also mention of the ii project by Guy Decoux, but for whatever reason that appears to be lost to internet history (it was hosted on an ftp server that is no longer live).

This project and the many others that it inspired live on until 1.9 when CRuby switched over to the YARV bytecode interpreter. Because a lot of internals had to change to support that switch, most of the projects from around this time no longer functioned on the newer Ruby versions.

Links

• README

2001-01-10 - ruth 0.0.1

A little while after nodeDump is released, Robert Feldt creates ruth (short for Ruby under the hood). It is effectively a generalized form of the nodeDump utility, and provides two Ruby-space methods, Ruby::Interpreter.parse and Ruby::Interpreter.method_body. These return something that is akin to s-expressions that can then be manipulated. As with nodeDump, it's a C extension that parses node.h to generate all of the necessary metadata.

Links

• readme

2001-10-20 - ripper 0.0.1

Following on the heels of the release of Ruby 1.7, Minero Aoki releases ripper, an event-driver Ruby parser. This is, to date, the most complete alternative Ruby parser, and boasts the ability to provide an entire AST or just a small subset. It functions by taking the parse.y from Ruby and modifying the actions. (This is the approach that almost every alternative implementation to CRuby ends up taking, as it's extremely difficult to replicate the exact parsing behavior without a copy of the grammar file.)

Ripper is one of the few projects of this era to survive the 1.8 to 1.9 migration, likely because Aoki was a core contributor and made sure it additionally worked on the YARV branch. Eventually this project would become the way that many other projects retreived AST information, though it was always marked as "experimental". Even today the README for ripper says Ripper is still early-alpha version..

Projects

Even though ripper has always been marked as "experimental", a number of projects rely on it for accessing the parse tree. They've mostly been using it since it got merged into trunk with 1.9. Those projects include:

• cane - a linter
• language_server-ruby - a language server
• prettier - a formatter
• rubyfmt - a formatter
• rufo - a formatter
• sandi_meter - a linter
• yard - a documentation generator

Links

• readme
• sample/comment.rb

2002-10-09 - MetaRuby 0.7.0

MetaRuby is a pretty interesting project that Mathieu Bouchard started around the Ruby 1.7 era. It was an attempt to implement a lot of the Ruby standard library in pure Ruby, predating the Rubinius project by a while. The reason I'm including it in this list is that in version 0.7.0, MetaRuby included RubySchema.rb, which was a method of validating the shape of the Ruby abstract syntax tree.

Links

• Overview
• RubySchema.rb

2004-11-10 - ParseTree 1.0.0

A little over a year after the release of 1.8, Ryan Davis released ParseTree. It used internals specific to 1.8 to build out s-expressions that you could programmatically access. Interestingly it was tested against all of the available gems at the time it was written with a tool called gauntlet.

Links

• README.txt
• lib/gauntlet_parsetree.rb

2006-06-05 - RubyNode 0.1.0

Still in the Ruby 1.8 to 1.9 timeline, Dominik Bathon released RubyNode. It was similar in spirit to ParseTree, except that it returns hashes of options on arrays and generally sticks a little closer to the internal node structure.

Links

• README.txt

2007-11-14 - ruby_parser 1.0.0

Just before 1.9 was released, Ryan Davis created a spiritual sequel to ParseTree that would function running with the 1.9 branch. He took the parse.y file from core Ruby and rewrote the actions to be pure Ruby before piping it through racc. This was a similar idea to ripper, except that it was in Ruby instead of C.

Links

lib/ruby_parser.y

Projects

A number of projects use ruby_parser as a way of accessing the syntax tree. Most of these were similarly created by Ryan Davis, but some others exist as well. Those include:

• dawnscanner - a security analyzer
• debride - an unused code analyzer
• fasterer - a performance linter
• flay - a code similarity analyzer
• flog - a code understandability analyzer
• railroader - a static security analyzer
• roodi - a linter

2009-10-26 - Ruby Intermediate Language

In 2009, a team of researchers at the University of Maryland, College Park put together a very interesting project meant to power a type system. It was a Ruby parser written in OCaml that standardized a lot of syntax into a small intermediate representation suitable for further analysis. The researchers (Michael Furr, Jong-hoon An, Jeffrey Foster, and Michael Hicks) presented it at the 2009 ACM conference.

Projects

This project went on to be a basis of further research projects that were all type systems that took advantage of the same parser, notably:

• druby
• rtc
• rubydust

2010-08-27 - laser 0.0.1

An interesting addition to this list is an excerpt from the academic side of Ruby. In 2010 Michael Edgar publishd his undergraduate thesis called laser (originally called wool, laser came from lexically- and statically-enriched Ruby). It was a linter, a type system, a documentation generation tool, and more. It featured a plugin system and performed semantic analysis. It was one of the more fully-fledged static analysis tools written for Ruby at this point. It originally parsed Ruby source using regular expressions just to check whitespace but upgraded to ripper eventually to do more complicated analyses.

2013-04-15 - parser 0.9.0

Around the 2.0.0 days, Peter Zotov created the parser gem. In the same tradition of other implementations of Ruby, it took the parse.y file from Ruby (and the lexer test suite from Ryan Davis' ruby_parser) and derived a new parser for creating syntax trees.

Links

• README.md
• lib/parser/ruby20.y

Projects

From its humble beginnings, parser ended up getting the attention of a lot of other developers because of its well-documented trees, easy-to-use APIs, and rewriting support. All kinds of other tools ended up being built on top of it, including:

• covered - a code coverage reporter
• deep-cover - a code coverage reporter
• erb-lint - an ERB file linter
• fast - an AST editor
• opal - a Ruby to JavaScript transpiler
• packwerk - an encapsulation analyzer
• querly - a method call finder
• rdl - a type checker
• reek - a code smell analyzer
• rubocop - a linter
• rubrowser - a module relationship grapher
• ruby-lint - a linter
• ruby-next - a transpiler and polyfill
• ruby_detective - a module relationship grapher
• rubycritic - a code quality reporter
• seeing_is_believing - an editor intermediate value display
• standard - a rubocop wrapper with fewer options
• steep - a static type checker
• unparser - a code generation from the parser AST
• vernacular - a source code manipulation
• yoda - a static analyzer and language server

2017-02-02 - tree-sitter 0.0.2

In earlier 2017, work began on tree-sitter-ruby, bringing Ruby support to the tree-sitter parser generator and incremental parsing library. While not entirely complete in terms of syntax, it's far enough along to power jump-to-definition and autocomplete tools.

The project itself is written in JavaScript like the other tree-sitter plugins, which makes it ideal to power a language server and VSCode plugin, which is exactly what it does in vscode-ruby.

2017-02-26 - typedruby

Taking the grammar from Ruby 2.4 and a rewrite of the lexer from ruby_parser, typedruby was created as a type system written in Rust. It gets a special notation here as it ended up being reused in other C++ projects, notably the sorbet type system (which vendors this parser).

Standards

2011-03-22 - JIS X 3017

Japanese Standards Association standard JIS X 3017.

2012-04-15 - ISO/IEC 30170:2012

International Organization for Standardization standard ISO/IEC 30170:2012.

Implementations

2001-09-10 - JRuby

In 2001, Jan Arne Petersen began work on a reimplementation of Ruby for the Java Virtual Machine that was a direct port of the Ruby 1.6 code in Java called JRuby. JRuby has undergone significant effort and change over the years. The parser itself has served as the basis for other Java projects to get their start as well, including TruffleRuby. Part of their efforts also involved reimplementing a lot of the standard library in Ruby itself, which also aided Rubinius and other projects.

Projects

In addition to helping other projects along with their reimplementation efforts, JRuby also served as the parser basis for ecstatic, a type system research project.

2006-07-12 - Rubinius

Following work he had done to create a reentrant Ruby parser and a book he had read on bytecode interpreters, Evan Phoenix began working on a Ruby-in-Ruby implementation that he called Rubinius. It took the parse.y of core Ruby and rewrote the actions to do things in the Ruby space. This actually went back and forth a bit between being implemented partly in C and partly in Ruby.

Projects

• melbourne - Rubinius's parser distributed as a gem
• pelusa - a linter that used the Rubinius infrastructure
• sydparse - a reentrant Ruby parser that ended up being the basis of the Rubinius project

2006-07-16 - Cardinal

Back in the Ruby 1.8 days, there was an interesting reimplementation of Ruby on the Parrot VM called Cardinal. This was one of the only attempts to reimplement Ruby that did not take the parse.y source and reengineer it. Instead it took the EBNF documentation from Ruby 1.6 and created its own PGE grammar.

2007-04-30 - IronRuby

Around the time that interest in Ruby implementations was starting to take off, Microsoft decided to invest in a Ruby implementation on the .NET framework called IronRuby. John Lam and the DLR Design Team worked on it for 3 years to make it able to run production workloads. Unfortunately in July 2010 Microsoft stopped funding the project, and ended up publishing it in 2011.

2008-03-13 - MacRuby

In 2008, Laurent Sansonetti began work on MacRuby, a Ruby 1.9 implementation that targetted the LLVM compiler infrastructure. It supported ahead-of-time and just-in-time compilation. It similarly took the parse.y and massaged it a bit. Fortunately, because it was written mostly in Objective-C, it could keep most of it in place because the actions were already written in C.

2012-04-07 - Topaz

In early 2012, work began on topaz, an implementation of Ruby targeting RPython. It also took the parse.y file from Ruby, but this time it reimplemented it in python with the grammar rules being written as function decorators.

2013-10-26 - TruffleRuby

In late 2013, a research team out of Oracle Labs forked JRuby and created TruffleRuby, a reimplementation of Ruby on the JVM that used the Graal dynamic compiler and the Truffle AST interpreter framework. As with JRuby, it used the same reengineered parser ported to Java.