YAML/JSON Support for Calyx

[Tariq Ali]

So I decided to spend some time working on getting Calyx to read YAML.

      def load(file)
        klass = Class.new(Calyx::Grammar)
        yaml = YAML.load_file(file)
        yaml.each do |key, value|
          if key == :start
            klass.send(:start, value)
          else
            klass.send(:rule, key, value)
          end
        end
        klass.new
      end

This method is not ready for production though because it cannot handle rules with more than one possible choice yet. So far, this is the type of YAML file it can handle:

:start: :hello_world
:hello_world: "Hello World"

But I don't think it's too difficult to get our method to handle multiple choices.

What I'm wondering is what would be the use for a person to write YAML for Calyx instead of writing the raw Ruby code. Is it to make Calyx simpler for non-programmers? Or is just because sometimes a programmer want to separate out data from code? I'm just curious, even though this does sound like an interesting challenge of building an abstraction on top of an existing abstraction. (And if I do get this method working, it should not be hard to generalize it to handle JSON objects as well. Or maybe even Tracery grammars...but I might be getting a bit ahead of myself.)

[Mark Rickerby]

Hi,

This is one of the concepts I haven’t thought through or documented in a lot of detail. But the basic motivation/rationale for constructing grammars via JSON or YAML is to provide the capability to edit grammars in more of a declarative/document format than Ruby code. It provides the foundation for building higher level tools and GUIs for editing grammars and also the potential for sharing grammars between different libraries in various languages.

Is it to make Calyx simpler for non-programmers? Or is just because sometimes a programmer want to separate out data from code?

Both of these make sense, but probably the first one more than the second.

[Mark Rickerby]

Or maybe even Tracery grammars...

Yes, that was definitely one of the things I had in mind for this, long-term.

[Tariq Ali]

I notice that every issue on the To-Do List is done with the one exception of JSON/YAML support. Do you have any additional features or refactoring being planned here? Or is this meant to be a more general statement for supporting other file formats, such as Tracery? I'm curious to know how I can best contribute to help this project succeed.

[Mark Rickerby]

Nothing major planned. The basic feature seems to be working (thanks!), but I want to spend a bit of time reviewing the JSON and YAML serialization formats and identifying anything that could be tweaked/improved. Weighted probability format is the main thing I’ve picked up so far. Everything else seems minor.

[Mark Rickerby]

Finally got around to documenting this in the README and adding some examples.