How to parse a certain text according to a bnf grammar and get a list of token
pankajrlal opened this issue · comments
The examples provided take a bnf grammar description and then are able to generate text that matches the grammar. How do I give the grammar a string compliant with bnf specification and then get a list of tokens out? This is something that pyparsing does.
Hi @prmammoth that's a feature I've been thinking about implementing for awhile now, I do think it'd be a valuable addition to this library. Taken in steps, I think the most straight forward approach is to implement an Earley parser that would be able to validate that input is or is not valid for the given grammar, then use the result to generate a "parse forest" of possible paths to that input from the grammar 🤔
I might take a swing at implementing this over the next week or two, if I have anything interesting to report I'll fill ya in here.
Quick update, I haven't forgotten abt this. I've got a prototype working for grammars accepting/rejecting sentences but am still chipping at returning the parse trees that show how a sentence fits the grammar. More specifically, I have an initial Earley parser worked out, just haven't wrapped up the logic to take the result and create a parse forest from it.
In case anyones interested in what I mean by "I have an initial Earley parser worked out", I'm hacking around on it all over here https://github.com/shnewto/earley. Fwiw, I suspect the tests there would be more informative than the prototype code 🤷♂️
@shnewto any update on this? Has the feature been abandoned?
Hi @artman41 thanks for pinging, this isn't abandoned but it's also not implemented yet. I'm hoping to have some happy updates regarding it in the next couple weeks.
just wanted to share that I've been poking at this feature recently. nothing to show yet, but it has not been forgotten!
@prmammoth @artman41 we've merged the implementation of this feature and are drafting the release for later today! it was a long time coming 😄 check out the example in the readme and reach out or create another issue if you have any questions!
Wonderful. Thanks a lot