Grammar "C18" fix for "extern_langauge_declaration"
mingodad opened this issue · comments
Testing the grammar "C18" with sqlite3.h I found that the rule extern_langauge_declaration should use [repeat function_definition_or_declaration] instead of [repeat declaration] because inside extern "C" {} block we can have again everything from a translation unit.
Also it fails to parse float point numbers like:
double dm = 1.1;
Output:
txl -p -v -Dscan test.c c.txl
OpenTxl v11.3.2 (1.4.24) (c) 2024 James R. Cordy and others
Bootstrapping TXL ...
... used 401 trees and 251 kids.
Scanning the TXL program c.txl
Parsing the TXL program
... used 10693 trees and 15975 kids.
Making the input language grammar tree
... used 1296 trees and 1518 kids.
Making the rule table
... used 2 trees and 0 kids.
Scanning the input file test.c
<key text="double"/>
<id text="dm"/>
<literal text="="/>
<floatnumber text="1.1"/>
<literal text=";"/>
Parsing the input file
[test.c, c.txl] : TXL0192E line 1 of test.c - Syntax error at or near:
double dm = >>> 1.1 <<< ;
When using the same definitions on Cpp.grm it does parse fine, so it's some other definition that's messing things.
Would be nice to the -analyze option to also analyze the scanner.
Where did you get a TXL C18 grammar?
The OpenTXL grammars have not yet been released.
From a link in http://www.txl.ca/txl-resources.html -> http://www.txl.ca/examples/Grammars/C18/C18.tar.gz
OK, obsolete, and not part of this repo. You'll find it when the OpenTXL grammar repo is released.
Where can I find a recent collection of grammars ?
Where can I find a recent collection of grammars ?
That's being negotiated with their owners. Hopefully to be open-sourced sometime soon.
It seems that you are one of the authors of it, aren't you the owner of it ?
Validated TXL Basis Grammar for C18 with Macros and Gnu Extensions
Version 6.4, November 2020
Copyright 1994-2020 James R. Cordy, Andrew J. Malton and Christopher Dahn
Licensed under the MIT open source license, see source for details.
Description:
Consolidated grammar for C18, ANSI, and K+P C with Gnu extensions
designed for large scale C analysis tasks. Validated on a large range
of open source C software including Bison, Cook, Gzip, Postgresql, SNNS,
Weltab, WGet, Apache HTTPD, the entire Linux 2.6 kernel, and the entire
FreeBSD 8.0 kernel.
Handles both preprocessed and unpreprocessed C code with with expanded or
unexpanded C macro calls.
Optionally handles but does not interpret C preprocesor directives,
except #ifdefs that violate syntactic boundaries. #ifdefs can be handled
using the separate Antoniol et al. transformation that keeps only the #else parts
and comments out the optional (#if, #elsif) parts (ifdef.txl).
Ignores and does not preserve comments. Optionally accepts and preserves
comments using an added approximate comment grammar (c-comments.grm).
Authors:
J.R. Cordy, Queen's University
A.J. Malton, University of Waterloo
C. Dahn, Drexel University
Example:
txl program.c c.txl
txl porogram.c ifdef.txl > program_ifdef.c; txl program_ifdef.c c.txl
Notes:
1. The syntax of the C language is not context-free, and there will always be
cases that cannot be accurately parsed by a context-free grammar.
In particular, there is an ambiguity between statements and declarations that
cannot be resolved without a two-pass parse.
2. While this grammar handles most unexpanded Gnu/Linux-style macro calls,
it cannot do so for all cases since macros may hide additional syntax.
Rev. 13.11.20
Sure, I am the owner of that one, and yes, the OpenTxl project one is based on it. But all recent work has been funded by an international company, and I need their agreement to release it. TXL has never been Gnu licensed (and never will be).
You're right on both issues, but they belong in https://github.com/CordyJ/OpenTxl-Grammars.
Since that repo is private, the solution to the second issue is to replace both occurrences of floatnumber with decfloatnumber in c.grm.
Thank you for reply !
Looking for occurrences of decfloatnumber in c.grm and finding none I did searched in all folders in my computer that are related to txl and found that https://github.com/CordyJ/Open-NiCad/tree/main/src/txl has more recent versions of several grammars including c.grm (that already has the decfloatnumber fix but is missing the extern_langauge_declaration fix, probably you could fix it there !)
Looking for occurrences of decfloatnumber in c.grm and finding none
You were to replace the word floatnumber with decfloatnumber, not the other way around. There are two occurrences in c.grm.
NiCad has grammars specially customized for it. In general, they are not the same as the general TXL grammars for the languages, although this one may be the same.
Thank you again for reply !
I understood that and I first tried to see if the decfloatnumber was already defined anywhere in it and when I didn't found then I searched everywhere I suspected I could find it.