Two !includes

[Chris Cleveland]

I'm trying to !include two different files in my lexer (or three if you include the top header):

/*!include:re2c "unicode_categories.re" */

pub fn lex_query(input: &[u8]) -> Vec<QuickTokenType> {
    let mut pos = 0;
    let mut start = 0;
    let len = input.len();
    let mut mark = 0;
    let mut tokens = vec![];
    use QuickTokenType::*;

    'lex: loop {
        // @formatter:off
        /*!re2c
        re2c:define:YYCTYPE = u8;
        re2c:define:YYPEEK  = "peek(input, pos, len)";
        re2c:define:YYSKIP  = "pos += 1;";
        re2c:define:YYBACKUP  = "mark = pos;";
        re2c:define:YYRESTORE = "pos = mark;";
        re2c:yyfill:enable = 0;

        !include "uax29.re";
        !include "query_tokens.re";

        [\x00] { return tokens;              /* EOF */ }
        *      { start = pos; continue 'lex; /* skip unrecognized */ }
        */
        // @formatter:on
    }
}

The problem is that the second one, "query_tokens.re", is ignored, as if it's not there at all.

If I comment out the first one, "uax29.re", then the second one gets included properly and ends up in the generated code.

[Chris Cleveland]

Never mind. There was a rule in the first file that was masking a rule in the second file. When I swapped the order of the files then the rule I wanted showed up properly.

This raises a question: is there a way to get notified when rules are in conflict? Suppose:

word = [a-z]+;

word { println!("word"); }
"hello" { println!("hello"); }

"hello" will not get recognized and fail silently. Swap the order of the bottom two lines and it will be recognized just fine.

[Ulya Trofimovich]

is there a way to get notified when rules are in conflict?

Yes, -Wunreachable-rules is supposed to do that. Or you can use -W to enable all warnings (see help or manpage for a list of warnings with description).