sql-ddl-parser: parser ignores valid string tokens that are also keywords

[David Baynard]

The following is valid in sqlite.

CREATE TABLE version (    schema TEXT PRIMARY KEY,    version INT NOT NULL);

The parser doesn’t recognise the column name schema, though.

Example

The parser returns the following.

[
  {
    "type": "create.table",
    "name": "version",
    "comment": "",
    "columns": [
      {
        "name": "",            // Missing `schema`
        "dataType": "TEXT",
        "default": "",
        "comment": "",
        "primaryKey": true,
        "autoIncrement": false,
        "unique": false,
        "nullable": true
      },
      {
        "name": "version",
        "dataType": "INT",
        "default": "",
        "comment": "",
        "primaryKey": false,
        "autoIncrement": false,
        "unique": false,
        "nullable": false
      }
    ],
    "indexes": [],
    "foreignKeys": []
  }
]

The tokenizer returns the following.

[ {"type":"keyword"    , "value":"CREATE" }
, {"type":"keyword"    , "value":"TABLE"  }
, {"type":"string"     , "value":"version"}
, {"type":"leftParen"  , "value":"("      }
, {"type":"keyword"    , "value":"schema" } // Not a keyword!
, {"type":"keyword"    , "value":"TEXT"   }
, {"type":"keyword"    , "value":"PRIMARY"}
, {"type":"keyword"    , "value":"KEY"    }
, {"type":"comma"      , "value":","      }
, {"type":"string"     , "value":"version"}
, {"type":"keyword"    , "value":"INT"    }
, {"type":"keyword"    , "value":"NOT"    }
, {"type":"keyword"    , "value":"NULL"   }
, {"type":"rightParen" , "value":")"      }
]

I wouldn’t use a keyword as a column name, but it seems the zotero developers haven’t run into any issues with it…

I don’t know what’s best here… this is valid sql, and so if sql-ddl-parser is going to handle valid sql then it should handle this. The current token handling ignores the keyword context, though, making this tricky.

https://github.com/vuerd/vuerd/blob/ef85789d778219d4b5e8fe412fbab7ee3599783f/packages/sql-ddl-parser/src/SQLParser.ts#L173-L179

It would be great if sql-ddl-parser were to handle this, but I appreciate adding context would be a whopping change, that might not be worthwhile.

The context is available in the parser. A hacky alternative might be to allow keywords here.

https://github.com/vuerd/vuerd/blob/ef85789d778219d4b5e8fe412fbab7ee3599783f/packages/sql-ddl-parser/src/sqlParser/create.table.ts#L121-L125

Of course, not all keywords are allowed as unquoted column names; commit, for example, causes an error.

A more prinicpled approach might be to identify and then allow valid keywords here.

---

What do you think?

[SeungHwan-Lee]

I think it's working after rewriting the parser
erd-editor v3.0.1
erd-editor-vscode v1.0.1