Configurable Parser
hasier opened this issue · comments
It's not a common use case, but I would find it very useful to be able to tweak how the Parser
works. For my use case, I'd like to tweak the keywords the Parser
recognises (for
, if
, block
...) in order to restrict them as necessary (but not extend them, as that would involve changing the actual behaviour of the Parser
, which I'm not aiming to tackle here).
Currently the Parser
recognises the keywords from a module-level constant, which is not easily editable just for 1 Parser
class.
Lines 18 to 33 in 953acd6
Besides, even if it were possible to specifically change a
Parser
, say by subclassing it and dynamically overwriting some of the methods, we would need to further subclass an Environment
and override some of its methods, as the Parser
is hardcoded in them.jinja/src/jinja2/environment.py
Lines 615 to 619 in 953acd6
I would find much easier to implement this behaviour if the Parser
had a class-level variable that could be swapped in an inheriting class, and the same with a class-level variable for the Parser
in the Environment
class. This is a pattern that's already followed for other internals, such as the CodeGenerator
class via the code_generator_class
class-variable.
jinja/src/jinja2/environment.py
Line 284 in 953acd6
See #1194. I plan to rewrite the parser and make it more extensible, so adding this right now only for the entire API to change doesn't seem ideal.
In your case, you can probably use an extension: https://jinja.palletsprojects.com/en/3.1.x/extensions/#writing-extensions
Or overriding Environment.parse
is fine, it's the only part of the parser API that's publicly documented.
Thanks for the suggestions @davidism!
Regarding the extensions, I don't think I can make use of that, because they seem to be only taken into account if the token does not match any of the predefined values, but what I need to do is to specifically override some of the default _statement_keywords
.
Lines 173 to 182 in 953acd6
About Environment.parse
, that's what I'm currently doing, and it's not the end of the world indeed, it just felt cleaner to have the class variable to configure it. I would still need to change the behaviour of the Parser itself, though.
Finally, #1194 looks exciting! I see that's still a work in progress and there seems to be quite some road ahead. I am guessing it will include some breaking changes in order to establish the new API when the time comes, so I was wondering if the changes I proposed could bridge the gap in the meantime? I could really use them instead of dynamically faffing with the available methods in the Parser, or overriding the corresponding methods and re-writing them in a child class. I managed to do it, so it's not a massive blocker, I was just feeling that something like this approach would make things simpler.
Anyway, I appreciate it might still not be the approach you want to go for in any case and would rather keep it as is 🙂 Thanks for taking a look!
it just felt cleaner to have the class variable to configure it
Subclassing and overriding methods is an accepted and standard way to extend things in both Python and Jinja.