Parsing error with the newest Linux kernel
xairy opened this issue · comments
I'm getting the following error when running my Kconfig-based script on the upstream Linux kernel:
kconfiglib.KconfigError: init/Kconfig:37: couldn't parse 'def_bool $(success,echo 'void foo(void) { asm inline (""); }' | $(CC) -x c - -c -o /dev/null)': macro expanded to blank string
The offending code is (added in commit eb111869301e1):
config CC_HAS_ASM_INLINE
def_bool $(success,echo 'void foo(void) { asm inline (""); }' | $(CC) -x c - -c -o /dev/null)
I see what the problem is. The preprocessor gets confused and thinks that the macro call ends after ...void). My fault, sorry about that!
Should be an easy fix (count opening/closing parentheses). I remember that logic from the C code, but somehow forgot to put it in.
Will fix tomorrow.
Here's a quickfix. Just needs some more cleanup and testing.
diff --git a/kconfiglib.py b/kconfiglib.py
index 3908985..45af2ac 100644
--- a/kconfiglib.py
+++ b/kconfiglib.py
@@ -2652,11 +2652,9 @@ class Kconfig(object):
start = i
i += 2 # Skip over "$("
- # Start of current macro argument
- arg_start = i
-
- # Arguments of this macro call
- new_args = []
+ arg_start = i # Start of current macro argument
+ new_args = [] # Arguments of this macro call
+ nest = 0 # Current parentheses nesting level
while 1:
match = _macro_special_search(s, i)
@@ -2664,7 +2662,16 @@ class Kconfig(object):
self._parse_error("missing end parenthesis in macro expansion")
- if match.group() == ")":
+ if match.group() == "(":
+ nest += 1
+ i = match.end()
+
+ elif match.group() == ")":
+ if nest:
+ nest -= 1
+ i = match.end()
+ continue
+
# Found the end of the macro
new_args.append(s[arg_start:match.start()])
@@ -2687,6 +2694,10 @@ class Kconfig(object):
len(prefix))
elif match.group() == ",":
+ if nest:
+ i = match.end()
+ continue
+
# Found the end of a macro argument
new_args.append(s[arg_start:match.start()])
arg_start = i = match.end()
@@ -7016,7 +7027,7 @@ _assignment_lhs_fragment_match = _re_match("[A-Za-z0-9_-]*")
_assignment_rhs_match = _re_match(r"\s*(=|:=|\+=)\s*(.*)")
# Special characters/strings while expanding a macro (')', ',', and '$(')
-_macro_special_search = _re_search(r"\)|,|\$\(")
+_macro_special_search = _re_search(r"\(|\)|,|\$\(")
# Special characters/strings while expanding a string (quotes, '\', and '$(')
_string_special_search = _re_search(r'"|\'|\\|\$\(')There also seems to also be a dependency loop now after that though. It's probably legit, because some defconfigs trigger infinite recursion, and just not being detected by the C tools due to expression rewriting stuff that they do. Bleh...
As a quickfix for that one, this will probably work (if it doesn't for your case, you'll get a stack blowup):
diff --git a/kconfiglib.py b/kconfiglib.py
index 45af2ac..d89d40b 100644
--- a/kconfiglib.py
+++ b/kconfiglib.py
@@ -1072,9 +1072,9 @@ class Kconfig(object):
self._build_dep()
# Check for dependency loops
- check_dep_loop_sym = _check_dep_loop_sym # Micro-optimization
- for sym in self.unique_defined_syms:
- check_dep_loop_sym(sym, False)
+ #check_dep_loop_sym = _check_dep_loop_sym # Micro-optimization
+ #for sym in self.unique_defined_syms:
+ # check_dep_loop_sym(sym, False)
# Add extra dependencies from choices to choice symbols that get
# awkward during dependency loop detectionWill need to check what can be done about it.
Pushed out a new 12.14.1 release with a fix for the preprocessor issue.
Still that pesky dependency loop left. Will look into it a bit later.
Your patch fixes the original issue, thanks!
As you pointed out, I'm now hitting the dependency loop issue. The quick fix doesn't work in my case, I'm getting the RuntimeError: maximum recursion depth exceeded crash.
Looking forward to a fix, thanks!
Bisected it to this commit.
Here's a quickfix to get it parsing:
diff --git a/drivers/gpu/drm/tiny/Kconfig b/drivers/gpu/drm/tiny/Kconfig
index 504763423d46..49d0b742b8bf 100644
--- a/drivers/gpu/drm/tiny/Kconfig
+++ b/drivers/gpu/drm/tiny/Kconfig
@@ -63,7 +63,7 @@ config TINYDRM_REPAPER
depends on DRM && SPI
select DRM_KMS_HELPER
select DRM_KMS_CMA_HELPER
- depends on THERMAL || !THERMAL
+ #depends on THERMAL || !THERMAL
help
DRM driver for the following Pervasive Displays panels:
1.44" TFT EPD Panel (E1144CS021)Strangely, the C tools detect a loop too if it's changed to depends on THERMAL.
That THERMAL || !THERMAL thing is a hack that turns it into depends on m if THERMAL is m and into depends on y otherwise.
Need to dig a bit more to see exactly what's going on here.
Removing stuff to find a minimal Kconfig that gives different behavior.
Debugging these tool differences is always a bit of a pain.
Edit: Running into a bunch of segfaults with the C tools now, so leaning towards an issue there. Could be unrelated though. Need to minimize some more.
Might've been sidetracked. Can segfault the C tools in three lines like this (crashes when printing the loop message... guessing because B isn't defined):
config A
bool
select B if B
Back to minimizing while avoiding segfaults. :/
Figured out what the issue is.
Boils down to this (after some rewriting):
config TINYDRM_REPAPER
tristate "DRM support for Pervasive Displays RePaper panels (V231)"
depends on THERMAL || !THERMAL
config THERMAL
bool "Generic Thermal sysfs driver"
depends on TINYDRM_REPAPER
Kconfiglib sees that as a dependency loop, while the C implementation simplifies THERMAL || !THERMAL to y before running dependency loop detection, and so never sees it.
That simplification is hiding a bug in the kernel Kconfig files in this case: THERMAL || !THERMAL is pointless (always y), due to THERMAL being bool. If you change the type of THERMAL to tristate, the C tools see the loop too.
Fixing that bug in the kernel Kconfig files will indirectly fix the dependency loop in Kconfiglib as well. I'll see if I can send a patch later.
@xairy
Sent a patch upstream that fixes it: https://www.spinics.net/lists/linux-kbuild/msg23455.html
@ulfalizer I've tested your patch earlier today and it fixed the issue for me. Thanks for looking into this!