Lightning CLI argparse error with specific python versions on 3.11 and 3.12
nilsleh opened this issue · comments
🐛 Bug report
We using the lightning CLI for a package we are trying to develop called lightning-uq-box. For testing we also utilize the CLI and recently encountered CI pytests failing on python 3.11. Tests are passing on main, where the CI uses python 3.11.8, but on more recent PRs, python 3.11.9 is picked and we encounter the following error:
lightning_uq_box/main.py:11: in get_uq_box_cli
return LightningCLI(
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/lightning/pytorch/cli.py:378: in __init__
self.parse_arguments(self.parser, args)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/lightning/pytorch/cli.py:528: in parse_arguments
self.config = parser.parse_args(args)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/jsonargparse/_deprecated.py:124: in patched_parse
cfg = parse_method(*args, _skip_check=_skip_check, **kwargs)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/jsonargparse/_core.py:390: in parse_args
cfg, unk = self.parse_known_args(args=args, namespace=cfg)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/jsonargparse/_core.py:261: in parse_known_args
namespace, args = self._parse_known_args(args, namespace)
/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/argparse.py:2128: in _parse_known_args
start_index = consume_optional(start_index)
def consume_optional(start_index):
# get the optional identified at this index
option_tuple = option_string_indices[start_index]
> action, option_string, sep, explicit_arg = option_tuple
E ValueError: not enough values to unpack (expected 4, got 3)
To reproduce
I have made a separate PR branch, where I have just made a random change to the README.md where one can see that the python version on the PR used is 3.11.9 while on the main branch it is still 3.11.8 (where the tests pass). The same thing happens for python 3.12 now, where 3.12.3 breaks, but 3.12.2 (on main) still passes.
Environment
Specified also in requirements/tests.yaml
- jsonargparse version: jsonargparse[signatures]==4.27.5
- Python version: 3.11, 3.12
- How jsonargparse was installed : pip install
- OS: Ubuntu, Mac, Linux
https://docs.python.org/3.12/whatsnew/changelog.html#python-3-12-3-final contains a few argparse-related changes. I'm guessing jsonargparse relies on undocumented or untested argparse internals and something changed.
It looks like this specific commit broke things for us: python/cpython@c02b7ae, python/cpython#114180
Thank you for reporting! This issue has already been fixed in #486. The fix will be included in a release tomorrow or the day after.