Add support for `typing.Protocol` classes in `CLI`

[Ruslan Mukhametshin]

🚀 Feature request

Add support for typing.Protocol classes in CLI.

Motivation

Currently I cannot use protocol classes for type annotations. For example:

In main.py:

import typing as tp

from jsonargparse import CLI


@tp.runtime_checkable
class ModelType(tp.Protocol):
    def predict(self, items: tp.Sequence[tp.Any]) -> tp.Any:
        ...


class Model:

    def __init__(self, batch_size: int) -> None:
        self._batch_size = batch_size

    def predict(self, items: tp.Sequence[tp.Any]) -> tp.Any:
        print(f"Predicting {len(items)} items with a batch size {self._batch_size}")


def main(model: ModelType) -> None:
    print(model)
    model.predict([1, 2, 3])


if __name__ == "__main__":
    CLI(main)

In config.yaml:

model:
  class_path: __main__.Model
  init_args:
    batch_size: 1

In terminal:

python -m main --config config.yaml

Prints <class '__main__.Model'> and gives error: TypeError: predict() missing 1 required positional argument: 'items'

Pitch

The above code should run without errors. Model instance should be printed, not the class.

Alternatives

[Mauricio Villegas]

Thank you for proposing this! I have thought in the past about adding support for Protocol. But note that runtime_checkable protocols don't work as people might expect, see for example https://discuss.python.org/t/is-there-a-downside-to-typing-runtime-checkable/20731/2. That is, for another class like

class OtherModel:
    def predict(self, different, parameters) -> different_return:
        ...

evaluation of issubclass(OtherModel, ModelType) gives True even though OtherModel is not really compatible.

The reason why currently the class __main__.Model is given to main is also because of a quirk in runtime_checkable, which is that isinstance(Model, ModelType) also evaluates to True, see _typehints.py#L972-L973. Fixing this would be easy. But I am inclined to not fix it. Better to keep it broken than fixing it for runtime_checkable and users getting surprising results.

Preferable to implement support for Protocol without the need to be runtime_checkable and do a more strict check to make sure that accepted classes actually match the protocol's signature.

[Ruslan Mukhametshin]

That would be great!

Preferable to implement support for Protocol without the need to be runtime_checkable and do a more strict check to make sure that accepted classes actually match the protocol's signature.

[Mauricio Villegas]

@rusmux please have a look at #526. It is only for exact signature match, otherwise it would become much more complex.

[Ruslan Mukhametshin]

That works, thank you!