More issues with Annotated

Question

More issues with Annotated

hodgespodge opened this issue a year ago · comments

Howdy again y'all.

I'm having issues with Annotated that lead me to believe issue #120 was not actually resolved.

After delving deeper It looks like functions with an Annotated type check just aren't working. Specifically, only the first specified constraint (the type) of the Annotated signature is actually being checked.

import sys
from beartype.vale import Is
from plum import dispatch
from typing_extensions import Annotated

class MyClass():
    def __init__(self) -> None:
        self.special_attr = 'special_value'
class AnotherClass():
    pass # notable difference: no special_attr

class ClassWithAnnotatedTypeChecker():

    class_type_check = Annotated[object, Is[lambda value: hasattr(value,'special_attr')]]

    @dispatch
    def myFunction(self, value: class_type_check):
        if hasattr(value,'special_attr'):
            print(f'Value has special_attr={value.special_attr}')
        else:
            print(f'ERROR: This code should not be reached- 1')


    string_type_check = Annotated[str, Is[lambda value: len(value) > 3]]

    @dispatch
    def myFunction(self, value: string_type_check):
        if len(value) > 3:
            print(f'Value is a string={value}')
        else:
            print(f'ERROR: This code should not be reached- 2')

    @dispatch
    def myFunction(self, value: int):
        print(f'Value is an int={value}')

my_class_with_annotated_type_checker = ClassWithAnnotatedTypeChecker()

my_class = MyClass()
print(Is[lambda value: hasattr(value,'special_attr')].is_valid(my_class)) # Returns True
my_class_with_annotated_type_checker.myFunction(my_class) # Works

another_class = AnotherClass()
print(Is[lambda value: hasattr(value,'special_attr')].is_valid(another_class)) # Returns False
my_class_with_annotated_type_checker.myFunction(another_class) # Should fail to dispatch... Instead, it dispatches to the str function

print(Is[lambda value: len(value) > 3].is_valid('Hello')) # Returns True
my_class_with_annotated_type_checker.myFunction('Hello') # Works

print(Is[lambda value: len(value) > 3].is_valid('Hi')) # Returns False
my_class_with_annotated_type_checker.myFunction('Hi') # Should fail to dispatch... Instead, it dispatches to the str function

my_class_with_annotated_type_checker.myFunction(1)

Filippo Vicentini · Answer 1 · Tue Oct 17 2023 05:03:18 GMT+0800 (China Standard Time)

MWE

In [1]: import sys
   ...: from beartype.vale import Is
   ...: from plum import dispatch, Signature
   ...: from typing_extensions import Annotated
   ...:
   ...: class MyClass():
   ...:     def __init__(self) -> None:
   ...:         self.special_attr = 'special_value'
   ...: class AnotherClass():
   ...:     pass # notable difference: no special_attr
   ...:
   ...: class_type_check = Annotated[object, Is[lambda value: hasattr(value,'special_attr')]]
   ...:
   ...: def g(value: class_type_check):
   ...:     if hasattr(value,'special_attr'):
   ...:         print(f'Value has special_attr={value.special_attr}')
   ...:     else:
   ...:         print(f'ERROR: This code should not be reached- 1')
   ...:
   ...: from plum.signature import resolve_pep563
   ...:
   ...: print(g.__annotations__)
   ...: resolve_pep563(g)
   ...: print(g.__annotations__)
   ...:
{'value': typing.Annotated[object, Is[lambda value: hasattr(value, 'special_attr')]]}
{'value': <class 'object'>}

it's our logic in

def resolve_pep563(f: Callable):
    """Utility function to resolve PEP563-style annotations and make editable.

    This function mutates `f`.

    Args:
        f (Callable): Function whose annotations should be resolved.
    """
    if hasattr(f, "__annotations__"):
        beartype_resolve_pep563(f)  # This mutates `f`.
        # Override the `__annotations__` attribute, since `resolve_pep563` modifies
        # `f` too.
        for k, v in typing.get_type_hints(f).items():
            f.__annotations__[k] = v

that breaks. Not beartype.

However, I really wonder if we can/should support this kind of type hints with plum...

Filippo Vicentini · Answer 2 · Tue Oct 17 2023 05:05:03 GMT+0800 (China Standard Time)

Digging deeper, the issue is with typing.get_type_hints (FYI @leycec ) which strips all annotations...

 def g(value: class_type_check):
     if hasattr(value,'special_attr'):
         print(f'Value has special_attr={value.special_attr}')
      else:
          print(f'ERROR: This code should not be reached- 1')
      

typing.get_type_hints(g)
{'value': object}
``

Cecil Curry · Answer 3 · Tue Oct 17 2023 08:58:05 GMT+0800 (China Standard Time)

...heh. I have been summoned and shall now dispense justice with the bear claw. Repeat after me, everyone:

Blow, blow, blow your bugs!
Gently down the gitter.
Unbearably, unbearably, unbearably, unbearably,
typing is but a dream.

Indeed, the typing module is blatantly busted in all possible ways. They mean well, but have no idea what they are doing.

This is why @beartype internally defines its own private utility functions rather than ever call anything published by typing. In this case, typing.get_type_hints() maliciously reduces PEP 593-compliant typing.Annotated[{type}, ...] type hints to simply {type}. It does other bad things as well in other unrelated edge cases. I don't know why. None of it really makes sense. I sigh and then shake my feeble little fist. ✊

I'm kinda unsure why Plum is even calling typing.get_type_hints(), though. Like, what's this suspicious iteration about?

        for k, v in typing.get_type_hints(f).items():
            f.__annotations__[k] = v

Ideally, that should be a noop. Does that actually do something meaningful? I rub my chin thoughtfully.

Wessel · Answer 4 · Wed Oct 18 2023 03:49:59 GMT+0800 (China Standard Time)

Hmm, I believe the original motivation for calling typing.get_type_hints was that it handles non-PEP563 forward references:

# None of this!
# from __future__ import annotations

from typing import get_type_hints

class A:
    def f(self, x: "A"):
        return x


a = A()
print(get_type_hints(a.f))
# {'x': <class '__main__.A'>}

@leycec, does Beartype happen to implement a utility function that would be superior replacement for typing.get_type_hints? Might we be so lucky?!

Wessel · Answer 5 · Sun Jan 21 2024 18:04:15 GMT+0800 (China Standard Time)

With @hodgespodge recent PR #126, this is now fixed:

from typing import Annotated

from beartype.vale import Is

from plum import dispatch


class SpecialClass:
    def __init__(self) -> None:
        self.special_attr = "special_value"


class NormalClass:
    pass


@dispatch
def f(value: Annotated[object, Is[lambda value: hasattr(value, "special_attr")]]):
    print("Value has `special_attr` defined!")


f(SpecialClass())
f(NormalClass())

Also note the pretty output due to @PhilipVinc's work. :D

Cecil Curry · Answer 6 · Tue Jan 23 2024 16:20:44 GMT+0800 (China Standard Time)

OMG. I so dropped the ball on this one, @wesselb:

@leycec, does Beartype happen to implement a utility function that would be superior replacement for typing.get_type_hints? Might we be so lucky?!

I... totally missed that desperate request. The answer, of course, is: "Probably! @beartype probably did have exactly what you needed!" Which is a shame, too. Dynamic evaluation of arbitrary forward references at runtime is really non-trivial. But... you must have already done it here, somehow. You made miracles happen without us.

I'm so sorry. @beartype definitely should have done that for Plum. If anyone's curious, @beartype has extensive facilities for resolving forward references in its currently private beartype._check.forward subpackage.

The most useful for Plum purposes would almost certainly be the beartype._check.forward.fwdref.make_forwardref_indexable_subtype() factory function. It's super-trivial. You just pass it an absolute module name and relative forward reference. That function then dynamically creates and returns a new forward reference proxy object, which dynamically (A) looks up the (possibly subscripted) type to which that proxy refers in a deferred manner (i.e., it waits as long as possible to do this) and (B) proxies isinstance() and issubclass() calls to that type.

The beartype.peps.resolve_pep563() function that Plum probably already calls internally leverages that functionality to resolve PEP 563-postponed forward references on behalf of Plum and everybody else.

Yar, I Speak Like a Pirate Now.

Incidentally, are we all aware that PEP 563 has now officially been deprecated and will be replaced by something that's probably equally broken called "annotation scopes" in Python ≥ 3.13 or 3.14, I think? I've been fighting annotation scopes throughout the @beartype 0.17.0 release cycle. So, I can confidently be the harbinger of doom: "Thare be trouble ahead, yar."

In fact, annotation scopes are so busted that I'm gonna have to push out a mass email to the official CPython typing-sig working group in a few weeks. We might have another Python 3.10-style "ohsh--" moment on our hands where the BFDL and Microsoft and OpenAI get involved and have to push back the next Python release date. It's ugly stuff.

Like, breaking PyTorch and ChatGPT scale of ugly stuff. Incredibly ugly stuff is what I'm saying. Sucks, honestly. Can't believe CPython is doing this after the PEP 563 debacle but... here we go again.

Wessel · Answer 7 · Thu Jan 25 2024 03:36:11 GMT+0800 (China Standard Time)

@leycec Nonono, you've got so much on your hands—there's absolutely no obligation to reply to every little request. :)

The most useful for Plum purposes would almost certainly be the (...)

Aha, this is amazing. Yes, it looks like beartype._check.forward could replace typing.get_type_hints! That's amazing. I'm going to look into this.

Like, breaking PyTorch and ChatGPT scale of ugly stuff. Incredibly ugly stuff is what I'm saying. Sucks, honestly. Can't believe CPython is doing this after the PEP 563 debacle but... here we go again.

Dear god, I don't even want to know. 😭 Why must typing in Python be so painful... Well, I hope your mass email reaches the right people.

Cecil Curry · Answer 8 · Thu Jan 25 2024 11:09:10 GMT+0800 (China Standard Time)

Yes, it looks like beartype._check.forward could replace typing.get_type_hints!

Superbness! Just let me know what sort of a public API Plum could benefit from and we'll make this magical unicorn happen. I'm unsure if @beartype currently has an exact analogue to typing.get_type_hints, but it's likely we have something that either (A) is already "close enough" or (B) can be hammered out until it's "close enough."

This is why I play video games before passing out. We didn't get a winter here in Canada, so Nordic skiing is right out. "Why so much freezing rain!?," I cry. Hope you're having an awesome winter in Amsterdam, though! May Plum live long and prosper. 🖖

Wessel · Answer 9 · Fri Jan 26 2024 16:51:26 GMT+0800 (China Standard Time)

@leycec That's fantastic. :) Will definitely do. I think we don't need an exact analogue of typing.get_type_hints—we should be able to do what we do now with beartype._check.forward.

By the way, very happy birthday!!! (And a massive congrats with the 0.17.0 release! :)) I hope you're having an awesome and relaxing day today. 😌

Cecil Curry · Answer 10 · Wed Jan 31 2024 14:39:03 GMT+0800 (China Standard Time)

Thanks so much for the kindest words from what is probably the Amsterdam coffee shop I wish I was inhabiting. Let me know if I can do anything to improve either the internal API or documentation of beartype._check.forward. Your will be done.

Oh – and please do let me know which currently private functions and/or types you eventually settle on calling. To avoid breaking Plum, I'd like to "quietly" publicize those somewhere – probably in beartype.peps, which is rapidly becoming the horse glue that binds @beartype + Plum. 🐴 🖌️

Wessel · Answer 11 · Sun Feb 11 2024 23:17:48 GMT+0800 (China Standard Time)

@leycec Will definitely do! :) I haven't yet had the opportunity to do this, but will let you know as soon as I do.