zware / desugar

Unravelling Python source code

desugar

Unravelling Python's syntactic sugar source code.

There are accompanying blog posts to go with all of the code in this repository.

Unravelled syntax

1. obj.attr ➠ builtins.getattr(obj, "attr") (including object.__getattribute__())
2. a + b ➠ operator.__add__(a, b)
3. a - b ➠ operator.__sub__(a, b)
4. a * b ➠ operator.__mul__(a, b)
5. a @ b ➠ operator.__matmul__(a, b)
6. a / b ➠ operator.__truediv__(a, b)
7. a // b ➠ operator.__floordiv__(a, b)
8. a % b ➠ operator.__mod__(a, b)
9. a ** b ➠ operator.__pow__(a, b)
10. a << b ➠ operator.__lshift__(a, b)
11. a >> b ➠ operator.__rshift__(a, b)
12. a & b ➠ operator.__and__(a, b)
13. a ^ b ➠ operator.__xor__(a, b)
14. a | b ➠ operator.__or__(a, b)
15. a += b ➠ a = operator.__iadd__(a, b)
16. a -= b ➠ a = operator.__isub__(a, b)
17. a *= b ➠ a = operator.__imul__(a, b)
18. a @= b ➠ a = operator.__imatmul__(a, b)
19. a /= b ➠ a = operator.__itruediv__(a, b)
20. a //= b ➠ a = operator.__ifloordiv__(a, b)
21. a %= b ➠ a = operator.__imod__(a, b)
22. a **= b ➠ a = operator.__ipow__(a, b)
23. a <<= b ➠ a = operator.__ilshift__(a, b)
24. a >>= b ➠ a = operator.__irshift__(a, b)
25. a &= b ➠ a = operator.__iand__(a, b)
26. a ^= b ➠ a = operator.__ixor__(a, b)
27. a |= b ➠ a = operator.__ior__(a, b)
28. ~ a ➠ operator.__invert__(a)
29. - a ➠ operator.__neg__(a)
30. + a ➠ operator.__pos__(a)
31. a == b ➠ operator.__eq__(a, b) (including object.__eq__())
32. a != b ➠ operator.__ne__(a, b) (including object.__ne__())
33. a < b ➠ operator.__lt__(a, b)
34. a <= b ➠ operator.__le__(a, b)
35. a > b ➠ operator.__gt__(a, b)
36. a >= b ➠ operator.__ge__(a, b)
37. a is b ➠ operator.is_(a, b)
38. a is not b ➠ operator.is_not(a, b)
39. not a ➠ operator.not_(a)
40. a in b ➠ operator.__contains__(b, a)
41. a not in b ➠ operator.not_(operator.__contains__(b, a))
42. a or b ➠ _temp if (_temp := a) else b
43. a and b ➠ _temp if not (_temp := a) else b
44. import a.b ➠ a = __import__('a.b', globals(), locals())
45. import a.b as c ➠ c = __import__('a', globals(), locals(), ['b'], 0).b
46. from .a import b ➠ b = __import__('a', globals(), locals(), ['b'], 1).b
47. from .a import b as c ➠ c = __import__('a', globals(), locals(), ['b'], 1).b
48. assert ... ➠ see below
49. for ... ➠ see below (including builtins.iter() and builtins.next())

assert ...

With message

assert a, b

➠

if __debug__:
    if not a:
        raise AssertionError(b)

Without a message

assert a

➠

if __debug__:
    if not a:
        raise AssertionError

for ...

Without else

for a in b:
    c

➠

_iter = iter(b)
while True:
    try:
        a = next(_iter)
    except StopIteration:
        break
    else:
        c
del _iter

With else

for a in b:
    c
else:
    d

➠

_iter = iter(b)
_looping = True
while _looping:
    try:
        a = next(_iter)
    except StopIteration:
        _looping = False
        continue
    else:
        c
else:
    d
del _iter, _looping

Syntax to (potentially) unravel

Keywords

Taken from the keyword module.

Expressions

1. yield

2. await ~

3. lambda

Statements

1. pass *

2. break

3. continue

4. if/elif/else

5. while/else

6. async for/else *

7. with *

8. async with *

9. def

10. async def ~

11. class ~

12. try/except/else/finally

13. global

14. nonlocal

15. del

16. raise/from

17. return

Tokens

Taken from the token module.

1. =

2. :=

3. [] for list display *

4. [] for list comprehensions *

5. [] for subscriptions (get, set, del), : for slicing *

6. {} for set display *

7. {} for set comprehensions *

8. {} for dictionary display *

9. {} for dictionary comprehensions *

10. () for tuple display *

11. () for calls

12. @ for decorators *

13. ,

14. ; *

15. ... *

Literals

The list below ignores literals which are represented via syntax above. For instance, lists are ignored as they are represented by [] tokens.

1. None
2. False
3. True
4. Bytes (b, r)
5. Strings (u, f, r; single line, multi-line)
6. Integers (base-10, b, o, x)
7. Floats (point, e)
8. Complex/imaginary numbers