IndentationError: unexpected indent
dataoverflow opened this issue · comments
Python 3.5.1 / deco from pip installed from github
Test case:
from deco import concurrent, synchronized
import time
class DecoClass:
@concurrent
def work(self):
time.sleep(0.1)
@synchronized
def run(self):
for _ in range(100):
self.work()
if __name__ == "__main__":
start = time.time()
x = DecoClass()
x.run()
print("Executing in serial should take 10 seconds")
print("Executing in parallel took:", time.time() - start, "seconds")
The error message:
File "<unknown>", line 1
@synchronized
^
IndentationError: unexpected indent
Thanks for finding this! The problem is that the @synchronization
decorator doesn't unindent the source of the function its attached to before trying to compile it into an AST. I wasn't able to find any built in functions to unindent the source lines, so the two manual ways I can think of are:
- Regex match indentation from the first line and remove it from all subsequent lines
- Call string.lstrip() on the lines up to a line matching a function definition but no further
Both seem kind of derpy, but I'll pick one and implement it some time today. If anyone has some input or alternative I'm happy to hear it.
I've committed b48e575 to address this issue and fix the bug with indentation handling.
However, I also ran across another issue with the example. Because the synchronized
class doesn't provide a __get__
method, it won't function correctly when used on class methods. To address this in the immediate future I added a __get__
method that throws an exception explaining what's happening. Perhaps in the future we can support this functionality but I'm not really sure if deco would become that much more useful for how much more complicated it would need to be.
@dataoverflow, if you had a specific use case in mind for this and could provide it I would be really interested to see how you would make use of this functionality.
@alex-sherman my use case is a class which follows interfaces from sklearn
(fit/predict etc) so it can be used along with existing sklearn
implementation. My fit()
method in my class has a loop which would benefit from concurrent execution. It'd be great if I could annotate methods of my class with deco
. Refactoring this code into a module purely so I can use deco
seem not to be a great option in my opinion.
Thanks for the reply! I don't have much familiarity with sklearn
so it's cool to see someone using deco
with it. I tried just briefly to get the decorators to work with class methods, but ran into some complications. I unfortunately don't have enough free time at the moment to work through this, but will definitely consider it at a later date. If you're interested here are the main complications:
Descriptor API and implementing get
Both concurrent and synchronized decorators need to implement __get__(self, obj, type = None)
methods in order to be used as class methods. That problem lies in being able to return basically a copy of the wrapper object that has a modified __call__
attribute which will contain the appropriate self
in it's *args
.
More complicated AST parsing
This is really the hardest problem to solve. The AST parsing has remained really simple so far because deco can expect concurrent functions to be in the global namespace. Once that assumption is gone, detecting which function calls are actually concurrent functions becomes more complicated.
TL;DR:
Working with class methods is maybe possible, but definitely hard. I hope to revisit this at a later date, but hopefully you'll still find deco
useful in the mean time.
Thanks again for mentioning this!
Alex, thank you for looking into this, even if at a later date! I found deco
very useful already. I am using it in a different part of my project and it works great.