How to pass multiple arguments ?
amitbcp opened this issue · comments
Amit Agarwal commented
I have taken the sample code from README.md and changed the function. But I am not able too pass the data :
Code :
import pypeln as pl
import time
from random import random
def slow_add1(x,y):
# time.sleep(random()) # <= some slow computation
return x + y
data = [{"x":x,"y":x*2} for x in range(10)]
stage = pl.thread.map(slow_add1, data , workers=2, maxsize=4)
list(stage)
I get the Error :
TypeError Traceback (most recent call last)
<ipython-input-28-30e5fac190e8> in <module>
----> 1 list(stage)
~/anaconda3/lib/python3.8/site-packages/pypeln/thread/stage.py in to_iterable(self, maxsize, return_index)
80
81 with supervisor:
---> 82 for elem in main_queue:
83 if return_index:
84 yield elem
~/anaconda3/lib/python3.8/site-packages/pypeln/thread/queue.py in __iter__(self)
89
90 try:
---> 91 x = self.get(timeout=pypeln_utils.TIMEOUT)
92 except Empty:
93 continue
~/anaconda3/lib/python3.8/site-packages/pypeln/thread/queue.py in get(self, block, timeout)
38 exception = Exception(f"\n\nOriginal: {exception}\n\n{trace}")
39
---> 40 raise exception
41
42 # get is implemented like this for thread but not for process
TypeError:
("slow_add1() missing 1 required positional argument: 'y'",)
Traceback (most recent call last):
File "/Users/aamita/anaconda3/lib/python3.8/site-packages/pypeln/thread/worker.py", line 91, in __call__
self.process_fn(
File "/Users/aamita/anaconda3/lib/python3.8/site-packages/pypeln/thread/worker.py", line 180, in __call__
self.apply(worker, elem, **kwargs)
File "/Users/aamita/anaconda3/lib/python3.8/site-packages/pypeln/thread/api/map.py", line 27, in apply
y = self.f(elem.value, **kwargs)
TypeError: slow_add1() missing 1 required positional argument: 'y'
I tried data as a list of supple, list of list but always the same error
Cristian Garcia commented
Hey! We don't expand tuples or dicts as arguments, in your example you have to do something like:
def slow_add1(elem):
x, y = elem["x"], elem["y"]
return x + y
Amit Agarwal commented
Thanks @cgarciae