scramjetorg / scramjet

Public tracker for Scramjet Cloud Platform, a platform that bring data from many environments together.

Home Page:https://www.scramjet.org

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Instance timeout

tomkeee opened this issue · comments

I try to count and return the number of lines in a big .csv file (532 MB) using Scramjet.
Unfortunately, after 25 minutes, the instance stops returning the output.

main.py


from scramjet.streams import Stream

result = 0 

def count(x):
    global result
    result += 1
    return result

def show_result(x):
    global result
    if result <100:
        return f"{result} \n"
    else:
        return f"{result} is bigger then 100 \n"

def run(context,input):            
    x = (Stream
        .read_from(input)
        .each(count)
        .map(show_result)
    )
    return x

I am wondering whether there is an instance's output timeout?