documentcloud / cloud-crowd

Parallel Processing for the Rest of Us

Home Page:https://github.com/documentcloud/cloud-crowd/wiki

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Timeout for jobs

lucascs opened this issue · comments

Hi,

It should be possible to specify a timeout for jobs, when adding it:

RestClient.post("#{CROWD_SERVER}/jobs", {
      :job => {
          'action' => 'word_count',
          'inputs' => ['a', 'b', 'c'],
          'timeout' => 60
      }.to_json
 });

so if an input takes more than 60 seconds to process, it is canceled...

and on merge, this input should be either ignored or have an error/default value.

Thanks

This sort of thing is highly specific to your action, or individual pieces of your action, and shouldn't be handled at the broad level of the job.

If you'd like to time out an action, just wrap the relevant code in a timeout block.

http://ruby-doc.org/stdlib/libdoc/timeout/rdoc/index.html

The Timeout::Error will propagate back up through CloudCrowd, and will cause the work unit and the job to be marked as "failed", and then call your application at the callback_url to tell it. Your application can take it from there.

Sound alright?

Sounds good to me =)

Thanks