jagot / SamplingTaskFarm.jl

SamplingTaskFarm.jl

Introduction

SamplingTaskFarm.jl is a simple Julia package that aids in the computation of a function over an interval. The interval can be sampled using different strategies (currently only uniform sampling is implemented), and the samples can be computed in a serial fashion, or spread out over a "task farm", i.e. different Julia instances on the same node that communicate over TCP sockets. Finally, after every finished sample, the results so far are saved to a file, such that the calculation can be restarted if it is interrupted.

Example

We create a Julia file with the following contents:

using SamplingTaskFarm

xs = StaticSampler(Float64, 0.0..1.0, 40, "datafile.txt")

work_fun = (i,x) -> begin
    println("Worker got sample #$i: $x")
    sleep(0.5)
    sin(2π*x)
end

# Optional plotting
plot_fun = (x,y) -> begin
    p = sortperm(x)
    # plot(x[p], y[p]) using your favourite plotting package
end

task_farm(work_fun, xs, port=2000, plot_fun=plot_fun)

Running this file in a Julia instance will start a server listening on localhost:2000. Running the same file again, in as many new Julia instances as you wish on the same machine will connect to the server and ask for tasks until all samples are computed. If you instead wish to compute the samples in a serial fashion in one Julia process only, simply replace task_farm(...) by

map(work_fun, xs, plot_fun=plot_fun)