cnuernber / dtype-next

A Clojure library designed to aid in the implementation of high performance algorithms and systems.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

functional api -- making the main data argument the first

daslu opened this issue · comments

Following the Zulip conversation that follows @ezmiller's work on the tablecloth column API (branch ethan/column-api-dev-branch-1), here are a few functions where we propose reordering the arguments.

The point is to make the main data argument the first so that the functions can be threading-macro-friendly.

@cnuernber said:

I think reordering arguments is the most interesting of these. We can reorder arguments any time a function doesn't take a varying number of primary arguments and we should do so such that the -> operator takes precedence. My original thought was that in general in clojure sequence arguments go last but I think this reasoning is only valid when there are potentially multiple sequences - the + operator or something like that.

Proposed changes:

  • cummax cummin cumprod cumsum: make data first.
  • kendalls-correlation pearsons-correlation spearmans-correlation: move lhs rhs before options
  • descriptive-statistics: make rdr or src-rdr first.
  • percentiles quartiles: make data first

is the -> threading ability to be consistent with some other uses. I would have expected the ->> to be more in line with the processing flows for collections (e.g. (->> something (map filter-or-other-func) ...)`

But maybe that's a row-major vs columnar bias I'm showing.

For reference, here is an old Zulip discussion of the -> vs ->> dilemma.

Once nice thing about threading macros is that ->> can be nested inside ->, so it is not so bad to handle datasets/tensors/buffers as first arguments but sequences as last arguments.

(require '[tech.v3.datatype :as dtype]
         '[tech.v3.datatype.functional :as fun])

(-> (dtype/make-reader :float32 100 (rand))
    (->> (filter #(> % 0.99)))
    dtype/->array
    (fun/* 10))

;; => [9.999570024615146 9.935718674352279]