DSL for inline functions

[Michael Schulte]

My proposal for the zero DSL we shortly discussed in #22 - that is also working with the nim syntax parser:

zero:
  map(conversion)
  pre:
    var next = conversion
    if indexed: # find out indexed -> either as proc-parameter or `ext`.indexed
      next = (idx, conversion)
  loop:
    let it = next # also tricky: adding backticks when needed...

zero: follows the definition
name(parameters)(parametersToInlineProc)
sections:

• pre or prepare initialize variables and constants
• init variable initialization before the loop (ext.initials)
• loop the actual loop action (ext.node)
• end added to end of the loop (ext.endLoop) - maybe not necessary
• final after the loop section - set the result (ext.finals)
• res shortcut for setting the result
• delegate delegate to other functions (like map, filter, etc.)

The above map could (hopefully) be translated to:

proc inlineMap(ext: ExtNimNode, indexed=false) : ExtNimNode {.compileTime.} = 
  if ext.node.len != 2:
    zfFail("number of parameters for 'map' must be 1! - for tuples use extra brackets.")
  let conversion = ext.adapt(1) # this is simply assigning each parameter
  # pre: this will be tricky - when referencing input parameters: a quote is needed
  var next = quote:
    `conversion`
  if indexed:
    let idxIdent = newIdentNode(zfIndexVariableName)
    ext.needsIndex = true
    next = quote:
      (`idxIdent`, `conversion`)
  # loop:
  let itIdent = ext.nextItNode()
  ext.node = quote:
    let `itIdent`= `next`
  result = ext

While map is not yet implemented - I got already:

zero:
  filter(test)
  loop:
    if test:
      nil

or:

zero:
  dropWhile(cond)
  init:
    var gate = false
  loop:
    if gate or not cond:
      gate = true
      nil

This version: see gist

Alternative: see gist

# normal inline function
zf_inline index(cond):
  init:
    result = -1
  loop:
    if cond:
      return idx

# function yielding an iterator
zf_iterator dropWhile(cond):
  init:
    var gate = false
  loop:
    if gate or not cond:
      gate = true
      nil

any thoughts? 💡

[Alexander Ivanov]

wow, amazing. this already works for those examples, right?

[Alexander Ivanov]

it really does look almost simple.

If it's not yet fully implemented, how much work is it? I was going to suggest you to not put too much work in the lib, if maybe fixing iterator chaining in Nim is possible. Just writing an iterator as in other languages will be probably always the best option (for Nim itself, lowers barrier for new users). Still, I asked core devs, and IIRC it's not very easy to add it right now, so probably our(and loopfusion) approach is still without an alternative.

On the other hand this looks very cool and it will probably make it feasible for most users to extend it.

[Alexander Ivanov]

why can't map be

map(f)
loop:
  f(it)

btw how do you convert test to test(it) ? and is that simpler for the user, or too magical?

[Michael Schulte]

Sure - maybe we could do map like that... And indexed...-bla as an extra function - there are not so many indexed-functions anyway. Maybe I was thinking along too generic approaches - that in themselves are complicated again.

but anyway - map in that sense had to be:

zero:
  map(f)
  loop:
    let it = f 

or

zf_iterator map(f):
  loop:
    let it = f

Not sure about f(it) - so to say defining your own iterator with it - but I don't think neither me nor the user would get how to do that 😉

So - it for me is still fixed and left-hand side it of = use triggers creating a new iterator - other it accesses the previous iterator.

So - which approach do you like more:

zero:
  myfun(...)
  ...

or

zf_inline myFun(...):
 ....

or any other ideas?!

[Michael Schulte]

... so - there is some cleanup to do, when we want to use the new DSL in our code and in the examples - not sure if it is feasible to convert everything to the new DSL - especially functions like sub or combinations. But still this approach leaves it open to the user if he/she maybe first uses the DSL and then - if the function needs to be more complicated - can output the created function with repr and use and extend it instead of the DSL one.

when using the DSL however - I think we are also able to auto-generate the extension function, maybe a default function for zero-functional itself - and using zfIterator vs zfInline (or similarly named macros) - the iterator is possible as in-between argument or generates a collection as last and the other one can only be used as last argument in the chain. Such checks could be done automatically.
Also: check for the number of parameters - maybe even, as you first suggested - optional types for the parameters (which is also kind of a documentation for the user) can also be checked by the compiler.
... so lots of ideas

[Michael Schulte]

... I added a new branch for zero-dsl - with the

zf_inline functionName(params):
  ...

syntax.

functions that set or return a result have to be last element, all other may return an iterator.

[Alexander Ivanov]

sorry for the late reply, i've seen the issues, always forgetting to actually write here

[Alexander Ivanov]

so yeah this looks good, let's merge zf_inline if it seems ready

[Michael Schulte]

... still have to document.
I'm also thinking about simplifications - especially concerning the collection result type. Maybe in case of returning a collection the result could first be created in a separate iterator, then the result type of the iterator is determined and used for the overall resulting collection type.

[Michael Schulte]

The general idea would be to generate the code like that (and use also types other than seq) - when the result should be a collection:

import macros, typetraits, zero_functional

macro iteratorTypeTd(td: typedesc): untyped =
  td.getType[1][1]

macro iteratorType*(td: untyped): untyped =
  quote:
    iteratorTypeTd(`td`.type)


when isMainModule:
  let a = @[1,2,3]

  # do some transformation and return the result as iterator
  iterator autoIterator(): auto = 
    for it0 in a:
      let it1 = it0 * 2
      let it2 = $it1
      yield it2

  # use the iterator - also to determine the result type
  let a2 = (
    proc(): auto =
      var res: seq[iteratorType(autoIterator)]
      result = zfInit(res)
      var idx = 0
      for it in autoIterator():
        zfAddItem(result, idx, it)
        idx += 1)()
  echo(a2)

[Michael Schulte]

OK - let's do it!