HigherOrderCO / HVM

A massively parallel, optimal functional runtime in Rust

Home Page:https://higherorderco.com

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Questions about Lambda operator and parallel evaluation.

vladov3000 opened this issue · comments

Are the lambda operator and let syntax necessary? E.g. aren't these rewrite rules sufficient to emulate them:

(Replace (Var var) value (Var var)) = value
(Replace (Var var) value (Lambda (Var arg) body) = 
    (Lambda (Var arg) (Replace (Var var) value body) -- where var != arg for proper scoping (idk how to encode this as a rewrite rule)
(Apply (Lambda (Var x) body) y) = (Replace (Var x) y body)

Example: (Apply (Lambda (Var X) (Var X)) (Var Y)) should evaluate to (Var Y). Is there a reason these exit as primitives (convenience? efficiency?) What about dup?

(Dup x) = (Dup' x x)
-- ... extra rules for lambda/replace that propagate the dup appropriately
-- can be used in code by pattern matching against Dup' e.g. (F (Dup' x y)) = Whatever

Basically, I am asking if rewrite rules are sufficient to encode all these computations (according to the interaction net paper I've read it seems so)? So what's the point of these primitives if this is intended as a bytecode/IR and not human-writable?

Secondly, I am still curious about how HVM handles parallel computation. I could probably dig around the code more but a succinct explanation would be helpful I think.

So suppose the following rules:

(And True x)  = x
(And False x) = False

And suppose I want to evaluate an expression in the form (And True (And False (And ...))). For each And, we can use the rewrite rules above to rewrite each nesting in parallel (assuming infinite cores). Is this done in practice and how? Like what struct in C do each of these evaluate to?

No, rewrite rules aren't sufficient to replicate lambdas. Your implementation of lambda doesn't work as you expect, as it isn't able to substitute inside arbitrary terms. For example, (Apply (Lambda (Var 0) (Pair (Var 0) 5)) 42) would get stuck on (Replace (Var 0) 42 (Pair (Var 0) 5)). The same for dup, you can't emulate it with rewrite rules alone. Truth is, rewrite rules are not very powerful, but they're fast, and that's why we have them. The real fundamentals of HVM are lams and dups; with these, you can do anything. So, the other way around is true: we could remove rewrite rules and have just lams/dups.

Secondly, I am still curious about how HVM handles parallel computation. I could probably dig around the code more but a succinct explanation would be helpful I think.

In simple terms, each built-in operation on HVM is atomic (lambda application, duplications, rewrite rules, substitution, etc.). As such, all it does is keep a work queue that is shared between threads, allowing the work to be distributed through all cores. That is actually quite easy once you're in the Interaction Net model. So, basically, whenever you have work that could be done in parallel, like (+ (+ 1 2) (+ 3 4)), HVM will do it in parallel. It will only not parallelize when the work is inherently sequential, like (+ 1 (+ 2 (+ 3 4))).

Is this done in practice and how? Like what struct in C do each of these evaluate to?

No, in your example you'd not have any parallelism, since each And depends on the inner And. You'd want to make a more tree-like expression, like (And (And ... ...) (And ... ...)) to let it parallelize. Not sure what you mean by "what struct in C", but each node is represented by a slice of memory with 64 * node_arity length. So, for example, And True False occupies 128 bits in memory. The exact memory layout is pretty simple, and is explained on the following link:

https://github.com/Kindelia/HVM/blob/master/src/runtime/base/memory.rs