Giters

plumatic / plumbing

Prismatic's Clojure(Script) utility belt

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

`eager-compile` throws an error with large graphs

frenchy64 opened this issue · comments

commented

In practice, defrecord has a field limit of around 122 (see here). Since eager-compile creates a defrecord with fields linear to the number of top-level graph nodes, it also fails with graphs larger than around 122 nodes with something like the following error:

#error {
 :cause "Method code too large!"
 :via
 [{:type clojure.lang.Compiler$CompilerException
   :message "Syntax error compiling deftype* at (/private/var/folders/2k/l1qp7ynj66g5dm8hmwxj9gtm0000gn/T/form-init4085182321200531753.clj:1:1)."
   :data #:clojure.error{:phase :compile-syntax-check, :line 1, :column 1, :source "/private/var/folders/2k/l1qp7ynj66g5dm8hmwxj9gtm0000gn/T/form-init4085182321200531753.clj", :symbol deftype*}
   :at [clojure.lang.Compiler analyzeSeq "Compiler.java" 7119]}
  {:type java.lang.IndexOutOfBoundsException
   :message "Method code too large!"
   :at [clojure.asm.MethodWriter computeMethodInfoSize "MethodWriter.java" 2061]}]
 :trace
 [[clojure.asm.MethodWriter computeMethodInfoSize "MethodWriter.java" 2061]
  [clojure.asm.ClassWriter toByteArray "ClassWriter.java" 457]
  [clojure.lang.Compiler$ObjExpr compile "Compiler.java" 4672]
  [clojure.lang.Compiler$NewInstanceExpr build "Compiler.java" 8086]
  [clojure.lang.Compiler$NewInstanceExpr$DeftypeParser parse "Compiler.java" 7939]
  [clojure.lang.Compiler analyzeSeq "Compiler.java" 7111]
  [clojure.lang.Compiler analyze "Compiler.java" 6793]
  [clojure.lang.Compiler analyze "Compiler.java" 6749]
  [clojure.lang.Compiler$BodyExpr$Parser parse "Compiler.java" 6122]
  [clojure.lang.Compiler$LetExpr$Parser parse "Compiler.java" 6440]
  [clojure.lang.Compiler analyzeSeq "Compiler.java" 7111]
  [clojure.lang.Compiler analyze "Compiler.java" 6793]
  [clojure.lang.Compiler analyze "Compiler.java" 6749]
  [clojure.lang.Compiler$BodyExpr$Parser parse "Compiler.java" 6124]
  [clojure.lang.Compiler$FnMethod parse "Compiler.java" 5471]
  [clojure.lang.Compiler$FnExpr parse "Compiler.java" 4033]
  [clojure.lang.Compiler analyzeSeq "Compiler.java" 7109]
  [clojure.lang.Compiler analyze "Compiler.java" 6793]
  [clojure.lang.Compiler eval "Compiler.java" 7178]
  [clojure.lang.Compiler eval "Compiler.java" 7136]
  [clojure.core$eval invokeStatic "core.clj" 3202]
  [clojure.core$eval invoke "core.clj" 3198]
  [plumbing.graph.positional$def_graph_record invokeStatic "positional.clj" 19]
  [plumbing.graph.positional$def_graph_record invoke "positional.clj" 12]
  [plumbing.graph.positional$def_graph_record invokeStatic "positional.clj" 15]
  [plumbing.graph.positional$def_graph_record invoke "positional.clj" 12]
  [plumbing.graph.positional$graph_form invokeStatic "positional.clj" 60]
  [plumbing.graph.positional$graph_form invoke "positional.clj" 51]
  [plumbing.graph.positional$positional_flat_compile invokeStatic "positional.clj" 72]
  [plumbing.graph.positional$positional_flat_compile invoke "positional.clj" 68]
  [plumbing.graph$eager_compile invokeStatic "graph.clj" 140]
  [plumbing.graph$eager_compile invoke "graph.clj" 129]
... ;; SNIP for confidentiality
}
commented

Thanks for the report! I'm not actively maintaining this at the moment, but happy to accept a fix.

commented

Thanks. The best fix is not obvious to me.

commented

Yeah, I'm not sure either.

commented

Idea: Add a max-positional argument that defaults to something small to def-graph-record, then attach that information via metadata to the return value. Then fix the record constructor to prepare an ext map with the overflow keys.

commented

I tried overflowing to the ext map, but now the generated code for the positional function can be too large. I found myself reimplementing the interpreted mode.

I think the specialization is intrinsically limited to certain graph sizes. I'm going to try a conditional in eager-compile that switches to interpreted mode above a certain graph size.

commented