Macroize standard library binding code to reduce boilerplate

[Adrian]

I've started making a few libraries that wrap c libraries and have been making some small improvements on the boilerplate for declaring a library interface. I think it might be helpful to wrap everything except the interface declaration (eg. obcjlib-fns) into a macro that does all the normal stuff you need to declare library bindings.

Notes:

• Added a top level form that will compile the bindings whenever *compile-files* is true so that consumers of the library don't have to do anything special as long as they are AOTing their code.
• Used if-class in initialize-objc so that the namespace can be loaded before the bindings class gets generated
• initialize-objc must be after the top level compilation so that initialize-objc will include the right code in the uberjar

Everything else basically matches the avclj example. If everything looks ok, I can create a macro that packages the idiom and submit a pull request. Thanks!

(def objclib-fns
  {:objc_getClass {:rettype :pointer
                   :argtypes [['classname :pointer]]}
   :clj_app_dir {:rettype :pointer
                 :argtypes []}
   ,})

;; Macroize from here down?
(defonce ^:private lib (dt-ffi/library-singleton #'objclib-fns))
(defn set-library-instance!
  [lib-instance]
  (dt-ffi/library-singleton-set-instance! lib lib-instance))

(dt-ffi/library-singleton-reset! lib)

(defn- find-fn
  [fn-kwd]
  (dt-ffi/library-singleton-find-fn lib fn-kwd))

(defmacro check-error
  [fn-def & body]
  `(let [error-val# (long (do ~@body))]
     (errors/when-not-errorf
      (>= error-val# 0)
      "Exception calling: (%d) - \"%s\""
      error-val# (if-let [err-name#  (get av-error/value->error-map error-val#)]
                   err-name#
                   (str-error error-val#)))
     error-val#))

(dt-ffi/define-library-functions com.phronemophobic.mobiletest.objc/objclib-fns find-fn check-error)

(defmacro if-class
  ([class-name then]
   `(if-class ~class-name
      ~then
      nil))
  ([class-name then else?]
   (let [class-exists (try
                        (Class/forName (name class-name))
                        true
                        (catch ClassNotFoundException e
                          false))]
     (if class-exists
       then
       else?))))

(defn compile-bindings [& args]
  ((requiring-resolve 'tech.v3.datatype.ffi.graalvm/define-library)
   objclib-fns
   nil
   { 
    :libraries [      ]
    :classname 'com.phronemophobic.objc.Bindings}))

(when *compile-files*
  (compile-bindings))

(defonce initialized?* (atom false))

(defn initialize-objc
  []
  (if-class com.phronemophobic.objc.Bindings
    (if (first (swap-vals!
                initialized?*
                (fn [init]
                  (when-not init
                    (set-library-instance! (com.phronemophobic.objc.Bindings.))
                    true))))
      1
      0)))

[Chris Nuernberger]

That all sounds completely reasonable to me. I would love a PR.

[Adrian]

This change turned out to be more subtle than I expected. I still need to write the doc strings, but the implementation can be found at:
branch: https://github.com/phronmophobic/dtype-next/tree/define-library-macro
code: https://github.com/phronmophobic/dtype-next/blob/define-library-macro/src/tech/v3/datatype/ffi.clj#L641

Usage:

(dt-ffi/define-library-interface
 {:objc_getClass {:rettype :pointer
                   :argtypes [['classname :pointer]]}

   :clj_app_dir {:rettype :pointer
                 :argtypes []}
   ,})

Combine interface def with function definitions
In my initial description, I thought it would be useful for the function definitions to be kept separate from the library definition, but I can't think of any compelling reasons. Alternatively, defining them inline means that it's easy to redef the library in the repl since it's just one expression.

Class compilation
Since graalvm can't instantiate classes dynamically at runtime (without extra configuration), the binding class must be instantiated directly, eg. (com.foo.mylib.Bindings.). However, if a macro emits (com.foo.mylib.Bindings.) without com.foo.mylib.Bindings existing, a syntax error is thrown with "Unable to resolve classname". Similar to defrecord and deftype, the way around that is to compile the class when the macro is called and then return the instantiation normally.

    (graal-native/if-defined-graal-native
     (if *compile-files*
       ((requiring-resolve 'tech.v3.datatype.ffi.graalvm/define-library)
        library-fns-val
        (eval symbols)
        (eval library-options))
       (try
         (Class/forName ~(name classname))
         (catch ClassNotFoundException e#
           nil)))
     (define-library
       library-fns-val
       (eval symbols)
       (eval library-options)))

There's a couple different use cases that dt-ffi/define-library-interface should support:

• using the jna and jdk ffi implementations at the repl
• graalvm native at the repl before compilation with native-image
• graalvm native at runtime when compiled

It seems like dtype-ffi/define-library does not support graalvm as an ffi implementation like jna/jdk yet, but it shouldn't be hard to refactor if that changes.

Eval usage

I know the eval looks funny here, but I think it's the right option in this particular case. Since the class must be compiled before the macro returns, we have 2 options. Either library-fns, symbols, and library-options must be compile time constants or we use eval. I think eval should be preferred so that the arguments can either be vars, constants, or programmatically constructed inline.

Library auto initialize

The macro will also automatically initialize the library the first time a library function is called.

[Chris Nuernberger]

• eval is fine here. I use assembly generation in a few places which is equivalent in some space. The rule is the input cannot come from the end user of the software and I fail to see how that could happen here.

• Good characterization of the three use cases!

• In order to use the full dtype ffi system in graal native we need to write a truffle native interface backend. Totally doable and then you would have full dynamic C support, identical in power to the jna/jdk-16 backend.

This is looking great so far.

[Adrian]

pull requested: #37

More changes:

• Added docs
• The :classname option is now evaluated consistently with the other options.
• :classname defaults to a unique classname for non-graalvm for repl redefinition, but still defaults to ns+Bindings for graalvm to make it easy create reflection configs.