is there a way to make setup of operations part of compilation process?

[Rijul Gupta]

I have been looking at various low level APIs and it appears most of them have this concept of dynamically setting up a computation graph or operation at execution time and then using that rather than compiling the entire graph ahead of time.

Are there any technical limitations that have driven this design choice, and are there any ways to circumvent it by fixing the computation graph at compile time and using the compiled functions directly?

For a minimal example I would like to implement a single fully connected layer with fixed weights that gets compiled to a fixed function in the binary.

[Marat Dukhan]

There is a technical reason. XNNPACK selects microkernels in runtime depending on the available instruction sets and core types of the host processor. As this information differs across devices, it isn't known until run-time.