Defining monotonicities associated with multiple outputs

[Alireza Karimi]

Hi, I came up with the Kronecker-factored lattice and wanted to give it a try. I was able to utilize this method successfully for a couple of multi-input single-output (MISO) regression problems (several features used to predict a single target) using the code provided in the TensorFlow Lattice repository. However, when I wanted to extend it to multi-input multi-output (MIMO) regression problems (several features used to predict multiple targets), I realized that it's not clear how to define the "monotonicities" argument when adopting a KFL layer with "units" > 1. Please note that when we have multiple outputs, the monotonicities would not be a simple list anymore, but a list of lists. For example, suppose that we have 5 features used to predict 3 target parameters. In this case each target variable could be monotonically or non-monotonically vary with respect to any of the features. So in this case, the monotonicities argument should be an array with a shape of (3, 5). The problem is that in the corresponding code, even when units > 1, the monotonicities value is supposed to be only a list (not a list of lists) with values of only {0, 1}. I really wonder if you could guide me on how to resolve this issue.

[mmilanifard]

Since the monotonicity requirements are different across each output, the projection operations to satisfy the constraints cannot be applied simultaneously for all output channels. This basically means that in order to satisfy the constraints, we first have to split the lattice parameters into separate tensors for each output, apply the projections separately for each piece, and stack the results back together. With this in mind, it makes more sense to just have separate layers for each output unit and setup the monotonicity constraints for each one separately.

The same argument applies to the use of premade models and canned estimators: the best practice with the current setup is to have a separate model setup/config for each output with its own monotonicity constraints. See #69 for a related discussion and an alternative regression solution if your class labels have ordinal semantics.