Is Crystals algorithm implemented in this repo?
arrowx123 opened this issue · comments
I am trying to reproduce the results shown in Experiment 1 of the Fast and Flexible Monotonic Functions with Ensembles of Lattices.
The Crystals algorithm for feature selection looks very promising, is there any way that I can use it with this repo or implement it using the current infrastructure?
Thank you very much for the help!
Implementing crystals is a non-trivial task since tensorflow does not natively support changing the graph structure mid training, but it can be done by running one sessions with an all-pairs model (constructed using the layer API), then extracting the torsion values out of the snapshot of the trained all-pairs model, and then using that to construct the final ensemble.
I suggest instead using the embedded tiny lattices already provided in this repo. These models do not do explicit feature selection/arrangement, but rather use a linear embedding of all input features for each lattice in the ensemble. Note that this is a strict superset of the feature selection (since a linear embedding can represent a subset selection), so you might in fact get better results out of ETLs compared to crystals (at some cost to model complexity and speed).
Hi Mahdi, thank you very for the detailed information! ❤️
I think another issue in terms of implementing crystals is that, for the current infrastructure, you can not set the features used for every single lattice of the ensemble.
I will definitely play with ETL models!