OryxProject / oryx

Oryx 2: Lambda architecture on Apache Spark, Apache Kafka for real-time large scale machine learning

Home Page:http://oryx.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Add model validation threshold for acceptance

srowen opened this issue · comments

Right now, when a model is built, several variations of the model can be built in order to select the hyperparameters that result in the best model according to some evaluation metric. However it might also be useful to further require that the metric meet some threshold before it's published. That way, if the 'best' model were quite bad, it would be rejected.

so this is another parameter that can be set from oryx.conf or is there more to it? Like the model coming up with a reasonable threshold that evolves with more training data.