confusing about threshold setting

[yawenouyang]

In our evaluation, the out-of-scope threshold was chosen to be the value which yielded the highest validation score across all intents, treating out-of-scope as its own intent.

I am a little confused by this sentence. Does it mean that we select oos's highest score on the known intention as the threshold in the validation set? If so, isn't oos's recall equal to 1 in each epoch of validation set, how do we early stop and select hyper-parameters?

[gxlarson]

Thanks for your interest.
We selected the out-of-scope threshold by maximizing the accuracy on all intents (including out-of-scope) on the validation set.