Threshold Used For Experimental Results

[Theo Linnemann]

Hi there, I was attempting to replicate your results on the Darpa dataset, but realized you didn't specify the threshold you used. I understand the threshold is user defined, but would like to know what value was used in the experimental setup. Could you please clarify how you calculate the MIDAS(R) ROC and what threshold you used?

Thanks!

[Siddharth Bhatia]

Hi Theo, thanks for trying out MIDAS. I have added auc.py (used to get the roc auc scores), original darpa dataset and darpa dataset in MIDAS format in the repository.
We don't use a threshold to evaluate. We directly evaluate on the scores using Sklearn's roc_auc_score.

[Theo Linnemann]

Ah, I understand now. Thanks for clarifying and nice work!