Threshold Used For Experimental Results
ZeroCool2u opened this issue · comments
Hi there, I was attempting to replicate your results on the Darpa dataset, but realized you didn't specify the threshold you used. I understand the threshold is user defined, but would like to know what value was used in the experimental setup. Could you please clarify how you calculate the MIDAS(R) ROC and what threshold you used?
Thanks!
Hi Theo, thanks for trying out MIDAS. I have added auc.py (used to get the roc auc scores), original darpa dataset and darpa dataset in MIDAS format in the repository.
We don't use a threshold to evaluate. We directly evaluate on the scores using Sklearn's roc_auc_score.
Ah, I understand now. Thanks for clarifying and nice work!