Classification blocked with multiple 1-dimensional features
mattvan83 opened this issue · comments
Hello,
I have problems when using multiple 1-D features with the following command:
neuropredict -m meta_data.csv \
-d mask_WMpet_av45_early.ero1.25mm.csv mask_WMpet_av45_early.ero1.5mm.csv \
-o outdir -t 0.8 -n 250 -k 'all' \
--gs_level 'exhaustive' --classifier "LinearSVC"
The CV trial were well launched for the first subgroup CN,MCI for both 1-D features but it seemed that at the end of CV trials the process got stuck at this stage:
>
> Python 3.7.3
> > SGE recognized, job set up with 40 slots.
> > Positive class inferred for AUC calculation: CN
> > Running neuropredict 0.5+34.g220af55.dirty
> >
> > Requested features for analysis:
> > get_data_matrix from /netapp/vol2_agewell/pro/IMAP/imap_mvh/CAT12/pet/Analyses/ML/All/CN_vs_MCI+AD/av45_early/pons/start0_dur4/mask_WMpet_av45_early.ero1.25mm.csv
> > get_data_matrix from /netapp/vol2_agewell/pro/IMAP/imap_mvh/CAT12/pet/Analyses/ML/All/CN_vs_MCI+AD/av45_early/pons/start0_dur4/mask_WMpet_av45_early.ero1.5mm.csv
> > Ignoring imputation strategy chosen, as no missing data were found!
> >
> > Data import is done.
> >
> >
> > Requested processing for the following subgroups:
> > DIS,CN
> >
> > --------------------------------------------------------------------------------
> > Processing subgroup : DIS,CN (1/1)
> > --------------------------------------------------------------------------------
> > SGE recognized, job set up with 40 slots.
> > Training percentage : 0.8
> > Number of CV repetitions : 250
> > Classifier chosen : linearsvc
> > Feature selection chosen : variancethreshold
> > Level of grid search : exhaustive
> > Number of processors : 40
> > Saving the results to
> > /netapp/vol2_agewell/pro/IMAP/imap_mvh/CAT12/pet/Analyses/ML/All/CN_vs_MCI+AD/av45_early/pons/start0_dur4/linearsvc/binary_WMmasks/CN_DIS
> >
> > -------------------------
> > All datasets contain:
> >
> > 101 samples, 2 classes, 1 features
> > Class CN : 71 samples
> > Class DIS : 30 samples
> > -------------------------
> >
> > Estimated chance accuracy : 0.500
> >
> > Different classes in the training set are stratified to match the smallest class!
> > Parallelizing the repetitions of CV with 40 processes ...
> > CV trial 52 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6525 weighted AUC: 0.7482
> > CV trial 78 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6631 weighted AUC: 0.8227
> > CV trial 50 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.5993 weighted AUC: 0.6596
> > CV trial 72 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6082 weighted AUC: 0.6596
> > CV trial 46 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7252 weighted AUC: 0.7021
> > CV trial 58 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8085 weighted AUC: 0.8475
> > CV trial 38 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6933 weighted AUC: 0.7518
> > CV trial 16 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7358 weighted AUC: 0.7801
> > CV trial 12 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7660 weighted AUC: 0.8794
> > CV trial 0 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6738 weighted AUC: 0.6950
> > CV trial 68 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7057 weighted AUC: 0.8369
> > CV trial 14 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 weighted AUC: 0.8794
> > CV trial 22 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7677 weighted AUC: 0.8333
> > CV trial 6 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6631 weighted AUC: 0.7234
> > CV trial 48 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7571 weighted AUC: 0.8298
> > CV trial 10 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6720 weighted AUC: 0.7624
> > CV trial 64 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8085 weighted AUC: 0.8191
> > CV trial 20 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7677 weighted AUC: 0.8688
> > CV trial 4 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8191 weighted AUC: 0.8688
> > CV trial 56 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7553 weighted AUC: 0.8262
> > CV trial 70 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8191 weighted AUC: 0.8440
> > CV trial 74 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 weighted AUC: 0.7553
> > CV trial 36 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 weighted AUC: 0.8050
> > CV trial 2 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 weighted AUC: 0.9043
> > CV trial 54 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8191 weighted AUC: 0.8830
> > CV trial 24 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7447 weighted AUC: 0.9043
> > CV trial 76 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7340 weighted AUC: 0.9220
> > CV trial 42 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7553 weighted AUC: 0.8759
> > CV trial 26 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 weighted AUC: 0.9078
> > CV trial 62 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7571 weighted AUC: 0.8103
> > CV trial 66 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6418 weighted AUC: 0.7624
> > CV trial 60 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8511 weighted AUC: 0.8972
> > CV trial 72 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6294 weighted AUC: 0.6418
> > CV trial 40 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7660 weighted AUC: 0.9149
> > CV trial 52 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6720 weighted AUC: 0.7340
> > CV trial 16 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 weighted AUC: 0.7411
> > CV trial 46 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6720 weighted AUC: 0.6489
> > CV trial 38 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6507 weighted AUC: 0.7376
> > CV trial 78 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7358 weighted AUC: 0.8014
> > CV trial 50 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6099 weighted AUC: 0.6454
> > CV trial 58 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8191 weighted AUC: 0.8298
> > CV trial 6 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7145 weighted AUC: 0.6879
> > CV trial 14 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 weighted AUC: 0.8688
> > CV trial 36 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6844 weighted AUC: 0.8156
> > CV trial 18 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 weighted AUC: 0.8014
> > CV trial 0 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.5904 weighted AUC: 0.6454
> > CV trial 20 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7571 weighted AUC: 0.8582
> > CV trial 68 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7677 weighted AUC: 0.7979
> > CV trial 4 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8511 weighted AUC: 0.8440
> > CV trial 74 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7660 weighted AUC: 0.7358
> > CV trial 56 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7447 weighted AUC: 0.8050
> > CV trial 34 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.5816 weighted AUC: 0.7518
> > CV trial 12 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7447 weighted AUC: 0.8723
> > CV trial 66 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 weighted AUC: 0.7376
> > CV trial 70 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6844 weighted AUC: 0.8227
> > CV trial 64 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7358 weighted AUC: 0.8121
> > CV trial 24 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 weighted AUC: 0.8794
> > CV trial 76 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7234 weighted AUC: 0.9007
> > CV trial 79 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.5904 weighted AUC: 0.7021
> > CV trial 53 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6720 weighted AUC: 0.7766
> > CV trial 2 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 weighted AUC: 0.8936
> > CV trial 73 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 weighted AUC: 0.9255
> > CV trial 54 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7571 weighted AUC: 0.8759
> > CV trial 62 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7571 weighted AUC: 0.8014
> > CV trial 17 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7358 weighted AUC: 0.7624
> > CV trial 22 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7677 weighted AUC: 0.8191
> > CV trial 39 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6631 weighted AUC: 0.7695
> > CV trial 60 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8191 weighted AUC: 0.9007
> > CV trial 51 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8404 weighted AUC: 0.8227
> > CV trial 26 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 weighted AUC: 0.9007
> > CV trial 42 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 weighted AUC: 0.8546
> > CV trial 48 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7571 weighted AUC: 0.8121
> > CV trial 59 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 weighted AUC: 0.8014
> > CV trial 69 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 weighted AUC: 0.7695
> > CV trial 47 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7039 weighted AUC: 0.7695
> > CV trial 21 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7553 weighted AUC: 0.9220
> > CV trial 28 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8316 weighted AUC: 0.9007
> > CV trial 37 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7660 weighted AUC: 0.8475
> > CV trial 57 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7660 weighted AUC: 0.8582
> > CV trial 1 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 weighted AUC: 0.8440
> > CV trial 40 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7447 weighted AUC: 0.9149
> > CV trial 30 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8085 weighted AUC: 0.8582
> > CV trial 18 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7358 weighted AUC: 0.7589
> > CV trial 65 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 weighted AUC: 0.7837
> > CV trial 75 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 weighted AUC: 0.8617
> > CV trial 25 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6720 weighted AUC: 0.7943
> > CV trial 13 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 weighted AUC: 0.8121
> > CV trial 79 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6525 weighted AUC: 0.6915
> > CV trial 61 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7553 weighted AUC: 0.7766
> > CV trial 71 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 weighted AUC: 0.8156
> > CV trial 32 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8316 weighted AUC: 0.9291
> > CV trial 3 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 weighted AUC: 0.8440
> > CV trial 53 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7039 weighted AUC: 0.7411
> > CV trial 55 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 weighted AUC: 0.8298
> > CV trial 15 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6933 weighted AUC: 0.8582
> > CV trial 23 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 weighted AUC: 0.7872
> > CV trial 34 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.5284 weighted AUC: 0.7376
> > CV trial 59 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7447 weighted AUC: 0.7713
> > CV trial 43 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 weighted AUC: 0.7766
> > CV trial 77 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7252 weighted AUC: 0.7305
> > CV trial 49 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7039 weighted AUC: 0.8528
> > CV trial 27 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 weighted AUC: 0.8191
> > CV trial 73 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7340 weighted AUC: 0.8936
> > CV trial 39 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.5798 weighted AUC: 0.7305
> > CV trial 47 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7145 weighted AUC: 0.7234
> > CV trial 69 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7145 weighted AUC: 0.7199
> > CV trial 21 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7021 weighted AUC: 0.9149
> > CV trial 44 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7677 weighted AUC: 0.8369
> > CV trial 65 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7447 weighted AUC: 0.7943
> > CV trial 41 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6826 weighted AUC: 0.7092
> > CV trial 80 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6011 weighted AUC: 0.7305
> > CV trial 51 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6738 weighted AUC: 0.8121
> > CV trial 57 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7128 weighted AUC: 0.8369
> > CV trial 75 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 weighted AUC: 0.8156
> > CV trial 1 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 weighted AUC: 0.8191
> > CV trial 63 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7340 weighted AUC: 0.8582
> > CV trial 82 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 weighted AUC: 0.8475
> > CV trial 3 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 weighted AUC: 0.8050
> > CV trial 23 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7039 weighted AUC: 0.7270
> > CV trial 61 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6933 weighted AUC: 0.7553
> > CV trial 55 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7340 weighted AUC: 0.8014
> > CV trial 43 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6418 weighted AUC: 0.7270
> > CV trial 77 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7145 weighted AUC: 0.7234
> > CV trial 25 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6613 weighted AUC: 0.7872
> > CV trial 84 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 weighted AUC: 0.9362
> > CV trial 30 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7358 weighted AUC: 0.8333
> > CV trial 19 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8298 weighted AUC: 0.8298
> > CV trial 37 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 weighted AUC: 0.8298
> > CV trial 88 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 weighted AUC: 0.8121
> > CV trial 96 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6507 weighted AUC: 0.7270
> > CV trial 86 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 weighted AUC: 0.8759
> > CV trial 13 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 weighted AUC: 0.7837
> > CV trial 41 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6206 weighted AUC: 0.6809
> > CV trial 5 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8617 weighted AUC: 0.9043
> > CV trial 102 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6933 weighted AUC: 0.8333
> > CV trial 80 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6011 weighted AUC: 0.7163
> > CV trial 92 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 weighted AUC: 0.9220
> > CV trial 15 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 weighted AUC: 0.8227
> > CV trial 104 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6436 weighted AUC: 0.7660
> > CV trial 35 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 weighted AUC: 0.8333
> > CV trial 106 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.5585 weighted AUC: 0.7163
> > CV trial 108 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6418 weighted AUC: 0.7163
> > CV trial 110 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 weighted AUC: 0.6844
> > CV trial 94 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8617 weighted AUC: 0.9255
> > CV trial 98 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7447 weighted AUC: 0.8652
> > CV trial 82 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 weighted AUC: 0.8546
> > CV trial 100 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7890 weighted AUC: 0.8652
> > CV trial 49 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 weighted AUC: 0.8440
> > CV trial 63 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7234 weighted AUC: 0.8475
> > CV trial 112 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 weighted AUC: 0.8475
> > CV trial 90 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8511 weighted AUC: 0.8511
> > CV trial 118 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6525 weighted AUC: 0.7411
> > CV trial 27 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 weighted AUC: 0.7943
> > CV trial 44 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7571 weighted AUC: 0.8014
> > CV trial 124 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 weighted AUC: 0.8085
> > CV trial 84 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7553 weighted AUC: 0.9255
> > CV trial 110 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.4645 weighted AUC: 0.6454
> > CV trial 31 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 weighted AUC: 0.9149
> > CV trial 96 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6401 weighted AUC: 0.7021
> > CV trial 108 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7039 weighted AUC: 0.6738
> > CV trial 126 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 weighted AUC: 0.8582
> > CV trial 81 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6631 weighted AUC: 0.7589
> > CV trial 116 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8085 weighted AUC: 0.9184
> > CV trial 104 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7145 weighted AUC: 0.7163
> > CV trial 106 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.5585 weighted AUC: 0.7163
> > CV trial 67 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7996 weighted AUC: 0.9007
> > CV trial 92 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 weighted AUC: 0.9043
> > CV trial 88 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7145 weighted AUC: 0.7695
> > CV trial 120 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7553 weighted AUC: 0.7695
> > CV trial 122 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7358 weighted AUC: 0.8511
> > CV trial 130 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6933 weighted AUC: 0.7376
> > CV trial 19 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8298 weighted AUC: 0.8085
> > CV trial 102 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 weighted AUC: 0.8262
> > CV trial 98 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 weighted AUC: 0.8688
> > CV trial 112 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7234 weighted AUC: 0.8298
> > CV trial 114 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7571 weighted AUC: 0.8652
> > CV trial 86 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 weighted AUC: 0.8475
> > CV trial 35 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 weighted AUC: 0.8121
> > CV trial 100 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7057 weighted AUC: 0.8475
> > CV trial 71 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 weighted AUC: 0.7996
> > CV trial 118 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6525 weighted AUC: 0.6986
> > CV trial 94 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7784 weighted AUC: 0.8901
> > CV trial 83 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6738 weighted AUC: 0.8475
> > CV trial 128 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 weighted AUC: 0.8369
> > CV trial 85 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6826 weighted AUC: 0.7695
> > CV trial 17 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6631 weighted AUC: 0.7447
> > CV trial 90 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8404 weighted AUC: 0.8511
> > CV trial 45 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8617 weighted AUC: 0.8191
> > CV trial 111 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6188 weighted AUC: 0.8333
> > CV trial 124 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6933 weighted AUC: 0.7713
> > CV trial 97 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7571 weighted AUC: 0.8582
> > CV trial 109 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 weighted AUC: 0.8652
> > CV trial 105 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7039 weighted AUC: 0.8369
> > CV trial 120 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7660 weighted AUC: 0.7340
> > CV trial 113 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7447 weighted AUC: 0.6702
> > CV trial 87 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7660 weighted AUC: 0.9362
> > CV trial 81 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6011 weighted AUC: 0.7305
> > CV trial 136 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.5904 weighted AUC: 0.8050
> > CV trial 132 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6738 weighted AUC: 0.7589
> > CV trial 116 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 weighted AUC: 0.8901
> > CV trial 99 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8298 weighted AUC: 0.8972
> > CV trial 119 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7021 weighted AUC: 0.8759
> > CV trial 103 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8191 weighted AUC: 0.9113
> > CV trial 95 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 weighted AUC: 0.6879
> > CV trial 134 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6720 weighted AUC: 0.8014
> > CV trial 130 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7660 weighted AUC: 0.6986
> > CV trial 93 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7677 weighted AUC: 0.8121
> > CV trial 125 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6933 weighted AUC: 0.7482
> > CV trial 101 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7784 weighted AUC: 0.8652
> > CV trial 91 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7021 weighted AUC: 0.9255
> > CV trial 85 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7039 weighted AUC: 0.7482
> > CV trial 128 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 weighted AUC: 0.8262
> > CV trial 107 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6826 weighted AUC: 0.7801
> > CV trial 111 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6188 weighted AUC: 0.7979
> > CV trial 5 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7784 weighted AUC: 0.8865
> > CV trial 31 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 weighted AUC: 0.9220
> > CV trial 138 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7784 weighted AUC: 0.8901
> > CV trial 114 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7784 weighted AUC: 0.8333
> > CV trial 121 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 weighted AUC: 0.8546
> > CV trial 113 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6933 weighted AUC: 0.6312
> > CV trial 97 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 weighted AUC: 0.8546
> > CV trial 28 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8528 weighted AUC: 0.8617
> > CV trial 89 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6418 weighted AUC: 0.6702
> > CV trial 109 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 weighted AUC: 0.8688
> > CV trial 122 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7660 weighted AUC: 0.8422
> > CV trial 87 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 weighted AUC: 0.9149
> > CV trial 105 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6933 weighted AUC: 0.8191
> > CV trial 83 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7571 weighted AUC: 0.8475
> > CV trial 95 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6826 weighted AUC: 0.6879
> > CV trial 140 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 weighted AUC: 0.8546
> > CV trial 7 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8191 weighted AUC: 0.8830
> > CV trial 125 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7039 weighted AUC: 0.6844
> > CV trial 142 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8085 weighted AUC: 0.8723
> > CV trial 136 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7358 weighted AUC: 0.7908
> > CV trial 103 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 weighted AUC: 0.8759
> > CV trial 119 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7021 weighted AUC: 0.8262
> > CV trial 99 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8085 weighted AUC: 0.8759
> > CV trial 132 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6436 weighted AUC: 0.7234
> > CV trial 129 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 weighted AUC: 0.7021
> > CV trial 91 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6915 weighted AUC: 0.9184
> > CV trial 126 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 weighted AUC: 0.8493
> > CV trial 150 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7660 weighted AUC: 0.7376
> > CV trial 93 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6011 weighted AUC: 0.8014
> > CV trial 144 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 weighted AUC: 0.7872
> > CV trial 148 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 weighted AUC: 0.7323
> > CV trial 146 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 weighted AUC: 0.7624
> > CV trial 107 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6294 weighted AUC: 0.7411
> > CV trial 152 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7447 weighted AUC: 0.8723
> > CV trial 101 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6950 weighted AUC: 0.8617
> > CV trial 115 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7234 weighted AUC: 0.9007
> > CV trial 154 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 weighted AUC: 0.8759
> > CV trial 134 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6401 weighted AUC: 0.8014
> > CV trial 29 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 weighted AUC: 0.8582
> > CV trial 158 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7128 weighted AUC: 0.8121
> > CV trial 166 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7340 weighted AUC: 0.7872
> > CV trial 137 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 weighted AUC: 0.7234
> > CV trial 168 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7252 weighted AUC: 0.7057
> > CV trial 160 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8191 weighted AUC: 0.8440
> > CV trial 123 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 weighted AUC: 0.8121
> > CV trial 117 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 weighted AUC: 0.8652
> > CV trial 164 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7677 weighted AUC: 0.8369
> > CV trial 45 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6950 weighted AUC: 0.7979
> > CV trial 170 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6011 weighted AUC: 0.7766
> > CV trial 150 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7447 weighted AUC: 0.7057
> > CV trial 7 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7447 weighted AUC: 0.8723
> > CV trial 121 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 weighted AUC: 0.8050
> > CV trial 174 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7784 weighted AUC: 0.8298
> > CV trial 176 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7677 weighted AUC: 0.8369
> > CV trial 162 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6826 weighted AUC: 0.7660
> > CV trial 156 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 weighted AUC: 0.7908
> > CV trial 127 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6933 weighted AUC: 0.8014
> > CV trial 8 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8723 weighted AUC: 0.9326
> > CV trial 172 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6117 weighted AUC: 0.7589
> > CV trial 144 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7553 weighted AUC: 0.7589
> > CV trial 148 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.5691 weighted AUC: 0.7021
> > CV trial 129 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.4965 weighted AUC: 0.6809
> > CV trial 180 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 weighted AUC: 0.8121
> > CV trial 140 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 weighted AUC: 0.8191
> > CV trial 146 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7660 weighted AUC: 0.7624
> > CV trial 142 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8191 weighted AUC: 0.8652
> > CV trial 154 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 weighted AUC: 0.8475
> > CV trial 115 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6915 weighted AUC: 0.8865
> > CV trial 29 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7340 weighted AUC: 0.8227
> > CV trial 166 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7021 weighted AUC: 0.7553
> > CV trial 89 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6418 weighted AUC: 0.6348
> > CV trial 133 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 weighted AUC: 0.8298
> > CV trial 168 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 weighted AUC: 0.6702
> > CV trial 184 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.5780 weighted AUC: 0.7589
> > CV trial 137 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6418 weighted AUC: 0.6950
> > CV trial 158 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7447 weighted AUC: 0.7837
> > CV trial 152 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7234 weighted AUC: 0.8262
> > CV trial 151 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.5479 weighted AUC: 0.7305
> > CV trial 131 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8191 weighted AUC: 0.9255
> > CV trial 160 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8085 weighted AUC: 0.8227
> > CV trial 123 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7358 weighted AUC: 0.7766
> > CV trial 32 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 weighted AUC: 0.0709
> > CV trial 135 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 weighted AUC: 0.7713
> > CV trial 170 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6011 weighted AUC: 0.7270
> > CV trial 188 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 weighted AUC: 0.6454
> > CV trial 145 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7234 weighted AUC: 0.7553
> > CV trial 117 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7677 weighted AUC: 0.8369
> > CV trial 156 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 weighted AUC: 0.7837
> > CV trial 127 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7039 weighted AUC: 0.7890
> > CV trial 186 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6826 weighted AUC: 0.7979
> > CV trial 178 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 weighted AUC: 0.7624
> > CV trial 141 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7340 weighted AUC: 0.8511
> > CV trial 147 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7039 weighted AUC: 0.8759
> > CV trial 180 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6418 weighted AUC: 0.7979
> > CV trial 174 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7784 weighted AUC: 0.8085
> > CV trial 149 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8085 weighted AUC: 0.8582
> > CV trial 162 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6507 weighted AUC: 0.7163
> > CV trial 172 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.5904 weighted AUC: 0.7270
> > CV trial 184 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.5674 weighted AUC: 0.7340
> > CV trial 151 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6826 weighted AUC: 0.7163
> > CV trial 176 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7677 weighted AUC: 0.7872
> > CV trial 194 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6525 weighted AUC: 0.7092
> > CV trial 188 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.4645 weighted AUC: 0.5993
> > CV trial 169 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7553 weighted AUC: 0.8688
> > CV trial 155 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7571 weighted AUC: 0.7872
> > CV trial 171 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6826 weighted AUC: 0.7766
> > CV trial 159 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6223 weighted AUC: 0.7482
> > CV trial 167 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 weighted AUC: 0.8511
> > CV trial 153 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 weighted AUC: 0.8652
> > CV trial 133 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7465 weighted AUC: 0.7766
> > CV trial 143 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7660 weighted AUC: 0.8617
> > CV trial 198 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 weighted AUC: 0.8333
> > CV trial 161 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 weighted AUC: 0.8475
> > CV trial 202 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 weighted AUC: 0.6950
> > CV trial 182 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 weighted AUC: 0.8333
> > CV trial 164 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6844 weighted AUC: 0.8191
> > CV trial 186 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6401 weighted AUC: 0.7411
> > CV trial 8 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8830 weighted AUC: 0.0674
> > CV trial 157 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7447 weighted AUC: 0.9574
> > CV trial 141 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7021 weighted AUC: 0.8617
> > CV trial 135 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6631 weighted AUC: 0.7695
> > CV trial 145 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7340 weighted AUC: 0.7199
> > CV trial 190 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7039 weighted AUC: 0.7553
> > CV trial 181 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 weighted AUC: 0.7979
> > CV trial 196 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 weighted AUC: 0.7766
> > CV trial 33 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8103 weighted AUC: 0.8936
> > CV trial 192 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8422 weighted AUC: 0.8546
> > CV trial 138 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7784 weighted AUC: 0.8901
> > CV trial 200 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8085 weighted AUC: 0.9468
> > CV trial 149 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 weighted AUC: 0.8546
> > CV trial 173 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 weighted AUC: 0.8475
> > CV trial 177 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 weighted AUC: 0.8014
> > CV trial 178 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7465 weighted AUC: 0.7340
> > CV trial 204 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 weighted AUC: 0.8404
> > CV trial 206 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 weighted AUC: 0.7872
> > CV trial 163 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 weighted AUC: 0.8085
> > CV trial 185 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 weighted AUC: 0.8298
> > CV trial 171 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7039 weighted AUC: 0.7801
> > CV trial 202 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6206 weighted AUC: 0.6950
> > CV trial 169 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7447 weighted AUC: 0.8440
> > CV trial 167 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 weighted AUC: 0.8369
> > CV trial 153 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7447 weighted AUC: 0.8369
> > CV trial 131 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8298 weighted AUC: 0.9326
> > CV trial 159 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6950 weighted AUC: 0.7340
> > CV trial 194 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6418 weighted AUC: 0.6879
> > CV trial 187 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7571 weighted AUC: 0.7553
> > CV trial 155 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6738 weighted AUC: 0.7660
> > CV trial 212 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 weighted AUC: 0.8546
> > CV trial 198 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6826 weighted AUC: 0.8191
> > CV trial 189 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 weighted AUC: 0.8794
> > CV trial 165 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 weighted AUC: 0.8652
> > CV trial 157 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7340 weighted AUC: 0.9397
> > CV trial 208 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7979 weighted AUC: 0.8546
> > CV trial 139 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6720 weighted AUC: 0.7855
> > CV trial 190 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7039 weighted AUC: 0.7305
> > CV trial 214 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.5780 weighted AUC: 0.6879
> > CV trial 181 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7128 weighted AUC: 0.7660
> > CV trial 210 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 weighted AUC: 0.8121
> > CV trial 143 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7553 weighted AUC: 0.8404
> > CV trial 9 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7039 weighted AUC: 0.8511
> > CV trial 196 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6738 weighted AUC: 0.7518
> > CV trial 182 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6933 weighted AUC: 0.8227
> > CV trial 147 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7039 weighted AUC: 0.8582
> > CV trial 175 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8298 weighted AUC: 0.9113
> > CV trial 216 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7252 weighted AUC: 0.7943
> > CV trial 173 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 weighted AUC: 0.8440
> > CV trial 177 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 weighted AUC: 0.7979
> > CV trial 206 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8085 weighted AUC: 0.7518
> > CV trial 179 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7571 weighted AUC: 0.8121
> > CV trial 204 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8085 weighted AUC: 0.8440
> > CV trial 218 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8298 weighted AUC: 0.8759
> > CV trial 185 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7660 weighted AUC: 0.8014
> > CV trial 226 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6312 weighted AUC: 0.8227
> > CV trial 224 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 weighted AUC: 0.8475
> > CV trial 220 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 weighted AUC: 0.7801
> > CV trial 195 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 weighted AUC: 0.7872
> > CV trial 200 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8511 weighted AUC: 0.9291
> > CV trial 163 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7465 weighted AUC: 0.7908
> > CV trial 161 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 weighted AUC: 0.8156
> > CV trial 230 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6401 weighted AUC: 0.8227
> > CV trial 199 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7447 weighted AUC: 0.8050
> > CV trial 212 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6613 weighted AUC: 0.8262
> > CV trial 187 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7145 weighted AUC: 0.7447
> > CV trial 232 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6844 weighted AUC: 0.7837
> > CV trial 207 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.4752 weighted AUC: 0.6064
> > CV trial 33 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 weighted AUC: 0.0975
> > CV trial 228 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 weighted AUC: 0.8440
> > CV trial 203 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7358 weighted AUC: 0.7766
> > CV trial 139 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6525 weighted AUC: 0.7624
> > CV trial 183 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7358 weighted AUC: 0.8262
> > CV trial 208 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 weighted AUC: 0.8404
> > CV trial 165 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 weighted AUC: 0.8191
> > CV trial 234 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8298 weighted AUC: 0.9468
> > CV trial 214 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6206 weighted AUC: 0.6631
> > CV trial 216 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7039 weighted AUC: 0.7589
> > CV trial 191 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8298 weighted AUC: 0.8511
> > CV trial 240 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7660 weighted AUC: 0.8440
> > CV trial 236 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 weighted AUC: 0.8262
> > CV trial 210 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7571 weighted AUC: 0.7943
> > CV trial 242 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8298 weighted AUC: 0.8582
> > CV trial 224 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7553 weighted AUC: 0.7943
> > CV trial 189 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7979 weighted AUC: 0.8387
> > CV trial 220 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 weighted AUC: 0.7163
> > CV trial 213 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6950 weighted AUC: 0.8156
> > CV trial 238 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6844 weighted AUC: 0.8085
> > CV trial 218 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 weighted AUC: 0.8582
> > CV trial 230 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6401 weighted AUC: 0.7730
> > CV trial 226 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6312 weighted AUC: 0.8014
> > CV trial 179 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 weighted AUC: 0.8014
> > CV trial 201 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6507 weighted AUC: 0.8298
> > CV trial 207 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.4645 weighted AUC: 0.5816
> > CV trial 246 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6720 weighted AUC: 0.8121
> > CV trial 248 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6117 weighted AUC: 0.7163
> > CV trial 195 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 weighted AUC: 0.7518
> > CV trial 199 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 weighted AUC: 0.7837
> > CV trial 183 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6525 weighted AUC: 0.8298
> > CV trial 244 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7358 weighted AUC: 0.8298
> > CV trial 203 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7571 weighted AUC: 0.7270
> > CV trial 209 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7340 weighted AUC: 0.8511
> > CV trial 205 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.5691 weighted AUC: 0.7801
> > CV trial 217 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8085 weighted AUC: 0.9149
> > CV trial 240 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 weighted AUC: 0.8191
> > CV trial 221 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7660 weighted AUC: 0.9291
> > CV trial 225 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7358 weighted AUC: 0.8262
> > CV trial 232 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6436 weighted AUC: 0.7660
> > CV trial 211 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7252 weighted AUC: 0.8262
> > CV trial 234 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8298 weighted AUC: 0.9326
> > CV trial 236 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7145 weighted AUC: 0.7943
> > CV trial 246 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6720 weighted AUC: 0.7730
> > CV trial 213 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8404 weighted AUC: 0.8050
> > CV trial 209 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7447 weighted AUC: 0.8316
> > CV trial 219 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6631 weighted AUC: 0.7305
> > CV trial 248 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.5071 weighted AUC: 0.7092
> > CV trial 201 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6613 weighted AUC: 0.7979
> > CV trial 67 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7163 weighted AUC: 0.8617
> > CV trial 228 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6933 weighted AUC: 0.8121
> > CV trial 9 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6933 weighted AUC: 0.8262
> > CV trial 217 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8085 weighted AUC: 0.8830
> > CV trial 215 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7766 weighted AUC: 0.9255
> > CV trial 244 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8085 weighted AUC: 0.8121
> > CV trial 175 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8298 weighted AUC: 0.8688
> > CV trial 231 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7145 weighted AUC: 0.7979
> > CV trial 227 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8085 weighted AUC: 0.8369
> > CV trial 241 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7465 weighted AUC: 0.8652
> > CV trial 211 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7571 weighted AUC: 0.8014
> > CV trial 205 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7039 weighted AUC: 0.7660
> > CV trial 235 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6933 weighted AUC: 0.8156
> > CV trial 191 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8298 weighted AUC: 0.8475
> > CV trial 249 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6826 weighted AUC: 0.6773
> > CV trial 233 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6206 weighted AUC: 0.7234
> > CV trial 225 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7358 weighted AUC: 0.7908
> > CV trial 247 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7021 weighted AUC: 0.8865
> > CV trial 237 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7872 weighted AUC: 0.8652
> > CV trial 242 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8298 weighted AUC: 0.8333
> > CV trial 229 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6720 weighted AUC: 0.7872
> > CV trial 219 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.5904 weighted AUC: 0.7199
> > CV trial 221 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7660 weighted AUC: 0.8901
> > CV trial 238 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6011 weighted AUC: 0.7943
> > CV trial 245 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8298 weighted AUC: 0.8528
> > CV trial 222 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8191 weighted AUC: 0.8794
> > CV trial 215 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7660 weighted AUC: 0.9078
> > CV trial 249 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6933 weighted AUC: 0.6312
> > CV trial 231 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7145 weighted AUC: 0.7447
> > CV trial 235 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6720 weighted AUC: 0.7943
> > CV trial 241 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8191 weighted AUC: 0.8582
> > CV trial 247 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6915 weighted AUC: 0.8511
> > CV trial 229 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6933 weighted AUC: 0.7624
> > CV trial 233 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6206 weighted AUC: 0.6915
> > CV trial 237 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7872 weighted AUC: 0.8546
> > CV trial 227 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7660 weighted AUC: 0.8262
> > CV trial 239 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7252 weighted AUC: 0.8901
> > CV trial 245 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8404 weighted AUC: 0.8121
> > CV trial 243 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7996 weighted AUC: 0.8688
> > CV trial 222 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8085 weighted AUC: 0.8723
> > CV trial 239 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7252 weighted AUC: 0.8901
> > CV trial 192 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8316 weighted AUC: 0.8298
> > CV trial 197 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8191 weighted AUC: 0.9539
> > CV trial 197 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7766 weighted AUC: 0.9433
> > CV trial 223 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.5709 weighted AUC: 0.7908
> > CV trial 243 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7890 weighted AUC: 0.8617
> > CV trial 193 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.8191 weighted AUC: 0.9255
> > CV trial 223 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.4982 weighted AUC: 0.7943
> > CV trial 193 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.8191 weighted AUC: 0.9397
>
If I launch the previous command line with only one 1-D feature as follows, everything is going well:
neuropredict -m meta_data.csv \
-d mask_WMpet_av45_early.ero1.25mm.csv \
-o outdir -t 0.8 -n 250 -k 'all' \
--gs_level 'exhaustive' --classifier "LinearSVC"
Do you have any idea?
Best,
Matthieu
I don't see an error msg here - post the very end of the output also?
also, for n=100 samples, you don't need 250 repetitions. 50 should be fine.
for debugging purposes, just run it with 10 reps, and see if it finishes?
The problem is that nothing comes after what what I outputed. All repetitions have run but the program seems to be blocked thereafter.
that's very weird - I will try look into this over the weekend
Hi Matthieu,
I ran neuropredict with your 1D feature sets on my laptop with the latest version (from #51), and it completed fine - see below.. Obviously there most things are different here (model, version etc), but I don't think 1D features are the problem. Try random forests model with only 10 reps?
(base) $ 17:37:15 SQuark-3 Sources >> neuropredict -m meta_data.csv -d mask_WMpet_av45_early.ero1.* -n 10 -c 1
neuropredict version 0.6.dev3+21.gee9da15.dirty for Classification
Positive class inferred for AUC calculation: DIS
Requested features for analysis:
get_data_matrix from /Users/Reddy/Downloads/Sources/mask_WMpet_av45_early.ero1.25mm.csv
get_data_matrix from /Users/Reddy/Downloads/Sources/mask_WMpet_av45_early.ero1.5mm.csv
Data import is done.
Requested processing for the following subgroups:
CN,DIS
--------------------------------------------------------------------------------
Processing subgroup : CN,DIS (1/1)
--------------------------------------------------------------------------------
CN_DIS:
101 samples, 2 modalities, dims: [1, 1]
Identifiers: 0, 1
Attributes:
Classes n=2, sizes CN: 71, DIS: 30
Ignoring imputation strategy chosen, as no missing data were found!
CURRENT EXPERIMENT:
--------------------------------------------------
Training percentage : 0.5
Number of CV repetitions : 10
Number of processors : 1
Dim reduction method : variancethreshold
Dim reduction size : tenth
Predictive model chosen : randomforestclassifier
Grid search level : light
Estimated chance accuracy : 0.500
Saving results to:
/Users/Reddy/Downloads/Sources/neuropredict_results/CN_DIS
CV run 0 dataset 0 : accuracy_score 0.667 balanced_accuracy_score 0.529
CV run 0 dataset 1 : accuracy_score 0.627 balanced_accuracy_score 0.529
CV run 1 dataset 0 : accuracy_score 0.784 balanced_accuracy_score 0.690
CV run 1 dataset 1 : accuracy_score 0.725 balanced_accuracy_score 0.630
CV run 2 dataset 0 : accuracy_score 0.529 balanced_accuracy_score 0.670
CV run 2 dataset 1 : accuracy_score 0.549 balanced_accuracy_score 0.682
CV run 3 dataset 0 : accuracy_score 0.725 balanced_accuracy_score 0.669
CV run 3 dataset 1 : accuracy_score 0.667 balanced_accuracy_score 0.647
CV run 4 dataset 0 : accuracy_score 0.765 balanced_accuracy_score 0.682
CV run 4 dataset 1 : accuracy_score 0.745 balanced_accuracy_score 0.691
CV run 5 dataset 0 : accuracy_score 0.706 balanced_accuracy_score 0.603
CV run 5 dataset 1 : accuracy_score 0.686 balanced_accuracy_score 0.548
CV run 6 dataset 0 : accuracy_score 0.725 balanced_accuracy_score 0.619
CV run 6 dataset 1 : accuracy_score 0.647 balanced_accuracy_score 0.625
CV run 7 dataset 0 : accuracy_score 0.725 balanced_accuracy_score 0.678
CV run 7 dataset 1 : accuracy_score 0.784 balanced_accuracy_score 0.718
CV run 8 dataset 0 : accuracy_score 0.706 balanced_accuracy_score 0.575
CV run 8 dataset 1 : accuracy_score 0.686 balanced_accuracy_score 0.562
CV run 9 dataset 0 : accuracy_score 0.725 balanced_accuracy_score 0.691
CV run 9 dataset 1 : accuracy_score 0.706 balanced_accuracy_score 0.676
Results saved to /Users/Reddy/Downloads/Sources/neuropredict_results/CN_DIS/results_neuropredict.pkl
Error in removing temp dir - remove it yourself:
{} /Users/Reddy/Downloads/Sources/neuropredict_results/CN_DIS/temp_dump
Metrics : accuracy_score, balanced_accuracy_score, area_under_roc
# runs : 20, # datasets : 2
accuracy_score
0 : median 0.7255 SD 0.0662
1 : median 0.6863 SD 0.0625
balanced_accuracy_score
0 : median 0.6695 SD 0.0532
1 : median 0.6388 SD 0.0618
area_under_roc
0 : median 0.7272 SD 0.0510
1 : median 0.6954 SD 0.0715
Results have been saved to
/Users/Reddy/Downloads/Sources/neuropredict_results/CN_DIS/results_neuropredict.pkl
All done.
Also, using the latest stable release from pypi 0.5.2 also works (after the fix for get_data_matrix()).. I'd say something in your setup isn't right.
(base) $ 18:20:58 SQuark-3 Sources >> neuropredict -m meta_data.csv -d mask_WMpet_av45_early.ero1.* -n 10 -c 1
Positive class inferred for AUC calculation: CN
Running neuropredict 0.5.2
Requested features for analysis:
get_data_matrix from /Users/Reddy/Downloads/Sources/mask_WMpet_av45_early.ero1.25mm.csv
get_data_matrix from /Users/Reddy/Downloads/Sources/mask_WMpet_av45_early.ero1.5mm.csv
Ignoring imputation strategy chosen, as no missing data were found!
Data import is done.
Requested processing for the following subgroups:
DIS,CN
--------------------------------------------------------------------------------
Processing subgroup : DIS,CN (1/1)
--------------------------------------------------------------------------------
Training percentage : 0.5
Number of CV repetitions : 10
Classifier chosen : randomforestclassifier
Feature selection chosen : variancethreshold
Level of grid search : light
Number of processors : 1
Saving the results to
/Users/Reddy/Downloads/Sources/neuropredict_results/CN_DIS
-------------------------
All datasets contain:
101 samples, 2 classes, 1 features
Class CN : 71 samples
Class DIS : 30 samples
-------------------------
Estimated chance accuracy : 0.500
Different classes in the training set are stratified to match the smallest class!
CV trial 0 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7369 weighted AUC: 0.7542
CV trial 0 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7458 weighted AUC: 0.8024
CV trial 1 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7702 weighted AUC: 0.8173
CV trial 1 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7768 weighted AUC: 0.7911
CV trial 2 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7702 weighted AUC: 0.7601
CV trial 2 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6256 weighted AUC: 0.6708
CV trial 3 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7304 weighted AUC: 0.7423
CV trial 3 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6815 weighted AUC: 0.7631
CV trial 4 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7637 weighted AUC: 0.7274
CV trial 4 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.5190 weighted AUC: 0.5958
CV trial 5 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7637 weighted AUC: 0.8030
CV trial 5 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7458 weighted AUC: 0.7815
CV trial 6 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.7256 weighted AUC: 0.7357
CV trial 6 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.7167 weighted AUC: 0.6839
CV trial 7 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6750 weighted AUC: 0.6756
CV trial 7 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6262 weighted AUC: 0.5863
CV trial 8 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6905 weighted AUC: 0.6500
CV trial 8 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6435 weighted AUC: 0.6685
CV trial 9 feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy: 0.6637 weighted AUC: 0.6869
CV trial 9 feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy: 0.6548 weighted AUC: 0.6827
The most frequently selected parameter values are:
For feature set 0 : mask_WMpet_av45_early.ero1.25mm
random_forest_clf__criterion : gini
random_forest_clf__max_features : sqrt
random_forest_clf__min_samples_leaf : 1
random_forest_clf__n_estimators : 50
For feature set 1 : mask_WMpet_av45_early.ero1.5mm
random_forest_clf__criterion : gini
random_forest_clf__max_features : sqrt
random_forest_clf__min_samples_leaf : 1
random_forest_clf__n_estimators : 50
Exporting accuracy distribution .. Done.
Exporting confusion matrices .. Done.
Exporting misclassfiication rates .. Done.
Exporting feature importance values .. Done.
Exporting subject-wise misclf frequencies .. Done.
Median performance summary:
feature 0 mask_WMpet_av45_early.ero1.25mm : balanced accuracy 0.73 AUC 0.74
feature 1 mask_WMpet_av45_early.ero1.5mm : balanced accuracy 0.67 AUC 0.68
Saving the visualizations to
/Users/Reddy/Downloads/Sources/neuropredict_results
reopen if you still find it as an issue.