Giters

johnantonn / cash-for-unsupervised-ad

Systematic Evaluation of CASH Search Strategies for Unsupervised Anomaly Detection

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Report polishing

johnantonn opened this issue · comments

commented

Additions/modifications:

  • Statistical test for normality of execution times and relevant graph
  • Graph(s) with confidence intervals for performance
  • H4: search space content
  • Statistical comparison of search algorithms (SMAC, Random Search, UE) using Friedmann and post-hoc Nemenyi tests
commented

The distribution of execution times is not normal based on:

  • Histogram plot (see figure below)
  • Shapiro-Wilk test that yields a p-value of 1e-7

image

That effectively means that the total budget estimation is not accurate.