`detector` in binary mode gives false result

[Romain Jennes]

Detectors that allow multiclass detection can't be used in binary mode. The result is that they classify all executables as packed.

Preparation steps

$ dataset make baseline -n 100 -f PE -p upx

Issue

PEiD, for example, gives correct results in multiclass mode:

$ detector baseline -d PEiD
<<snipped>>
Detection results:
  ────────  ─────────  ──────  ─────────
  Accuracy  Precision  Recall  F-Measure
  83.00%    100.00%    63.04%  77.33%
  ────────  ─────────  ──────  ─────────

But fails in binary mode because classes everything as packed:

$ detector baseline -b -d PEiD
<<snipped>>
Detection results:
  ────────  ─────────  ───────  ─────────
  Accuracy  Precision  Recall   F-Measure
  29.00%    29.00%     100.00%  44.96%
  ────────  ─────────  ───────  ─────────

In contrary, detectors that can only work in binary mode give good results:

$ detector baseline -b -d REMINDer
<<snipped>>
 Detection results:
  ────────  ─────────  ───────  ─────────
  Accuracy  Precision  Recall   F-Measure
  97.00%    93.88%     100.00%  96.84%
  ────────  ─────────  ───────  ─────────