Contains experimentation notebooks for my Keras Example Consistency Training with Supervision. This example also provides a template for performing semi-supervised / weakly supervised learning.
Promising results on CIFAR-10-C with the process shown in the example:
More things one can incorporate:
- Incorporate more data during training the student.
- Filter high confidence predictions from teacher during training the student.
- Use recipes like Stochastic Depth for training the teacher. The current example uses Stochastic Weight Averaging to induce geometric ensembling.
Full-scale experiments are available here.
- ML-GDE program for providing GCP credits.