Automated tools for benchmarking
RobertTLange opened this issue · comments
I would like to have a set of utilities that automatically generates jobs and schedules them for the different hardware platforms. E.g. CPU/GPU/TPU. This should probably all run on GCP and with docker containers for reproducibility. Have a look pyhpc for benchmarking standards and potentially integrate with mle-toolbox GCP experiment setup.
Also add nice visualizations!