llandsmeer / iotf

Inferior olive benchmark

Requirements

• numpy
• matplotlib
• tensorflow - make sure you are using the right version if you want to run on CUDA
• onnxruntime
• onnxruntime-gpu (optional) - to run on CUDA and TensorRT
• Graphcore tensorflow SDK (optional) - to run on Graphcore
• GroqWare SDK (optional) - to run on Groq
• tf2onnx

Usage

Running benchmarks (creating a benchmark.txt file). Which Runners to benchmark is chosen at runtime based on available hardware.

$ git clone https://github.com/llandsmeer/iotf
$ cd iotf
$ python3 -m ioperf

Runnings tests (with optional graphcore tests specialization)

$ pytest ioperf --disable-warnings [-k graphcore]
[..]
2 passed, 11 deselected, 3 warnings in 6.03s

After making changes, run bash build-docs.sh and commit

Module documentation

• docs
  • model
  • bench
    • benchmark
    • model_configuration
    • tests
      • test_model_configuration
  • runners
    • groqchip_runner_opt2_nocopy
    • onnx_tensorrt_runner
    • groqchip_runner_opt1
    • base_runner
    • groqchip_runner
    • onnx_cpu_runner
    • onnx_cuda_runner
    • tests
      • test_graphcore_runner_with_shim
      • test_unroll_gj_on_cpu
      • test_groqchip_runner
      • test_base_runner
      • test_onnx_cpu_runner
      • test_onnx_gpu_runner
    • onnx_base_runner
    • graphcore_runner

Authors

• Max Engelen
• Lennart Landsmeer