argmaxinc / WhisperKit

On-device Speech Recognition for Apple Silicon

Home Page:https://takeargmax.com/blog/whisperkit

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Implement memory and latency regression tests

atiorh opened this issue · comments

Implement tests to transcribe long audio files (at least several minutes worth) and measure the memory and latency over time. This is to guard against memory leaks or slowdowns potentially being introduced by new PRs (e.g. #40 fixed by #56 thanks to @finnvoor!)

Any pointers on how to start with this. I'd like to take it up.

I assume the tests would be functionaltests and we'd be measuring the metrics but not sure on the details.

@Abhinay1997 Check inside FunctionalTests.swift for the measure calls. We'll want to save these results somewhere so they can be compared to in subsequent runs, but we'd need to makes sure it is hardware specific so we don't think there's a regression on a slower computer vs a faster one.