Threshold value in engine.py not working?

[Dominique Chen]

hello,

first of all, thank you for this great library!

I managed to make it work on my M1 MacBook Air, and trying out my personal hotword detection, but the threshold value does not seem to be working on my environment.

In engine.py:

    def __init__(
            self,
            hotword:str,
            reference_file:str,
            threshold:float=0.995,
            activation_count=2,
            continuous=True,
            verbose = False):

And this is my script:

import os
from eff_word_net.streams import SimpleMicStream
from eff_word_net.engine import HotwordDetector
from eff_word_net import samples_loc

hotword_hw = HotwordDetector(
        hotword="hotword",
        reference_file = "hotword_ref.json",
        activation_count=3
    )

mic_stream = SimpleMicStream()
mic_stream.start_stream()

print("Say hotword ")
while True :
    frame = mic_stream.getFrame()
    result = hotword_hw.checkFrame(frame)
        print("Wakeword uttered")
        print(hotword_hw.getMatchScoreFrame(frame))

and when I run this script, the checkFrame returns true even when the getMatchScoreFrame returns under the threshold, like:

Wakeword uttered
0.9371609374494279
Wakeword uttered
0.9164050520717595
Wakeword uttered
0.9082509350226378
...

Could you please take a look at this?

Thank you!

[Aman Rangapur]

Can you please mention your exact concern?

[Dominique Chen]

My concern is that even I set the threshold to 0.995, the checkFrame method does not seem to take it into account (it passes even when the score is lower than 0.995)

[Aman Rangapur]

We will fix it and update you shortly.

[Chidhambararajan]

This is a result of confusion caused by a bit overcomplicated api, check out the latest release which i pushed today . Super simplified api logic. Upgrade your installation to the latest one and let me know

[Dominique Chen]

@TheSeriousProgrammer thank you for the update!
I just used your latest code v2.2, here's what I did:

• regenerated ref file using python -m eff_word_net.generate_reference with the same sound files as before
• updated my detection script using your new demo script (https://github.com/Ant-Brain/EfficientWord-Net#try-your-first-single-hotword-detection-script)
• run the code
  and here are the issues:
• it detects ( Wakeword uttered ) even when I speak just one syllable of the wake word
• it detects ( Wakeword uttered ) even when I say something else
• the threshold value does not seem to be working (it detects even when result["confidence"] is lower than the threshold value specified in the HotwordDetector constructor)

Could you please take a look at these?
Let me know if you need my sound files to reproduce the issue.
Thank you.

[Chidhambararajan]

@TheSeriousProgrammer thank you for the update!
I just used your latest code v2.2, here's what I did:

• regenerated ref file using python -m eff_word_net.generate_reference with the same sound files as before
• updated my detection script using your new demo script (https://github.com/Ant-Brain/EfficientWord-Net#try-your-first-single-hotword-detection-script)
• run the code
  and here are the issues:
• it detects ( Wakeword uttered ) even when I speak just one syllable of the wake word
• it detects ( Wakeword uttered ) even when I say something else
• the threshold value does not seem to be working (it detects even when result["confidence"] is lower than the threshold value specified in the HotwordDetector constructor)

Could you please take a look at these?
Let me know if you need my sound files to reproduce the issue.
Thank you.

Instead of modifying threshold in engine.py can you declare it your hotword instance of your script? I guess you are modifying it in a different file than what the program is looking into

[Dominique Chen]

Instead of modifying threshold in engine.py can you declare it your hotword instance of your script? I guess you are modifying it in a different file than what the program is looking into

No, I was actually declaring threshold in the constructor as below:

import os
from eff_word_net.streams import SimpleMicStream
from eff_word_net.engine import HotwordDetector
from eff_word_net import samples_loc

customHotword_hw = HotwordDetector(
        hotword="customHotword",
        reference_file = "/customHotword_ref.json",
        threshold=0.95, #min confidence required to consider a trigger
        relaxation_time = 0.8 #default value ,in seconds
    )

mic_stream = SimpleMicStream()
mic_stream.start_stream()

print("Say nukabot ")
while True :
    frame = mic_stream.getFrame()
    result = customHotword_hw.scoreFrame(frame)
    if result==None :
        #no voice activity
        continue
    if(result["match"]):
        print("Wakeword uttered",result["confidence"])

[Chidhambararajan]

unable to reproduce the error , i forgot to the push latest changes the package to pypi but now have, consider upgrading your package and clearing your virtual environment incase of any cache

[Dominique Chen]

Thank you, I will try that and let you know again.

[Chidhambararajan]

Closing the issue due to inactivity feel free to open it ,if issue persists