Do php-opencv support full ssd_mobilenet_v2?

[Jeppelelle]

First of, thank you morozovsk for creating this version of php-opencv, it seems great 👍

One question though: Does it work with the full ssd_mobilenet_v2?

The example here: https://github.com/php-opencv/php-opencv-examples/blob/master/detect_objects_by_dnn_mobilenet.php

The example is using ssdlite_mobilenet_v2 and ssd_mobilenet_v1, both of them are having some problems with alot of my animal pictures.

I am using ssd_mobilenet_v2 in Golang and it is much better at recognising animals, almost as good as the caffe model in your example: https://github.com/php-opencv/php-opencv-examples/blob/master/classify_image_by_dnn_mobilenet.php

So i have downloaded the "MobileNet-SSD v2" weights and config from: https://jeanvitor.com/tensorflow-object-detecion-opencv/

I updated my code in "detect_objects_by_dnn_mobilenet.php" to this:

$src = imread($PhotoSource); // opencv loads image to matrix with BGR order


$blob = \CV\DNN\blobFromImage($src, 0.017, new \CV\Size(300,300), new Scalar(127.5, 127.5, 127.5), true, false); // convert image to 4 dimensions matrix
//var_export($blob);

//$net = \CV\DNN\readNetFromTensorflow('models/ssd_mobilenet_v1_coco/frozen_inference_graph.pb', 'models/ssd_mobilenet_v1_coco/ssd_mobilenet_v1_coco.pbtxt');
$net = \CV\DNN\readNetFromTensorflow('opencv/models/mobilenetv2/frozen_inference_graph.pb', 'opencv/models/mobilenetv2/ssd_mobilenet_v2_coco_2018_03_29.pbtxt');
//$net = \CV\DNN\readNetFromTensorflow('opencv/models/ssdlite_mobilenet_v2_coco/frozen_inference_graph.pb', 'opencv/models/ssdlite_mobilenet_v2_coco/ssdlite_mobilenet_v2_coco.pbtxt');


$net->setInput($blob, "");

$r = $net->forward();
// var_export($r);



$rectangles = [];
for ($i = 0; $i < $r->shape[2]; $i++) {
    $classId = $r->atIdx([0,0,$i,1]);
    $confidence = intval($r->atIdx([0,0,$i,2]) * 100);


    if ($classId && $confidence > 3) {
        $startX = $r->atIdx([0,0,$i,3]) * $src->cols;
        $startY = $r->atIdx([0,0,$i,4]) * $src->rows;
        $endX = $r->atIdx([0,0,$i,5]) * $src->cols;
        $endY = $r->atIdx([0,0,$i,6]) * $src->rows;

        $scalar = new Scalar(0, 0, 0);
        \CV\rectangle($src, $startX, $startY, $endX, $endY, $scalar, 4);

        $text = 'ClassID=' . $classId . '  ' . " $confidence% ";
        echo '<br><br>' . $text;
        \CV\rectangle($src, $startX, $startY - 40, $startX + 18 * strlen($text), $startY, new Scalar(0,0,0), -2);
        \CV\putText($src, " $text ", new \CV\Point($startX, $startY - 10), 0, 1, new Scalar(255,255,255), 0);
    }
}

I had to change to $confidence > 3 in the if statement because the confidence score is so low now and the output is really odd, the boxes is not at all correct & i have tested this on 100 pictures and most of the times it just outputs several classId of 1, it rarely finds anything with another class than 1:

As one can see from the picture, the Caffe classifier is by far the best one, on second place is ssdlite_mobilenet_v2 and at last place and does not work at all is ssd_mobilenet_v2 that i downloaded and used in the code.

So, i guess, is ssd_mobilenet_v2 not compatible with php-opencv?

[Vladimir Goncharov]

Try to use old version of ssd_mobilenet_v2_coco_2018_03_29.pbtxt https://raw.githubusercontent.com/opencv/opencv_extra/4643f97dc1ff9d13b9076029c780fc2f79028310/testdata/dnn/ssd_mobilenet_v2_coco_2018_03_29.pbtxt
I think it was broken in this commit: opencv/opencv_extra@4643f97#diff-f3b333da79a489785ea86ff41c986508

All versions after don't work for me.
May be it was changed for support of opencv 4.1.2.

[Jeppelelle]

Thank you so much, it worked great, now it knows its a bird instead of a dog 👍:

The combination of readNetFromTensorflow() & readNetFromCaffe() seems to be a great combination to categorize/classify images

Really appreciate your help, thanks