`vl_covdet_extract_orientations()` does not check OOM condition of `vl_covdet_extract_orientations_for_frame()`

[Niklas Hambüchen]

In

vlfeat/vl/covdet.c

Lines 2877 to 2880 in 1b9075f

 VlCovDetFeatureOrientation* orientations = 
 vl_covdet_extract_orientations_for_frame(self, &numOrientations, feature.frame) ; 
  
 for (j = 0 ; j < (signed)numOrientations ; ++j) { 

vl_covdet_extract_orientations_for_frame() is called and its return value is not checked, despite the called function's docs:

The function returns @c NULL if memory is insufficient.

Thus vl_covdet_extract_orientations() can silently continue despite allocation failure.

---

A proper fix requires an API change:

The type signature of vl_covdet_extract_orientations() needs to be changed to be able to report allocation failure, because it calls a function that can fail to allocate.

I suggest that like vl_covdet_put_image(), it should return int with semantics:

The function fails by returing ::VL_ERR_ALLOC if the memory is insufficient.

[Niklas Hambüchen]

Another thing that's odd is that some functions in covdet.c that return vl_set_last_error(...) return vl_bool instead of the int, thus losing the information of what the error was.

It's unclear to me if that is intended, or an oversight.

Especially because the documented failure reasons also seem to be missing some reasons:

 /** The function returns @c NULL if memory is insufficient. **/
vl_covdet_extract_orientations_for_frame (VlCovDet * self,
                                          vl_size * numOrientations,
                                          VlFrameOrientedEllipse frame)
{
  // ..
  err = vl_covdet_extract_patch_helper(self,

and vl_covdet_extract_patch_helper() can return:

    if ((x0 < 0 || x1 > (signed)width-1 || y0 < 0 || y1 > (signed)height-1) &&
        !self->allowPaddedWarping) {
      return vl_set_last_error(VL_ERR_EOF, "Frame out of image.");
    }

So there's another reason why vl_covdet_extract_orientations_for_frame() can fail, that's not "memory is insufficient".