Discrepancy between OKS Calculation Equation and Actual Code Implementation

[Chan-Yang Ju]

    def computeOks(self, imgId, catId):
        p = self.params
        # dimention here should be Nxm
        gts = self._gts[imgId, catId]
        dts = self._dts[imgId, catId]
        inds = np.argsort([-d['score'] for d in dts], kind='mergesort')
        dts = [dts[i] for i in inds]
        if len(dts) > p.maxDets[-1]:
            dts = dts[0:p.maxDets[-1]]
        # if len(gts) == 0 and len(dts) == 0:
        if len(gts) == 0 or len(dts) == 0:
            return []
        ious = np.zeros((len(dts), len(gts)))
        sigmas = p.kpt_oks_sigmas

        # # # Problem spot            
        vars = (sigmas * 2)**2     
        # # #
        k = len(sigmas)

In accordance with the formula for Object Keypoint Similarity (OKS) provided by the Coco API, OKS = Σi[exp(-di2/2s2κi2)δ(vi>0)] / Σi[δ(vi>0)], we should calculate the "2s2κi2" part in the code as 2 * (s ** 2) * (k_i ** 2).

However, in your method code (computeOks) located in "cocoapi/PythonAPI/pycocotools/cocoeval.py," there is an omission in calculating 's,' and there seems to be an issue with the calculation of the variable using "vars = (sigmas * 2)**2."

While it is true that you disregard 's' because this evaluation is for pose joints, I believe it is more accurate to define 'vars' as (sigmas ** 2) * 2.

If I am mistaken, please correct me.

[Baalon]

I also noticed this mistake yesterday. Furthermore, a few lines down the code, the rest of the OKS calculation has:

229    e = (dx**2 + dy**2) / vars / (gt['area']+np.spacing(1)) / 2
230    if k1 > 0:
231      e=e[vg > 0]
232    ious[i, j] = np.sum(np.exp(-e)) / e.shape[0]

So the "2 * (s ** 2) * (k_i ** 2)" is actually written here, line 229.
That means that from my understanding, to get the proper OKS formula:

• vars should be "(sigmas ** 2)" and neither the actual "vars = (sigmas * 2)**2", nor the version you propose "(sigmas ** 2) * 2" OR
• the "/2" at the end of line 229 should be removed AND vars calculation should be changed to what you propose.

Out of these, I think the first option makes more sense.

Note also, that while the behaviour of line 229 is maybe correctly interpreted by Python (I hope this was tested and validated), it also seems flawed, and breaks first-order logics as it is ambiguous and can be interpreted in ways that make the calculation wrong. The calculation is missing several changes (in particular parenthesises) to ensure that the behaviour is correct at all times. In my opinion, a more proper version would be to write it as:
229 e = (dx**2 + dy**2) / (vars * (gt['area']+np.spacing(1)) * 2)