Option for multiple base skin colours

[Robert O'Rourke]

This is more of a question in light of the recent debate on Twitter about their neural net saliency based approach getting things wrong.

The current default with the upper and lower lightness / darkness bounds seems like a reasonable compromise between speed and accuracy but how much speed is sacrificed by adding say another 2-3 base colours? What are your thoughts on accepting an array of skin colours? E.g.

[
   [ r1, g1, b1 ],
   [ r2, g2, b2 ],
   [ r3, g3, b3 ],
]

Effectively adding another loop around this part:

https://github.com/jwagner/smartcrop.js/blob/master/smartcrop.js#L211-L223

I should also add I'm actually sure what the range of skin tones the upper and lower lightness bounds includes at the moment. Is that documented anywhere?

[Jonas Wagner]

Just for reference this has already been discussed here: #9

The skin detection was designed to handle different skin colors and they are also represented in the testing dataset.

skinColor is a bit of a misnomer here. It's a normalized luminance vector in a gamma[1] color space. So it's more of a line than a point. If I recall my experiments back then correctly the different skin colors did indeed somewhat resemble a line. At least when ignoring the different types of pigment that does also have some correlation with the physical world - the density of pigments is a single free variable resulting in a line. But yes it’s definitely far from perfect.

Yes one could add more colors and then pick the max() of the results, yes the performance would quite possibly be adequate, yes, that will likely increase the true positive rate but also at the cost of higher false positives. The thing is that the intensity and color of the light will in many cases be a much greater factor than skin color deviating off that line just a bit.

What would be needed to improve this aspect is a more scientific approach: reviewing the literature, evaluating the options and then implementing one of them. But as you also mentioned this is a highly politicized debate.

This is a hobby project. I work on these projects to a large extent to relax and regain motivation. Participating in politically loaded debates really doesn't do either of those things for me. So I hope you can understand that I'm currently not up to revisit this aspect.

[1] Not sure why I made that choice, possibly just because I could get away with it.

[Robert O'Rourke]

Thanks for the response! Totally get that it's a hobby project, and I do think you've taken a considered and clever approach in the current implementation. You've answered the question really - didn't want to make this a charged or loaded debate in any way. I'll close this out.