#
Human Augmented Training Data in Learning To Rank ##AbstractLearning to rank methods and their applications such as relative attributes are extensively used in AI. Though there is a variety of learning to rank methods, those which use pairwise guidance are especially popular. However, pairwise learning to rank methods require a substantial amount of training for the learning algo- rithms to perform well. In this paper we explore how human local annotation can be used to augment existing training beyond the given training pairs. Our empirical results show the promise of such a direction is substantial. ##
IntroductionExisting pairwise learning to rank rely on high quality training pairs and substantial training samples. It has been widely shown and observed pragmatically in many fields that the more training data available the more useful the ranking function. Conversely, if inconsistent training data is provided performance can be greatly impacted. Our core idea is to use human feedback in the form of localized annotations to generate more training data.The form of feedback is for the human expert to outline what parts of the object are the reason for the ranking.
(a) Given training images A and B, ordered in strength as
##
Approach Step 1:Get users' judgement in two images regarding attribute strength.Step 2:Generate augmented training data.
Step 3:Rank images via RankSVM with original training data and augmented data.
###Augmenting constraints in RankSVM
We now have a way to augment each original ordered training pair
- are the constraints generated from the synthetic pairs that enforce a new margin
$\mu \ge 0$ , which is parameter that reflects our confidence in the synthetic augmented data. When$\mu=1$ , the augmented training pairs are weighted equally as the original data pairs, and when$\mu<1$ or$\mu>1$ , we give them less or more weight compared to the original pairs, respectively. In practice, we set$\mu$ via cross validation.
##
Experimental Result ###Datasets- LFW10, which is a subset of the Labeled Faces in the Wild dataset. It consists of 2000 human face images (1000 for training and 1000 for testing) with different poses, lighting conditions, and background clutter.
- Shoes-with-attributes (SWA), which is downloaded from like.com and consists of 14658 shoe images;
###Ranking accuracy with augmented training data We perform detailed analyses using CNN features(Convolutionary Neural Network) and compare only to the stronger original-only baseline due to its superiority over other features.
###Robustness of adding augmented training data --- ##Claim Given human's rantionale, we demonstrate one way to teach machine intuitively. We plan to add more dataset to prove the correctness of idea:human augmented training data. And we would like to understand why this phenomenon exists.