The purpose of this analysis is to ensure SellBy stakeholders' are as informed as possible when evaluating the value of participating in the Amazon Vine program. While Amazon started the program to better cultivate unbiased reviews from the platform's most trusted reviewers, SellBy's stakeholders are interested if an analysis of Amazon review data reveals any positivity bias for reviews generated by members of the Vine program.
Amazon has made datasets for the US based reviews of nearly 50 different product categories publicly available. For this analysis, the reviews dataset for the Watches category was selected and has been examined to determine if any bias exists towards reviews in the Vine program.
The questions used in evaluating bias and results of the Watches category analysis are as follows:
-
How many Vine reviews and non-Vine reviews were there?
- There were 47 Vine reviews and 8,362 non-Vine reviews.
- There were 47 Vine reviews and 8,362 non-Vine reviews.
-
How many Vine reviews were 5 stars? How many non-Vine reviews were 5 stars?
- 15 Vine reviews were 5 stars. 4,332 non-Vine reviews were 5 stars.
- 15 Vine reviews were 5 stars. 4,332 non-Vine reviews were 5 stars.
-
What percentage of Vine reviews were 5 stars? What percentage of non-Vine reviews were 5 stars?
- 31.9% of Vine reviews were 5 stars. 51.8% of non-Vine reviews were 5 stars.
- 31.9% of Vine reviews were 5 stars. 51.8% of non-Vine reviews were 5 stars.
The results of this analysis demonstrates no positivity bias exists among reviews that are part of the Vine program. In fact, the analysis of the Watches category suggests that the honest, expert reviewers Amazon has selected to participate in the Vine program for the category are much tougher critics than their non-Vine peers. However, this analysis is far from conclusive and further analysis of the Amazon Vine data is warranted. Given that SellBy has a diverse product catalogue, analyzing only one of the many product different categories available for study is insufficient.
To make the best decisions possible, SellBy's stakeholders should have an analysis of potential positivity bias for each of the company's largest segments and/or those for which they are considering enrolling in the Vine program. Given that Vine reviewers are not product category agnostic -rather they are reviewers acknowledged with having an expertise within specific categories- it would be prudent for SellBy stakeholders to evaluate relevant categories individually, in certain buckets and as a whole. This holistic analysis will allow SellBy to properly assess the potential value of the Vine program and how to best structure their participation.