mathchi / DS_Analyze-and-Present-AB-Test-Results

Data Science Analyze and Present A/B Test Results

-------- company recently introduced a new bidding type, “average bidding”, as an alternative to its exisiting bidding type, called “maximum bidding”. One of our clients, --------.com, has decided to test this new feature and wants to conduct an A/B test to understand if average bidding brings more conversions than maximum bidding.

In this A/B test, --------.com randomly splits its audience into two equally sized groups, e.g. the test and the control group. A --------- company ad campaign with “maximum bidding” is served to “control group” and another campaign with “average bidding” is served to the “test group”.

The A/B test has run for 1 month and --------.com now expects you to analyze and present the results of this A/B test.

You should answer the following questions in your presentation:

1. How would you define the hypothesis of this A/B test?
2. Can we conclude statistically significant results?
3. Which statistical test did you use, and why?
4. Based on your answer to Question 2, what would be your recommendation to client?

Hints:

1. Your presentation should last about 15 minutes, and should be presented in English.
2. The ultimate success metric for HotelsForYou.com is Number of Purchases. Therefore, you should focus on Purchase metrics for statistical testing.
3. Explain the concept of statistical testing for a non-technical audience.
4. The customer journey for this campaign is:
   1. User sees an ad (Impression)
   2. User clicks on the website link on the ad (Website Click)
   3. User makes a search on the website (Search)
   4. User views details of a product (View Content)
   5. User adds the product to the cart (Add to Cart)
   6. User purchases the product (Purchase)
5. Use visualizations to compare test and control group metrics, such as Website Click Through Rate, Cost per Action, and Conversion Rates in addition to Purchase numbers.
6. If you see trends, anomalies or other patterns, discuss these in your presentation.
7. You can make assumptions if needed.

Note: I translated also Turkish language