1.Classical AB test 2. Sequential AB test 3. AB testing with Machine Learning
An advertising company is running an online ad for a client with the intention of increasing brand awareness. The advertiser company earns money by charging the client based on user engagements with the ad it designed and serves via different platforms. To increase its market competitiveness, the advertising company provides a further service that quantifies the increase in brand awareness as a result of the ads it shows to online users. The main objective of this project is to test if the ads that the advertising company runs resulted in a significant lift in brand awareness.
SmartAd is a mobile first advertiser agency. It designs intuitive touch-enabled advertising. It provides brands with an automated advertising experience via machine learning and creative excellence. Their company is based on the principle of voluntary participation which is proven to increase brand engagement and memorability 10 x more than static alternatives. SmartAd provides an additional service called Brand Impact Optimiser (BIO), a lightweight questionnaire, served with every campaign to determine the impact of the creative, the ad they design, on various upper funnel metrics, including memorability and brand sentiment. As a Machine learning engineer in SmartAd, one of your tasks is to design a reliable hypothesis testing algorithm for the BIO service and to determine whether a recent advertising campaign resulted in a significant lift in brand awareness.
The analysis objective of this project are divided into 4 sub-objectives that overall guides the workflow
- Setting up A/B testing framework
- Setting up repeatable ML framework
- Performing A/B testing with classical, sequential and Machine learning methods
- using MLOps best practices
- Extracting statistically valid insights in relation to the business objective
- Apply your code pipeline to a new dataset
Task 1: A/B testing framework
Task 2: A/B testing with Machine Learning
Task 3: Apply your pipeline to a new dataset
Task 4 : Interpretation & Reporting