This project analyzes the performance of a digital marketing ad campaign using A/B Testing methodologies. The goal was to quantify the incremental lift in user conversions generated by the ads compared to a baseline Public Service Announcement (PSA) control group.
Using a dataset of 588,000+ user records, this analysis provides data-backed recommendations to optimize marketing spend and scale successful campaigns.
- Quantify Impact: Calculate the Conversion Rate and Lift generated by the ads.
- Validate Results: Use Statistical Hypothesis Testing (Chi-Square) to ensure results are not due to random chance.
- Optimize Strategy: Segment data by day/time to identify high-performance windows for budget allocation.
##Formula for Lift:
-
Statistical Significance: We employ a Chi-Square Test (
$\chi^2$ ) to determine if the observed lift is statistically valid (p-value < 0.05) or due to random chance.
- Language: Python 3.x
- Libraries: Pandas, NumPy, SciPy, Matplotlib, Seaborn
-
Techniques: Data Cleaning, GroupBy Aggregations, A/B Testing, Chi-Square Test (
$\chi^2$ )
The dataset contains user interaction data with the following fields:
user id: Unique identifier for each user.test group: "ad" (Treatment) vs. "psa" (Control).converted: Boolean (True= Bought,False= Didn't Buy).total ads: Number of ads seen by the user.most ads day: Day of the week the user saw the most ads.most ads hour: Hour of the day the user saw the most ads.
| Metric | Control Group (PSA) | Test Group (Ad) |
|---|---|---|
| Conversion Rate | 1.79% | 2.55% |
| Lift Generated | - | +43.09% 🚀 |
- Test Used: Chi-Square Test for Independence
- P-Value:
1.99e-13(< 0.05) - Conclusion: The results are Statistically Significant. We can reject the null hypothesis with >99% confidence.
Based on the analysis, the following actions are recommended:
- Scale the Campaign: The ads are driving a massive 43% incremental lift in conversions. The budget should be increased to reach a wider audience.
- Optimize Day-Parting: Segmentation analysis (see notebook) revealed that [Insert Best Day] has the highest engagement. Future ad spend should be weighted heavily towards these peak days to maximize ROI.
- Clone the repository:
git clone [https://github.com/yourusername/marketing-lift-analysis.git](https://github.com/ankit2021-nit/Ad-Campaign-Optimization-ROI-Measurement.git)
- Install dependencies:
pip install pandas numpy matplotlib seaborn scipy
- Run the Jupyter Notebook:
jupyter notebook Campaign_Lift_Analysis.ipynb
Created by Ankit Prasad