This project is designed as a complete educational guide for anyone starting out in data science and looking to understand the A/B testing workflow in depth.
It combines theory and practice to help you learn not only the statistical concepts behind controlled experiments, but also how to apply them step by step using Python.
File: A_B_Testing_Theoretical_Guide.pdf
This document introduces the theoretical foundations of A/B Testing in a structured and accessible way. It walks through:
- What A/B Testing is and why it matters
- How to define a business problem and set success metrics
- Formulating statistical hypotheses
- Understanding concepts like MDE, power, and p-values
- Designing an experiment and calculating sample size
- Ensuring internal validity and avoiding common pitfalls
- Making data-driven decisions from results
It’s ideal for anyone who wants to understand the "why" behind each step before jumping into code.
File: A_B_Testing_Practical_Notebook.ipynb
This Jupyter Notebook is the hands-on companion to the theoretical guide. Using a simulated dataset, it applies each concept from theory into practice:
- Setting up hypotheses and key metrics
- Calculating sample size and planning the experiment
- Validating the experimental design (randomization, balance, instrumentation)
- Performing statistical tests (Z-test, t-tests, Mann–Whitney)
- Visualizing results by day and hour
- Segmenting user behavior to extract insights
The notebook is written and structured for learners — with clear comments, visualizations, and real-world explanations.
The dataset used is available on Kaggle.
- People new to data science looking for a structured example of A/B testing
- Students who want to reinforce statistical concepts with code
- Analysts or developers transitioning into experimentation and product analytics
A_B_Testing_Theoretical_Guide.pdf– Theoretical guideA_B_Testing_Practical_Notebook.ipynb– Jupyter Notebook implementation