View Proposal

Proposer
William Yoo
Title
AI-Driven A/B Testing for Enhanced Digital Ad Optimization
Goal
This project explores how AI and machine learning can enhance A/B and multi-group A/B testing methods in the context of digital marketing advertisements. It focuses on how different ad variations such as phrasing, design, or visual elements can influence user engagement and interaction outcomes.
Description
In the first stage, the study will use open-source datasets related to ad performance or online marketing, applying statistical techniques and machine learning models to evaluate group differences and identify key factors influencing success metrics. Datasets will be sourced from public platforms such as Kaggle to ensure relevance and accessibility. The second stage involves designing and conducting a new A/B or multi-group A/B test. AI tools like large language models or image generators will be used to create diverse ad variants for testing. These versions will be deployed online in a controlled environment to gather real user responses. The collected data will then be analyzed to determine which creative elements most significantly impact metrics like click-through rates or user preferences. By combining AI-driven content generation with data science techniques, the project aims to demonstrate how A/B testing can be enhanced for more informed and dynamic marketing decisions.
Resources
Zhang, A., Lipton, Z. C., Li, M. and Smola, A. J. (2023). Dive into Deep Learning. Cambridge University Press. https://D2L.ai (Key text: read Chapters 9, 10 ,11) Liu, C. H. B., Cardoso, Â., Couturier, P. and McCoy, E. J. (2021). Datasets for Online Controlled Experiments. In J. Vanschoren and S. Yeung (Eds.), Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks Vol. 1. https://datasets-benchmarks-proceedings.neurips.cc/paper/2021/file/274ad4786c3abca69fa097b85867d9a4-Paper-round2.pdf Quin, F., Weyns, D., Galster, M. and Costa Silva, C. (2024). A/B testing: A systematic literature review. Journal of Systems and Software, 211(2024), Article 112011. https://doi.org/10.1016/j.jss.2024.112011 Wang, D. et. al. (2025). AgentA/B: Automated and Scalable Web A/B Testing with Interactive LLM Agents. Arxiv. https://arxiv.org/abs/2504.09723
Background
Url
Difficulty Level
Moderate
Ethical Approval
None
Number Of Students
1
Supervisor
William Yoo
Keywords
a/b testing, two-sample tests, multi-arm bandit, ai agent
Degrees
Bachelor of Science in Statistical Data Science
Back to List