Mediation Analysis in Online Experiments at Booking.com: Disentangling Direct and Indirect Effects
Authors:
Bahattin Tolga Öztan,
Zoé van Havre,
Caio Gomes,
Lukas Vermeer
Abstract:
Online experimentation is at the core of Booking.com's customer-centric product development. While randomised controlled trials are a powerful tool for estimating the overall effects of product changes on business metrics, they often fall short in explaining the mechanism of change. This becomes problematic when decision-making depends on being able to distinguish between the direct effect of a tr…
▽ More
Online experimentation is at the core of Booking.com's customer-centric product development. While randomised controlled trials are a powerful tool for estimating the overall effects of product changes on business metrics, they often fall short in explaining the mechanism of change. This becomes problematic when decision-making depends on being able to distinguish between the direct effect of a treatment on some outcome variable and its indirect effect via a mediator variable. In this paper, we demonstrate the need for mediation analyses in online experimentation, and use simulated data to show how these methods help identify and estimate direct causal effect. Failing to take into account all confounders can lead to biased estimates, so we include sensitivity analyses to help gauge the robustness of estimates to missing causal factors.
△ Less
Submitted 29 October, 2018;
originally announced October 2018.
Complex Politics: A Quantitative Semantic and Topological Analysis of UK House of Commons Debates
Authors:
Stefano Gurciullo,
Michael Smallegan,
María Pereda,
Federico Battiston,
Alice Patania,
Sebastian Poledna,
Daniel Hedblom,
Bahattin Tolga Oztan,
Alexander Herzog,
Peter John,
Slava Mikhaylov
Abstract:
This study is a first, exploratory attempt to use quantitative semantics techniques and topological analysis to analyze systemic patterns arising in a complex political system. In particular, we use a rich data set covering all speeches and debates in the UK House of Commons between 1975 and 2014. By the use of dynamic topic modeling (DTM) and topological data analysis (TDA) we show that both memb…
▽ More
This study is a first, exploratory attempt to use quantitative semantics techniques and topological analysis to analyze systemic patterns arising in a complex political system. In particular, we use a rich data set covering all speeches and debates in the UK House of Commons between 1975 and 2014. By the use of dynamic topic modeling (DTM) and topological data analysis (TDA) we show that both members and parties feature specific roles within the system, consistent over time, and extract global patterns indicating levels of political cohesion. Our results provide a wide array of novel hypotheses about the complex dynamics of political systems, with valuable policy applications.
△ Less
Submitted 13 October, 2015;
originally announced October 2015.
