Research

My research is motivated by the choices individuals face upon the introduction of new technologies and products and by the aggregate dynamics accruing from these disruptions. I am interested in the digitization process and in how its outcomes are shaping the ways individuals interact with technology and with each other. I am equally interested in the methods that aim at identifying causal effects, namely in randomized experiments. In particular, I am interested randomized experiments in networked environments, in which interactions among treatment units create additional challenges. I am also interested in the combination of randomized experiments with machine learning methods to identify heterogeneous treatment effects and to achieve optimal treatment assignment to each treatment unit.

Work in Progress

Could Reward Uncertainty Encourage Social Referrals? Evidence from a Large-scale Field Experiment, with Andy Tao Li and Ting Li

Abstract: The integration of uncertainty into referral reward structures by various platforms has opened new avenues for enhancing referral outcomes, a potential yet to be fully understood. To fill this gap, our research conducted a two-month randomized experiment with over 160,000 users to investigate the causal effects of uncertain rewards on social referral behaviors and to identify the mechanisms behind these effects. The result distinguishes between the effects of uncertainty on the sender’s rewards versus the recipient’s rewards. Our findings reveal that uncertainty in the sender’s reward leads to a 14% increase in the number of referrals, and recipients of these invitations are more likely to engage in additional referring activities. In contrast, uncertainty in the recipient’s reward results in a 48% reduction in referrals, with invited recipients displaying a decreased inclination to make further referrals. Using additional online experiments, we identified distinct mechanisms driving these asymmetric effects: on the sender’s side, the introduction of uncertainty alleviates feelings of guilt and prompts more strategic selection of recipients, resulting in higher rates of referral sharing and acceptance, and consequently, an increased total number of referrals. Conversely, on the recipient’s side, the negative effects of uncertainty are primarily due to a reduced perception of fairness and social pressure, which discourage recipients from engaging in the referral process. Our study sheds light on the complex dynamics of reward uncertainty in referral programs, offering novel insights into how it can be optimized to foster more engaged referral networks.

Generative AI and Student Performance: Evidence from a Large-Scale Intervention in a Digital Business Course, with Dimitrios Tsekouras

Abstract: The emergence of generative artificial intelligence (AI), especially Large Language Models (LLM) such as ChatGPT, has created the potential for disrupting established practices in multiple areas such as the labor market, healthcare, and education. In the context of education, the use of AI tools based on LLMs can transform the way students learn, with, e.g., the implementation of virtual tutors helping students understand new concepts or helping students draft essays and improve their writing skills. In this paper, we examine the impact of ChatGPT on student performance in the context of a large course on Digital Business at a European business school. We use data from two editions of the course. The first edition of the course was taught before the introduction of ChatGPT in November 2022, while for the second edition, the use of ChatGPT was made mandatory for the first essay and optional for the second. We assess the impact of the use of ChatGPT by comparing essay grades across the two cohorts of students. We find that the use of ChatGPT has a negative impact in all rubric attributes for the first essay, except for writing quality. For the second essay, the use of ChatGPT has a positive impact on writing quality but no impact on the other rubric attributes. We also find that weaker students benefit the most from the use of ChatGPT w.r.t. total grade, evidence, and relevance.

Competition and Learning: The Impact of Gamified Competitive Structures on User Engagement in the Educational Online Platforms, with Agnieszka Kloc and Ting Li

Abstract: We study the impact of competition intensity on user engagement in an online learning platform environment. Our study focuses on how the specific design of a competitive feature — leaderboard— affects student engagement with learning. We run a randomized field experiment with almost four thousand high school level students preparing for their final exams and observe that, counterintuitively, the less intensive the competition is the more engaged the users become. More specifically, students engage more when the scores of the group they compete against are more spread out and when they are farther away from both their upward and downward competitors. Additionally, students with low confidence in their knowledge level become more active when the upward competitor is farther away, while the competitively inclined users decrease their activity when the same happens. Our results imply that there is no one-competition-fit-all, and the implementation of a competitive feature should be crafted with attention to the intended competitive conditions and the type of user that is going to be participating in the competitive environment.