Understanding Users’ Affective States During Issue Resolution in Open Source Software Projects

In this work, we explore users’ affective states during issue resolution in open source software (OSS) projects, and study the correlations between these states and their future retention. While many studies focus on users’ sentiment polarities, few have delved into the complex cognitive processes underlying issue resolution. This work proposes a nine-state model that describes users’ affective states by combining sentiment polarities with emotion. With this model, we perform affective state estimation from users’ issue comments with the state-of-the-art large language models (LLMs). Experimental results on existing benchmarks suggest that LLMs are effective in estimating the affective states from issue comments, and the finetuned RoBERTa-based estimator achieves the best performance with a 69.01% accuracy. With the estimator, we analyze the dynamics of users’ affective states during issue resolution in 114 real-world OSS projects and find significant differences between popular projects under active maintenance and inactive projects. Moreover, we perform regression analysis and find significant correlations between users’ affective states during issue resolution and their future retention and activeness in participating in issue discussions and making contributions. Compared with existing factors, we improve the average goodness-of-fit of regression models by 42.56% and 12.08%, for user retention and future activeness, after extending the factors to include users’ affective states, respectively. The results suggest that experiencing confusion and frustration is negatively associated with a user’s future retention, while being engaged corresponds to a higher likelihood of future participation. Our study shows the importance of maintaining an engaged and positive atmosphere in OSS teams.