-
Psychometrics in Behavioral Software Engineering: A Methodological Introduction with Guidelines
Authors:
Daniel Graziotin,
Per Lenberg,
Robert Feldt,
Stefan Wagner
Abstract:
A meaningful and deep understanding of the human aspects of software engineering (SE) requires psychological constructs to be considered. Psychology theory can facilitate the systematic and sound development as well as the adoption of instruments (e.g., psychological tests, questionnaires) to assess these constructs. In particular, to ensure high quality, the psychometric properties of instruments…
▽ More
A meaningful and deep understanding of the human aspects of software engineering (SE) requires psychological constructs to be considered. Psychology theory can facilitate the systematic and sound development as well as the adoption of instruments (e.g., psychological tests, questionnaires) to assess these constructs. In particular, to ensure high quality, the psychometric properties of instruments need evaluation. In this paper, we provide an introduction to psychometric theory for the evaluation of measurement instruments for SE researchers. We present guidelines that enable using existing instruments and developing new ones adequately. We conducted a comprehensive review of the psychology literature framed by the Standards for Educational and Psychological Testing. We detail activities used when operationalizing new psychological constructs, such as item pooling, item review, pilot testing, item analysis, factor analysis, statistical property of items, reliability, validity, and fairness in testing and test bias. We provide an openly available example of a psychometric evaluation based on our guideline. We hope to encourage a culture change in SE research towards the adoption of established methods from psychology. To improve the quality of behavioral research in SE, studies focusing on introducing, validating, and then using psychometric instruments need to be more common.
△ Less
Submitted 8 June, 2021; v1 submitted 20 May, 2020;
originally announced May 2020.
-
Agility is responsiveness to change: An essential definition
Authors:
Lucas Gren,
Per Lenberg
Abstract:
There is some ambiguity of what agile means in both research and practice. Authors have suggested a diversity of different definitions, through which it is difficult to interpret what agile really is. The concept, however, exists in its implementation through agile practices. In this vision paper, we argue that adopting an agile approach boils down to being more responsive to change. To support th…
▽ More
There is some ambiguity of what agile means in both research and practice. Authors have suggested a diversity of different definitions, through which it is difficult to interpret what agile really is. The concept, however, exists in its implementation through agile practices. In this vision paper, we argue that adopting an agile approach boils down to being more responsive to change. To support this claim, we relate agile principles, practices, the agile manifesto, and our own experiences to this core definition. We envision that agile transformations would be, and are, much easier using this definition and contextualizing its implications.
△ Less
Submitted 12 February, 2020; v1 submitted 22 September, 2019;
originally announced September 2019.
-
The Importance of Conflict Resolution Techniques in Autonomous Agile Teams
Authors:
Lucas Gren,
Per Lenberg
Abstract:
Today, software companies usually organize their work in teams. Social science research on team development has shown that for a team to reach a productive and autonomous stage, it has to be able to manage internal conflicts and disagreements efficiently. To better facilitate the team development process, we argue that software engineers' needs additional training in negotiation skills and conflic…
▽ More
Today, software companies usually organize their work in teams. Social science research on team development has shown that for a team to reach a productive and autonomous stage, it has to be able to manage internal conflicts and disagreements efficiently. To better facilitate the team development process, we argue that software engineers' needs additional training in negotiation skills and conflict resolution. In this position paper, we outline ideas for what aspects to consider in such training. As an example, we argue that a majority of the conflicts originate from team-level factors and that they, therefore, should be managed on the team-level instead of in relation to dyads.
△ Less
Submitted 5 April, 2019;
originally announced April 2019.
-
What software engineering can learn from research on affect in social psychology
Authors:
Lucas Gren,
Per Lenberg,
Karolina Ljungberg
Abstract:
Social psychology researchers have, traditionally, focused on the construct of thinking rather than on feeling. Since the beginning of the 21st century, social science researchers have, however, increasingly explored the effects of affect. Their work has repeatedly recognized that affects play a crucial role in determining people's behavior. In this short paper, we argue that software engineering…
▽ More
Social psychology researchers have, traditionally, focused on the construct of thinking rather than on feeling. Since the beginning of the 21st century, social science researchers have, however, increasingly explored the effects of affect. Their work has repeatedly recognized that affects play a crucial role in determining people's behavior. In this short paper, we argue that software engineering studies on affect would benefit from using more of the knowledge that social science researchers have acquired. Without accounting for their findings, we risk re-inventing the wheel. Also, without a profound understanding of the complex interplay between social context and affect, we risk creating overly simplistic solutions that might have considerable long-term adverse effects for software engineers.
△ Less
Submitted 18 March, 2019;
originally announced March 2019.
-
Misaligned Values in Software Engineering Organizations
Authors:
Per Lenberg,
Robert Feldt,
Lars Göran Wallgren Tengberg
Abstract:
The values of software organizations are crucial for achieving high performance; in particular, agile development approaches emphasize their importance. Researchers have thus far often assumed that a specific set of values, compatible with the development methodologies, must be adopted homogeneously throughout the company. It is not clear, however, to what extent such assumptions are accurate.
P…
▽ More
The values of software organizations are crucial for achieving high performance; in particular, agile development approaches emphasize their importance. Researchers have thus far often assumed that a specific set of values, compatible with the development methodologies, must be adopted homogeneously throughout the company. It is not clear, however, to what extent such assumptions are accurate.
Preliminary findings have highlighted the misalignment of values between groups as a source of problems when engineers discuss their challenges. Therefore, in this study, we examine how discrepancies in values between groups affect software companies' performance.
To meet our objectives, we chose a mixed method research design. First, we collected qualitative data by interviewing fourteen (\textit{N} = 14) employees working in four different organizations and processed it using thematic analysis. We then surveyed seven organizations (\textit{N} = 184). Our analysis indicated that value misalignment between groups is related to organizational performance. The aligned companies were more effective, more satisfied, had higher trust, and fewer conflicts.
Our efforts provide encouraging findings in a critical software engineering research area. They can help to explain why some companies are more efficient than others and, thus, point the way to interventions to address organizational challenges.
△ Less
Submitted 15 October, 2018; v1 submitted 14 October, 2018;
originally announced October 2018.
-
A Method to Assess and Argue for Practical Significance in Software Engineering
Authors:
Richard Torkar,
Carlo A. Furia,
Robert Feldt,
Francisco Gomes de Oliveira Neto,
Lucas Gren,
Per Lenberg,
Neil A. Ernst
Abstract:
A key goal of empirical research in software engineering is to assess practical significance, which answers whether the observed effects of some compared treatments show a relevant difference in practice in realistic scenarios. Even though plenty of standard techniques exist to assess statistical significance, connecting it to practical significance is not straightforward or routinely done; indeed…
▽ More
A key goal of empirical research in software engineering is to assess practical significance, which answers whether the observed effects of some compared treatments show a relevant difference in practice in realistic scenarios. Even though plenty of standard techniques exist to assess statistical significance, connecting it to practical significance is not straightforward or routinely done; indeed, only a few empirical studies in software engineering assess practical significance in a principled and systematic way.
In this paper, we argue that Bayesian data analysis provides suitable tools to assess practical significance rigorously. We demonstrate our claims in a case study comparing different test techniques. The case study's data was previously analyzed (Afzal et al., 2015) using standard techniques focusing on statistical significance. Here, we build a multilevel model of the same data, which we fit and validate using Bayesian techniques. Our method is to apply cumulative prospect theory on top of the statistical model to quantitatively connect our statistical analysis output to a practically meaningful context. This is then the basis both for assessing and arguing for practical significance.
Our study demonstrates that Bayesian analysis provides a technically rigorous yet practical framework for empirical software engineering. A substantial side effect is that any uncertainty in the underlying data will be propagated through the statistical model, and its effects on practical significance are made clear.
Thus, in combination with cumulative prospect theory, Bayesian analysis supports seamlessly assessing practical significance in an empirical software engineering context, thus potentially clarifying and extending the relevance of research for practitioners.
△ Less
Submitted 25 December, 2020; v1 submitted 26 September, 2018;
originally announced September 2018.
-
Psychological Safety and Norm Clarity in Software Engineering Teams
Authors:
Per Lenberg,
Robert Feldt
Abstract:
In the software engineering industry today, companies primarily conduct their work in teams. To increase organizational productivity, it is thus crucial to know the factors that affect team effectiveness. Two team-related concepts that have gained prominence lately are psychological safety and team norms. Still, few studies exist that explore these in a software engineering context.
Therefore, w…
▽ More
In the software engineering industry today, companies primarily conduct their work in teams. To increase organizational productivity, it is thus crucial to know the factors that affect team effectiveness. Two team-related concepts that have gained prominence lately are psychological safety and team norms. Still, few studies exist that explore these in a software engineering context.
Therefore, with the aim of extending the knowledge of these concepts, we examined if psychological safety and team norm clarity associate positively with software developers' self-assessed team performance and job satisfaction, two important elements of effectiveness.
We collected industry survey data from practitioners (N = 217) in 38 development teams working for five different organizations. The result of multiple linear regression analyses indicates that both psychological safety and team norm clarity predict team members' self-assessed performance and job satisfaction. The findings also suggest that clarity of norms is a stronger (30\% and 71\% stronger, respectively) predictor than psychological safety.
This research highlights the need to examine, in more detail, the relationship between social norms and software development. The findings of this study could serve as an empirical baseline for such, future work.
△ Less
Submitted 5 February, 2018;
originally announced February 2018.
-
Qualitative software engineering research -- reflections and guidelines
Authors:
Per Lenberg,
Robert Feldt,
Lucas Gren,
Lars Göran Wallgren Tengberg,
Inga Tidefors,
Daniel Graziotin
Abstract:
Researchers are increasingly recognizing the importance of human aspects in software development. Since qualitative methods are used to explore human behavior in-depth, we believe that studies using such methods will become more common. Existing qualitative software engineering guidelines do not cover the full breadth of qualitative methods and the knowledge on how to use them like in social scien…
▽ More
Researchers are increasingly recognizing the importance of human aspects in software development. Since qualitative methods are used to explore human behavior in-depth, we believe that studies using such methods will become more common. Existing qualitative software engineering guidelines do not cover the full breadth of qualitative methods and the knowledge on how to use them like in social sciences.
The purpose of this study was to extend the software engineering community's current body of knowledge regarding available qualitative methods and their quality assurance frameworks, and to provide recommendations and guidelines for their use. With the support of an epistemological argument and a survey of the literature, we suggest that future research would benefit from (1) utilizing a broader set of research methods, (2) more strongly emphasizing reflexivity, and (3) employing qualitative guidelines and quality criteria.
We present an overview of three qualitative methods commonly used in social sciences but rarely seen in software engineering research, namely interpretative phenomenological analysis, narrative analysis, and discourse analysis. Furthermore, we discuss the meaning of reflexivity in relation to the software engineering context and suggest means of fostering it.
Our paper will help software engineering researchers better select and then guide the application of a broader set of qualitative research methods.
△ Less
Submitted 10 July, 2023; v1 submitted 22 December, 2017;
originally announced December 2017.
-
Software Engineers' Attitudes Towards Organizational Change - an Industrial Case Study
Authors:
Per Lenberg,
Lars Göran Wallgren,
Robert Feldt
Abstract:
In order to cope with a complex and changing environment, industries seek to find new and more efficient ways to conduct their business. According to previous research, many of these change efforts fail to achieve their intended aims. Researchers have therefore sought to identify factors that increase the likelihood of success and found that employees' attitude towards change is one of the most cr…
▽ More
In order to cope with a complex and changing environment, industries seek to find new and more efficient ways to conduct their business. According to previous research, many of these change efforts fail to achieve their intended aims. Researchers have therefore sought to identify factors that increase the likelihood of success and found that employees' attitude towards change is one of the most critical.
The ability to manage change is especially important in software engineering organizations, where rapid changes in influential technologies and constantly evolving methodologies create a turbulent environment. Nevertheless, to the best of our knowledge, no studies exist that explore attitude towards change in a software engineering organization.
In this case study, we have used industry data to examine if the knowledge about the intended change outcome, the understanding of the need for change, and the feelings of participation affect software engineers' openness to change and readiness for change respectively, two commonly used attitude constructs. The result of two separate multiple regression analysis showed that openness to change is predicted by all three concepts, while readiness for change is predicted by need for change and participation. In addition, our research also provides a hierarchy with respect to the three predictive constructs' degree of impact.
Ultimately, our result can help managers in software engineering organizations to increase the likelihood of successfully implementing change initiatives that result in a changed organizational behavior. However, the first-order models we propose are to be recognized as early approximations that captures the most significant effects and should therefore, in future research, be extended to include additional software engineering unique factors.
△ Less
Submitted 21 January, 2016;
originally announced January 2016.
