-
SecCityVR: Visualization and Collaborative Exploration of Software Vulnerabilities in Virtual Reality
Authors:
Dennis Wüppelman,
Enes Yigitbas
Abstract:
Security vulnerabilities in software systems represent significant risks as potential entry points for malicious attacks. Traditional dashboards that display the results of static analysis security testing often use 2D or 3D visualizations, which tend to lack the spatial details required to effectively reveal issues such as the propagation of vulnerabilities across the codebase or the appearance o…
▽ More
Security vulnerabilities in software systems represent significant risks as potential entry points for malicious attacks. Traditional dashboards that display the results of static analysis security testing often use 2D or 3D visualizations, which tend to lack the spatial details required to effectively reveal issues such as the propagation of vulnerabilities across the codebase or the appearance of concurrent vulnerabilities. Additionally, most reporting solutions only treat the analysis results as an artifact that can be reviewed or edited asynchronously by developers, limiting real-time, collaborative exploration. To the best of our knowledge, no VR-based approach exists for the visualization and interactive exploration of software security vulnerabilities. Addressing these challenges, the virtual reality (VR) environment SecCityVR was developed as a proof-of-concept implementation that employs the code city metaphor within VR to visualize software security vulnerabilities as colored building floors inside the surrounding virtual city. By integrating the application's call graph, vulnerabilities are contextualized within related software components. SecCityVR supports multi-user collaboration and interactive exploration. It provides explanations and mitigations for detected issues. A user study comparing SecCityVR with the traditional dashboard find-sec-bugs showed the VR approach provided a favorable experience, with higher usability, lower temporal demand, and significantly lower frustration despite having longer task completion times. This paper and its results contribute to the fields of collaborative and secure software engineering, as well as software visualization. It provides a new application of VR code cities to visualize security vulnerabilities, as well as a novel environment for security audits using collaborative and immersive technologies.
△ Less
Submitted 25 April, 2025;
originally announced April 2025.
-
Effects of Human Avatar Representation in Virtual Reality on Inter-Brain Connection
Authors:
Enes Yigitbas,
Christian Kaltschmidt
Abstract:
Increasing advances in affordable consumer hardware and accessible software frameworks are now bringing Virtual Reality (VR) to the masses. Especially collaborative VR applications where different people can work together are gaining momentum. In this context, human avatars and their representations are a crucial aspect of collaborative VR applications as they represent a digital twin of the end-u…
▽ More
Increasing advances in affordable consumer hardware and accessible software frameworks are now bringing Virtual Reality (VR) to the masses. Especially collaborative VR applications where different people can work together are gaining momentum. In this context, human avatars and their representations are a crucial aspect of collaborative VR applications as they represent a digital twin of the end-users and determine how one is perceived in a virtual environment. When it comes to the effect of avatar representation on the end-users of collaborative VR applications, so far mostly questionnaires have been used to assess the quality of avatar representations. A promising alternative to objectively measure the effect of avatar representation is the investigation of inter-brain connections during the usage of a collaborative VR application. However, the combination of immersive VR applications and inter-brain connections has not been fully researched yet. Thus, our work investigates how different human avatar representations (real (RL), full-body (FB), and head-hand (HH)) affect inter-brain connections. For this purpose, we have designed and conducted a hyperscanning study with eight pairs. The main results of our hyperscanning study show that the number of significant sensor pairs was the highest in the RL, medium in the FB, and lowest in the HH condition indicating that an avatar that looks more like a real human enables more significant sensor pairs to appear in an EEG analysis.
△ Less
Submitted 29 October, 2024;
originally announced October 2024.
-
Supporting Construction and Architectural Visualization through BIM and AR/VR: A Systematic Literature Review
Authors:
Enes Yigitbas,
Alexander Nowosad,
Gregor Engels
Abstract:
The Architecture, Engineering, Construction, and Facility Management (AEC/FM) industry deals with the design, construction, and operation of complex buildings. Today, Building Information Modeling (BIM) is used to represent information about a building in a single, non-redundant representation. Here, Augmented Reality (AR) and Virtual Reality (VR) can improve the visualization and interaction with…
▽ More
The Architecture, Engineering, Construction, and Facility Management (AEC/FM) industry deals with the design, construction, and operation of complex buildings. Today, Building Information Modeling (BIM) is used to represent information about a building in a single, non-redundant representation. Here, Augmented Reality (AR) and Virtual Reality (VR) can improve the visualization and interaction with the resulting model by augmenting the real world with information from the BIM model or allowing a user to immerse in a virtual world generated from the BIM model. This can improve the design, construction, and operation of buildings. While an increasing number of studies in HCI, construction, or engineering have shown the potential of using AR and VR technology together with BIM, often research remains focused on individual explorations and key design strategies. In addition to that, a systematic overview and discussion of recent works combining AR/VR with BIM are not yet fully covered. Therefore, this paper systematically reviews recent approaches combining AR/VR with BIM and categorizes the literature by the building's lifecycle phase while systematically describing relevant use cases. In total, 32 out of 447 papers between 2017 and 2022 were categorized. The categorization shows that most approaches focus on the construction phase and the use case of review and quality assurance. In the design phase, most approaches use VR, while in the construction and operation phases, AR is prevalent.
△ Less
Submitted 21 June, 2023;
originally announced June 2023.
-
Self-Adaptive Digital Assistance Systems for Work 4.0
Authors:
Enes Yigitbas,
Stefan Sauer,
Gregor Engels
Abstract:
In the era of digital transformation, new technological foundations and possibilities for collaboration, production as well as organization open up many opportunities to work differently in the future. The digitization of workflows results in new forms of working which is denoted by the term Work 4.0. In the context of Work 4.0, digital assistance systems play an important role as they give users…
▽ More
In the era of digital transformation, new technological foundations and possibilities for collaboration, production as well as organization open up many opportunities to work differently in the future. The digitization of workflows results in new forms of working which is denoted by the term Work 4.0. In the context of Work 4.0, digital assistance systems play an important role as they give users additional situation-specific information about a workflow or a product via displays, mobile devices such as tablets and smartphones, or data glasses. Furthermore, such digital assistance systems can be used to provide instructions and technical support in the working process as well as for training purposes. However, existing digital assistance systems are mostly created focusing on the "design for all" paradigm neglecting the situation-specific tasks, skills, preferences, or environments of an individual human worker. To overcome this issue, we present a monitoring and adaptation framework for supporting self-adaptive digital assistance systems for Work 4.0. Our framework supports context monitoring as well as UI adaptation for augmented (AR) and virtual reality (VR)-based digital assistance systems. The benefit of our framework is shown based on exemplary case studies from different domains, e.g. context-aware maintenance application in AR or warehouse management training in VR.
△ Less
Submitted 30 November, 2022;
originally announced November 2022.
-
Virtual Reality in University Teaching: Experiences from a Computer Science Seminar
Authors:
Enes Yigitbas
Abstract:
Due to the corona pandemic, numerous courses were held using digital solutions in order to be able to continue teaching. Conventional collaboration tools (Zoom, Big Blue Button, etc.) were used in particular to digitally map a synchronous session for teaching and learning purposes. While these conventional collaboration tools offer a solid basis for communication between learners and teachers, asp…
▽ More
Due to the corona pandemic, numerous courses were held using digital solutions in order to be able to continue teaching. Conventional collaboration tools (Zoom, Big Blue Button, etc.) were used in particular to digitally map a synchronous session for teaching and learning purposes. While these conventional collaboration tools offer a solid basis for communication between learners and teachers, aspects such as presence or a realistic type of interaction are neglected. In this work, we report on the experiences from a computer science seminar where virtual reality (VR) technology was used as an alternative solution for teaching and group work. The benefits of VR compared to conventional collaboration tools were examined using questionnaires and interviews with the participants. On the one hand, the results show the high potential of VR to increase the clarity and experienceability of learning content and to promote cooperation through social presence. On the other hand, the use of VR brings with it some technical and organizational difficulties that should be taken into account in the didactic implementation.
△ Less
Submitted 22 November, 2022;
originally announced November 2022.
-
Technical Report: Comparative Evaluation of AR-based, VR-based, and Traditional Basic Life Support Training
Authors:
Enes Yigitbas,
Sebastian Krois,
Timo Renzelmann,
Gregor Engels
Abstract:
Basic life support (BLS) is crucial in the emergency response system as sudden cardiac arrest is still a major cause of death worldwide. In the majority of cases, cardiac arrest is witnessed out-of-hospital where execution of BLS including resuscitation through by-standers gets indispensable. However, survival rates of cardiac arrest victims could majorly increase if BLS skills would be trained re…
▽ More
Basic life support (BLS) is crucial in the emergency response system as sudden cardiac arrest is still a major cause of death worldwide. In the majority of cases, cardiac arrest is witnessed out-of-hospital where execution of BLS including resuscitation through by-standers gets indispensable. However, survival rates of cardiac arrest victims could majorly increase if BLS skills would be trained regularly. In this context, technology-enhanced BLS training approaches utilizing augmented (AR) and virtual reality (VR) have been proposed in recent works. However, these approaches are not compliant with the medical BLS guidelines or focus only on specific steps of BLS training such as resuscitation. Furthermore, most of the existing training approaches do not focus on automated assessment to enhance efficiency and effectiveness through fine-grained real-time feedback. To overcome these issues, we present a novel AR- and VR-based training environment which supports a comprehensive BLS training compliant with the medical guidelines. Our training environment combines AR-/VR-based BLS training with an interactive haptic manikin that supports automated assessment, real-time feedback, and debriefing in an integrated environment. We have conducted a usability evaluation where we analyze the efficiency, effectiveness, and user satisfaction of BLS training based on our AR and VR environment against traditional BLS training. Results of the evaluation indicate that AR and VR technology have the potential to increase engagement in BLS training, improve high-quality resuscitation, and reduce the cognitive workload compared to traditional training.
△ Less
Submitted 7 July, 2022;
originally announced July 2022.
-
Collaborative Software Modeling in Virtual Reality
Authors:
Enes Yigitbas,
Simon Gorissen,
Nils Weidmann,
Gregor Engels
Abstract:
Modeling is a key activity in conceptual design and system design. Through collaborative modeling, end-users, stakeholders, experts, and entrepreneurs are able to create a shared understanding of a system representation. While the Unified Modeling Language (UML) is one of the major conceptual modeling languages in object-oriented software engineering, more and more concerns arise from the modeling…
▽ More
Modeling is a key activity in conceptual design and system design. Through collaborative modeling, end-users, stakeholders, experts, and entrepreneurs are able to create a shared understanding of a system representation. While the Unified Modeling Language (UML) is one of the major conceptual modeling languages in object-oriented software engineering, more and more concerns arise from the modeling quality of UML and its tool support. Among them, the limitation of the two-dimensional presentation of its notations and lack of natural collaborative modeling tools are reported to be significant. In this paper, we explore the potential of using Virtual Reality (VR) technology for collaborative UML software design by comparing it with classical collaborative software design using conventional devices (Desktop PC, Laptop). For this purpose, we have developed a VR modeling environment that offers a natural collaborative modeling experience for UML Class Diagrams. Based on a user study with 24 participants, we have compared collaborative VR modeling with conventional modeling with regard to efficiency, effectiveness, and user satisfaction. Results show that the use of VR has some disadvantages concerning efficiency and effectiveness, but the user's fun, the feeling of being in the same room with a remote collaborator, and the naturalness of collaboration were increased.
△ Less
Submitted 27 July, 2021;
originally announced July 2021.
-
VREUD -- An End-User Development Tool to Simplify the Creation of Interactive VR Scenes
Authors:
Enes Yigitbas,
Jonas Klauke,
Sebastian Gottschalk,
Gregor Engels
Abstract:
Recent advances in Virtual Reality (VR) technology and the increased availability of VR-equipped devices enable a wide range of consumer-oriented applications. For novice developers, however, creating interactive scenes for VR applications is a complex and cumbersome task that requires high technical knowledge which is often missing. This hinders the potential of enabling novices to create, modify…
▽ More
Recent advances in Virtual Reality (VR) technology and the increased availability of VR-equipped devices enable a wide range of consumer-oriented applications. For novice developers, however, creating interactive scenes for VR applications is a complex and cumbersome task that requires high technical knowledge which is often missing. This hinders the potential of enabling novices to create, modify, and execute their own interactive VR scenes. Although recent authoring tools for interactive VR scenes are promising, most of them focus on expert professionals as the target group and neglect the novices with low programming knowledge. To lower the entry barrier, we provide an open-source web-based End-User Development (EUD) tool, called VREUD, that supports the rapid construction and execution of interactive VR scenes. Concerning construction, VREUD enables the specification of the VR scene including interactions and tasks. Furthermore, VREUD supports the execution and immersive experience of the created interactive VR scenes on VR head-mounted displays. Based on a user study, we have analyzed the effectiveness, efficiency, and user satisfaction of VREUD which shows promising results to empower novices in creating their interactive VR scenes.
△ Less
Submitted 1 July, 2021;
originally announced July 2021.
-
Simplifying Robot Programming using Augmented Reality and End-User Development
Authors:
Enes Yigitbas,
Ivan Jovanovikj,
Gregor Engels
Abstract:
Robots are widespread across diverse application contexts. Teaching robots to perform tasks, in their respective contexts, demands a high domain and programming expertise. However, robot programming faces high entry barriers due to the complexity of robot programming itself. Even for experts robot programming is a cumbersome and error-prone task where faulty robot programs can be created, causing…
▽ More
Robots are widespread across diverse application contexts. Teaching robots to perform tasks, in their respective contexts, demands a high domain and programming expertise. However, robot programming faces high entry barriers due to the complexity of robot programming itself. Even for experts robot programming is a cumbersome and error-prone task where faulty robot programs can be created, causing damage when being executed on a real robot. To simplify the process of robot programming, we combine Augmented Reality (AR) with principles of end-user development. By combining them, the real environment is extended with useful virtual artifacts that can enable experts as well as non-professionals to perform complex robot programming tasks. Therefore, Simple Programming Environment in Augmented Reality with Enhanced Debugging (SPEARED) was developed as a prototype for an AR-assisted robot programming environment. SPEARED makes use of AR to project a robot as well as a programming environment onto the target working space. To evaluate our approach, expert interviews with domain experts from the area of industrial automation, robotics, and AR were performed. The experts agreed that SPEARED has the potential to enrich and ease current robot programming processes.
△ Less
Submitted 15 June, 2021;
originally announced June 2021.
-
Enhancing Human-in-the-Loop Adaptive Systems through Digital Twins and VR Interfaces
Authors:
Enes Yigitbas,
Kadiray Karakaya,
Ivan Jovanovikj,
Gregor Engels
Abstract:
Self-adaptation approaches usually rely on closed-loop controllers that avoid human intervention from adaptation. While such fully automated approaches have proven successful in many application domains, there are situations where human involvement in the adaptation process is beneficial or even necessary. For such "human-in-the-loop" adaptive systems, two major challenges, namely transparency and…
▽ More
Self-adaptation approaches usually rely on closed-loop controllers that avoid human intervention from adaptation. While such fully automated approaches have proven successful in many application domains, there are situations where human involvement in the adaptation process is beneficial or even necessary. For such "human-in-the-loop" adaptive systems, two major challenges, namely transparency and controllability, have to be addressed to include the human in the self-adaptation loop. Transparency means that relevant context information about the adaptive systems and its context is represented based on a digital twin enabling the human an immersive and realistic view. Concerning controllability, the decision-making and adaptation operations should be managed in a natural and interactive way. As existing human-in-the-loop adaptation approaches do not fully cover these aspects, we investigate alternative human-in-the-loop strategies by using a combination of digital twins and virtual reality (VR) interfaces. Based on the concept of the digital twin, we represent a self-adaptive system and its respective context in a virtual environment. With the help of a VR interface, we support an immersive and realistic human involvement in the self-adaptation loop by mirroring the physical entities of the real world to the VR interface. For integrating the human in the decision-making and adaptation process, we have implemented and analyzed two different human-in-the-loop strategies in VR: a procedural control where the human can control the decision making-process and adaptations through VR interactions (human-controlled) and a declarative control where the human specifies the goal state and the configuration is delegated to an AI planner (mixed-initiative). We illustrate and evaluate our approach based on an autonomic robot system that is accessible and controlled through a VR interface.
△ Less
Submitted 19 March, 2021;
originally announced March 2021.
-
Description Languages for Consistency Management Scenarios Based on Examples from the Industry Automation Domain
Authors:
Anthony Anjorin,
Enes Yigitbas,
Erhan Leblebici,
Andy Schürr,
Marius Lauder,
Martin Witte
Abstract:
To cope with the increasing complexity of developing and maintaining modern (software) systems, multiple abstractions (models) of the same system can be established and used to allow different domain experts to collaborate and contribute their respective expertise. This divide-and-conquer, model-based approach requires, however, support for a concurrent engineering process, i.e., providing a means…
▽ More
To cope with the increasing complexity of developing and maintaining modern (software) systems, multiple abstractions (models) of the same system can be established and used to allow different domain experts to collaborate and contribute their respective expertise. This divide-and-conquer, model-based approach requires, however, support for a concurrent engineering process, i.e., providing a means of checking, restoring, and ensuring the consistency of all involved and concurrently maintained models. The task of providing such support is often referred to as consistency management.
Although there exist various approaches to consistency management and numerous (industrial) case studies described in the literature on bidirectional transformations (bx), there is currently no uniform description of diverse but related industrial applications of model synchronisation and other forms of consistency management. This makes it challenging to detect similarities and differences related to requirements, constraints, applied techniques and tools. It is thus difficult to compare and transfer knowledge gained from (successful) projects to other bx approaches or even other bx tools for the same general approach.
In this paper, therefore, we propose a description language for envisioned scenarios in the problem domain of consistency management, as well as a complementary description language for solution strategies in terms of method fragments and method patterns in the solution domain of Model-Driven Engineering (MDE). Our work is inspired by previous research in the bx and MDE communities, and is also based on our collective experience from over ten years of investigating a series of application scenarios in the industry automation section together with Siemens AG as an industrial partner. [Abridged due to arXiv]
△ Less
Submitted 28 March, 2018;
originally announced March 2018.
