-
pyGANDALF -- An open-source, Geometric, ANimation, Directed, Algorithmic, Learning Framework for Computer Graphics
Authors:
John Petropoulos,
Manos Kamarianakis,
Antonis Protopsaltis,
George Papagiannakis
Abstract:
In computer graphics (CG) education, the challenge of finding modern, versatile tools is significant, particularly when integrating both legacy and advanced technologies. Traditional frameworks, often reliant on solid, yet outdated APIs like OpenGL, limit the exploration of cutting-edge graphics techniques. To address this, we introduce pyGANDALF, a unique, lightweight, open-source CG framework bu…
▽ More
In computer graphics (CG) education, the challenge of finding modern, versatile tools is significant, particularly when integrating both legacy and advanced technologies. Traditional frameworks, often reliant on solid, yet outdated APIs like OpenGL, limit the exploration of cutting-edge graphics techniques. To address this, we introduce pyGANDALF, a unique, lightweight, open-source CG framework built on three pillars: Entity-Component-System (ECS) architecture, Python programming, and WebGPU integration. This combination sets pyGANDALF apart by providing a streamlined ECS design with an editor layer, compatibility with WebGPU for state-of-the-art features like compute and ray tracing pipelines, and a programmer-friendly Python environment. The framework supports modern features, such as Physically Based Rendering (PBR) capabilities and integration with Universal Scene Description (USD) formats, making it suitable for both educational demonstrations and real-world applications. Evaluations by expert users confirmed that pyGANDALF effectively balances ease of use with advanced functionality, preparing students for contemporary CG development challenges.
△ Less
Submitted 25 September, 2024;
originally announced September 2024.
-
Decoupled Edge Physics algorithms for collaborative XR simulations
Authors:
George Kokiadis,
Antonis Protopsaltis,
Michalis Morfiadakis,
Nick Lydatakis,
George Papagiannakis
Abstract:
This work proposes a novel approach to transform any modern game engine pipeline, for optimized performance and enhanced user experiences in Extended Reality (XR) environments. Decoupling the physics engine from the game engine pipeline and using a client-server N-1 architecture creates a scalable solution, efficiently serving multiple graphics clients on Head-Mounted Displays (HMDs) with a single…
▽ More
This work proposes a novel approach to transform any modern game engine pipeline, for optimized performance and enhanced user experiences in Extended Reality (XR) environments. Decoupling the physics engine from the game engine pipeline and using a client-server N-1 architecture creates a scalable solution, efficiently serving multiple graphics clients on Head-Mounted Displays (HMDs) with a single physics engine on edge-cloud infrastructure. This approach ensures better synchronization in multiplayer scenarios without introducing overhead in single-player experiences, maintaining session continuity despite changes in user participation. Relocating the Physics Engine to an edge or cloud node reduces strain on local hardware, dedicating more resources to high-quality rendering and unlocking the full potential of untethered HMDs. We present four algorithms that decouple the physics engine, increasing frame rates and Quality of Experience (QoE) in VR simulations, supporting advanced interactions, numerous physics objects, and multi-user sessions with over 100 concurrent users. Incorporating a Geometric Algebra interpolator reduces inter-calls between dissected parts, maintaining QoE and easing network stress. Experimental validation, with more than 100 concurrent users, 10,000 physics objects, and softbody simulations, confirms the technical viability of the proposed architecture, showcasing transformative capabilities for more immersive and collaborative XR applications without compromising performance.
△ Less
Submitted 17 July, 2024;
originally announced July 2024.
-
UniSG^GA: A 3D scenegraph powered by Geometric Algebra unifying geometry, behavior and GNNs towards generative AI
Authors:
Manos Kamarianakis,
Antonis Protopsaltis,
Dimitris Angelis,
Paul Zikas,
Mike Kentros,
George Papagiannakis
Abstract:
This work presents the introduction of UniSG^GA, a novel integrated scenegraph structure, that to incorporates behavior and geometry data on a 3D scene. It is specifically designed to seamlessly integrate Graph Neural Networks (GNNs) and address the challenges associated with transforming a 3D scenegraph (3D-SG) during generative tasks. To effectively capture and preserve the topological relations…
▽ More
This work presents the introduction of UniSG^GA, a novel integrated scenegraph structure, that to incorporates behavior and geometry data on a 3D scene. It is specifically designed to seamlessly integrate Graph Neural Networks (GNNs) and address the challenges associated with transforming a 3D scenegraph (3D-SG) during generative tasks. To effectively capture and preserve the topological relationships between objects in a simplified way, within the graph representation, we propose UniSG^GA, that seamlessly integrates Geometric Algebra (GA) forms. This novel approach enhances the overall performance and capability of GNNs in handling generative and predictive tasks, opening up new possibilities and aiming to lay the foundation for further exploration and development of graph-based generative AI models that can effectively incorporate behavior data for enhanced scene generation and synthesis.
△ Less
Submitted 18 June, 2023;
originally announced June 2023.
-
AR-Assisted Surgical Care via 5G networks for First Aid Responders
Authors:
Manos Kamarianakis,
Antonis Protopsaltis,
George Papagiannakis
Abstract:
Surgeons should play a central role in disaster planning and management due to the overwhelming number of bodily injuries that are typically involved during most forms of disaster. In fact, various types of surgical procedures are performed by emergency medical teams after sudden-onset disasters, such as soft tissue wounds, orthopaedic traumas, abdominal surgeries, etc. HMD-based Augmented Reality…
▽ More
Surgeons should play a central role in disaster planning and management due to the overwhelming number of bodily injuries that are typically involved during most forms of disaster. In fact, various types of surgical procedures are performed by emergency medical teams after sudden-onset disasters, such as soft tissue wounds, orthopaedic traumas, abdominal surgeries, etc. HMD-based Augmented Reality (AR), using state-of-the-art hardware such as the Magic Leap or the Microsoft HoloLens, have long been foreseen as a key enabler for clinicians in surgical use cases, especially for procedures performed outside of the operating room.
This paper describes the Use Case (UC) "AR-assisted emergency surgical care", identified in the context of the 5G-EPICENTRE EU-funded project. Specifically, the UC will experiment with holographic AR technology for emergency medical surgery teams, by overlaying deformable medical models directly on top of the patient body parts, effectively enabling surgeons to see inside (visualizing bones, blood vessels, etc.) and perform surgical actions following step-by-step instructions. The goal is to combine the computational and data-intensive nature of AR and Computer Vision algorithms with upcoming 5G network architectures deployed for edge computing so as to satisfy real-time interaction requirements and provide an efficient and powerful platform for the pervasive promotion of such applications. By developing the necessary Virtual Network Functions (VNFs) to manage data-intensive services (e.g., prerendering, caching, compression) and by exploiting available network resources and Multi-access Edge Computing (MEC) support, provided by the 5G-EPICENTRE infrastructure, this UC aims to provide powerful AR-based tools, usable on site, to first-aid responders.
△ Less
Submitted 1 March, 2023;
originally announced March 2023.
-
Project Elements: A computational entity-component-system in a scene-graph pythonic framework, for a neural, geometric computer graphics curriculum
Authors:
George Papagiannakis,
Manos Kamarianakis,
Antonis Protopsaltis,
Dimitris Angelis,
Paul Zikas
Abstract:
We present the Elements project, a lightweight, open-source, computational science and computer graphics (CG) framework, tailored for educational needs, that offers, for the first time, the advantages of an Entity-Component-System (ECS) along with the rapid prototyping convenience of a Scenegraph-based pythonic framework. This novelty allows advances in the teaching of CG: from heterogeneous direc…
▽ More
We present the Elements project, a lightweight, open-source, computational science and computer graphics (CG) framework, tailored for educational needs, that offers, for the first time, the advantages of an Entity-Component-System (ECS) along with the rapid prototyping convenience of a Scenegraph-based pythonic framework. This novelty allows advances in the teaching of CG: from heterogeneous directed acyclic graphs and depth-first traversals, to animation, skinning, geometric algebra and shader-based components rendered via unique systems all the way to their representation as graph neural networks for 3D scientific visualization. Taking advantage of the unique ECS in a a Scenegraph underlying system, this project aims to bridge CG curricula and modern game engines (MGEs), that are based on the same approach but often present these notions in a black-box approach. It is designed to actively utilize software design patterns, under an extensible open-source approach. Although Elements provides a modern (i.e., shader-based as opposed to fixed-function OpenGL), simple to program approach with Jupyter notebooks and unit-tests, its CG pipeline is not black-box, exposing for teaching for the first time unique challenging scientific, visual and neural computing concepts.
△ Less
Submitted 27 June, 2023; v1 submitted 15 February, 2023;
originally announced February 2023.
-
MAGES 4.0: Accelerating the world's transition to VR training and democratizing the authoring of the medical metaverse
Authors:
Paul Zikas,
Antonis Protopsaltis,
Nick Lydatakis,
Mike Kentros,
Stratos Geronikolakis,
Steve Kateros,
Manos Kamarianakis,
Giannis Evangelou,
Achilleas Filippidis,
Eleni Grigoriou,
Dimitris Angelis,
Michail Tamiolakis,
Michael Dodis,
George Kokiadis,
John Petropoulos,
Maria Pateraki,
George Papagiannakis
Abstract:
In this work, we propose MAGES 4.0, a novel Software Development Kit (SDK) to accelerate the creation of collaborative medical training applications in VR/AR. Our solution is essentially a low-code metaverse authoring platform for developers to rapidly prototype high-fidelity and high-complexity medical simulations. MAGES breaks the authoring boundaries across extended reality, since networked par…
▽ More
In this work, we propose MAGES 4.0, a novel Software Development Kit (SDK) to accelerate the creation of collaborative medical training applications in VR/AR. Our solution is essentially a low-code metaverse authoring platform for developers to rapidly prototype high-fidelity and high-complexity medical simulations. MAGES breaks the authoring boundaries across extended reality, since networked participants can also collaborate using different virtual/augmented reality as well as mobile and desktop devices, in the same metaverse world. With MAGES we propose an upgrade to the outdated 150-year-old master-apprentice medical training model. Our platform incorporates, in a nutsell, the following novelties: a) 5G edge-cloud remote rendering and physics dissection layer, b) realistic real-time simulation of organic tissues as soft-bodies under 10ms, c) a highly realistic cutting and tearing algorithm, d) neural network assessment for user profiling and, e) a VR recorder to record and replay or debrief the training simulation from any perspective.
△ Less
Submitted 19 January, 2023; v1 submitted 19 September, 2022;
originally announced September 2022.
-
Progressive tearing and cutting of soft-bodies in high-performance virtual reality
Authors:
Manos Kamarianakis,
Antonis Protopsaltis,
Dimitris Angelis,
Michail Tamiolakis,
George Papagiannakis
Abstract:
We present an algorithm that allows a user within a virtual environment to perform real-time unconstrained cuts or consecutive tears, i.e., progressive, continuous fractures on a deformable rigged and soft-body mesh model in high-performance 10ms. In order to recreate realistic results for different physically-principled materials such as sponges, hard or soft tissues, we incorporate a novel soft-…
▽ More
We present an algorithm that allows a user within a virtual environment to perform real-time unconstrained cuts or consecutive tears, i.e., progressive, continuous fractures on a deformable rigged and soft-body mesh model in high-performance 10ms. In order to recreate realistic results for different physically-principled materials such as sponges, hard or soft tissues, we incorporate a novel soft-body deformation, via a particle system layered on-top of a linear-blend skinning model. Our framework allows the simulation of realistic, surgical-grade cuts and continuous tears, especially valuable in the context of medical VR training. In order to achieve high performance in VR, our algorithms are based on Euclidean geometric predicates on the rigged mesh, without requiring any specific model pre-processing. The contribution of this work lies on the fact that current frameworks supporting similar kinds of model tearing, either do not operate in high-performance real-time or only apply to predefined tears. The framework presented allows the user to freely cut or tear a 3D mesh model in a consecutive way, under 10ms, while preserving its soft-body behaviour and/or allowing further animation.
△ Less
Submitted 18 September, 2022;
originally announced September 2022.
-
Assessing unconstrained surgical cuttings in VR using CNNs
Authors:
Ilias Chrysovergis,
Manos Kamarianakis,
Mike Kentros,
Dimitris Angelis,
Antonis Protopsaltis,
George Papagiannakis
Abstract:
We present a Convolutional Neural Network (CNN) suitable to assess unconstrained surgical cuttings, trained on a dataset created with a data augmentation technique.
We present a Convolutional Neural Network (CNN) suitable to assess unconstrained surgical cuttings, trained on a dataset created with a data augmentation technique.
△ Less
Submitted 2 May, 2022;
originally announced May 2022.
-
Realistic soft-body tearing under 10ms in VR
Authors:
Manos Kamarianakis,
Antonis Protopsaltis,
Michail Tamiolakis,
George Papagiannakis
Abstract:
We present a novel integration of a real-time continuous tearing algorithm for 3D meshes in VR, suitable for devices of low CPU/GPU specifications, along with a suitable particle decomposition that allows soft-body deformations on both the original and the torn model.
We present a novel integration of a real-time continuous tearing algorithm for 3D meshes in VR, suitable for devices of low CPU/GPU specifications, along with a suitable particle decomposition that allows soft-body deformations on both the original and the torn model.
△ Less
Submitted 2 May, 2022;
originally announced May 2022.
-
"Deep Cut": An all-in-one Geometric Algorithm for Unconstrained Cut, Tear and Drill of Soft-bodies in Mobile VR
Authors:
Manos Kamarianakis,
Nick Lydatakis,
Antonis Protopsaltis,
John Petropoulos,
Michail Tamiolakis,
Paul Zikas,
George Papagiannakis
Abstract:
In this work, we present an integrated geometric framework: "deep- cut" that enables for the first time a user to geometrically and algorithmically cut, tear and drill the surface of a skinned model without prior constraints, layered on top of a custom soft body mesh deformation algorithm. Both layered algorithms in this frame- work yield real-time results and are amenable for mobile Virtual Reali…
▽ More
In this work, we present an integrated geometric framework: "deep- cut" that enables for the first time a user to geometrically and algorithmically cut, tear and drill the surface of a skinned model without prior constraints, layered on top of a custom soft body mesh deformation algorithm. Both layered algorithms in this frame- work yield real-time results and are amenable for mobile Virtual Reality, in order to be utilized in a variety of interactive application scenarios. Our framework dramatically improves real-time user experience and task performance in VR, without pre-calculated or artificially designed cuts, tears, drills or surface deformations via predefined rigged animations, which is the current state-of-the-art in mobile VR. Thus our framework improves user experience on one hand, on the other hand saves both time and costs from expensive, manual, labour-intensive design pre-calculation stages.
△ Less
Submitted 11 August, 2021;
originally announced August 2021.
-
A computational medical XR discipline
Authors:
George Papagiannakis,
Walter Greenleaf,
Michael Cole,
Mark Zhang,
Rabi Datta,
Mathias Delahaye,
Eleni Grigoriou,
Manos Kamarianakis,
Antonis Protopsaltis,
Philippe Bijlenga,
Nadia Magnenat-Thalmann,
Eleftherios Tsiridis,
Eustathios Kenanidis,
Kyriakos Vamvakidis,
Ioannis Koutelidakis,
Oliver A Kannape
Abstract:
Computational Medical Extended Reality (CMXR), brings together life sciences and neuroscience with mathematics, engineering and computer science. It unifies computational science (scientific computing) with intelligent extended reality and spatial computing for the medical field. It significantly differs from previous "Clinical XR" or "Medical XR" terms, as it is focusing on how to integrate compu…
▽ More
Computational Medical Extended Reality (CMXR), brings together life sciences and neuroscience with mathematics, engineering and computer science. It unifies computational science (scientific computing) with intelligent extended reality and spatial computing for the medical field. It significantly differs from previous "Clinical XR" or "Medical XR" terms, as it is focusing on how to integrate computational methods from neural simulation to computational geometry, computational vision and computer graphics with deep learning models to solve specific hard problems in medicine and neuroscience: from low/no-code/genAI authoring platforms to deep learning XR systems for training, planning, operative navigation, therapy and rehabilitation.
△ Less
Submitted 1 July, 2024; v1 submitted 9 August, 2021;
originally announced August 2021.
