Immersive Experiences and XR: A Game Engine or Multimedia Streaming Problem?
Authors:
Simon N. B. Gunkel,
Emmanouil Potetsianakis,
Tessa E. Klunder,
Alexander Toet,
Sylvie S. Dijkstra-Soudarissanane
Abstract:
Recent improvements in Extended Reality (XR) technology created an increase in XR products and solutions in the industry, while raising new requirements for new or improved architectural concepts. This need can be particularly complex as XR applications often relate both to 3D geometric rendering and multimedia paradigms. This paper outlines the main concepts relevant to XR, both from a game engin…
▽ More
Recent improvements in Extended Reality (XR) technology created an increase in XR products and solutions in the industry, while raising new requirements for new or improved architectural concepts. This need can be particularly complex as XR applications often relate both to 3D geometric rendering and multimedia paradigms. This paper outlines the main concepts relevant to XR, both from a game engineering and multimedia streaming system perspective. XR requires new metadata and media/game orchestration to allow complex interaction between users, objects, and (volumetric) multimedia content, which also results in new requirements on synchronisation (i.e., for global object state and positioning). Furthermore, the paper presents the functional blocks needed in new XR system architectures and how they will glue both (game and media) spaces together. The discussion of functional components and architecture relates to the ongoing activities in relevant standardisation bodies like Khronos, MPEG, and 3GPP. To make XR successful on the long term, the industry needs to agree on interoperable solutions and how to merge both game and media paradigms to allow complex multi-user XR applications.
△ Less
Submitted 14 January, 2022;
originally announced January 2022.
MPEG Media Enablers For Richer XR Experiences
Authors:
Emmanuel Thomas,
Emmanouil Potetsianakis,
Thomas Stockhammer,
Imed Bouazizi,
Mary-Luc Champel
Abstract:
With the advent of immersive media applications, the requirements for the representation and the consumption of such content has dramatically increased. The ever-increasing size of the media asset combined with the stringent motion-to-photon latency requirement makes the equation of a high quality of experience for XR streaming services difficult to solve. The MPEG-I standards aim at facilitating…
▽ More
With the advent of immersive media applications, the requirements for the representation and the consumption of such content has dramatically increased. The ever-increasing size of the media asset combined with the stringent motion-to-photon latency requirement makes the equation of a high quality of experience for XR streaming services difficult to solve. The MPEG-I standards aim at facilitating the wide deployment of immersive applications. This paper describes part 13, Video Decoding Interface, and part 14, Scene Description for MPEG Media of MPEG-I which address decoder management and the virtual scene composition, respectively. These new parts intend to make complex media rendering operations and hardware resources management hidden from the application, hence lowering the barrier for XR application to become mainstream and accessible to XR experience developers and designers. Both parts are expected to be published by ISO at the end of 2021.
△ Less
Submitted 9 October, 2020;
originally announced October 2020.
