-
Connectivity for AI enabled cities -- A field survey based study of emerging economies
Authors:
Dibakar Das,
Jyotsna Bapat,
Angeliki Katsenou,
Sushmita Shrestha
Abstract:
The impact of Artificial Intelligence (AI) is transforming various aspects of urban life, including, governance, policy and planning, healthcare, sustainability, economics, entrepreneurship, etc. Although AI immense potential for positively impacting urban living, its success depends on overcoming significant challenges, particularly in telecommunications infrastructure. Smart city applications, s…
▽ More
The impact of Artificial Intelligence (AI) is transforming various aspects of urban life, including, governance, policy and planning, healthcare, sustainability, economics, entrepreneurship, etc. Although AI immense potential for positively impacting urban living, its success depends on overcoming significant challenges, particularly in telecommunications infrastructure. Smart city applications, such as, federated learning, Internet of Things (IoT), and online financial services, require reliable Quality of Service (QoS) from telecommunications networks to ensure effective information transfer. However, with over three billion people underserved or lacking access to internet, many of these AI-driven applications are at risk of either remaining underutilized or failing altogether. Furthermore, many IoT and video-based applications in densely populated urban areas require high-quality connectivity. This paper explores these issues, focusing on the challenges that need to be mitigated to make AI succeed in emerging countries, where more than 80% of the world population resides and urban migration grows. In this context, an overview of a case study conducted in Kathmandu, Nepal, highlights citizens' aspirations for affordable, high-quality internet-based services. The findings underscore the pressing need for advanced telecommunication networks to meet diverse user requirements while addressing investment and infrastructure gaps. This discussion provides insights into bridging the digital divide and enabling AI's transformative potential in urban areas.
△ Less
Submitted 16 January, 2025;
originally announced January 2025.
-
Connecting the Unconnected: A DT Case Study of Nomadic Nodes Deployment in Nepal
Authors:
Ioannis Mavromatis,
Angeliki Katsenou,
Klodian Bardhi,
Evangelos Xenos,
Dimitra Simeonidou
Abstract:
This paper addresses the challenge of robust cellular connectivity in dense, underdeveloped urban environments, specifically focusing on Kathmandu, Nepal. As cities grow, existing cellular infrastructure struggles to meet the demand for reliable, high-throughput, and low-latency communication services. The lack of investment in new technologies and the intricacies of the cities' landscape pose eve…
▽ More
This paper addresses the challenge of robust cellular connectivity in dense, underdeveloped urban environments, specifically focusing on Kathmandu, Nepal. As cities grow, existing cellular infrastructure struggles to meet the demand for reliable, high-throughput, and low-latency communication services. The lack of investment in new technologies and the intricacies of the cities' landscape pose even more difficulties for robust connectivity. This work addresses the above challenges in a cost-effective and flexible way. We investigate the deployment of LTE Nomadic Nodes (NNs) at scale in order to enhance network capacity and coverage. Utilising a Digital Twin (DT), we simulate and optimise NN placement, considering Kathmandu's physical and environmental characteristics. Our approach leverages the DRIVE DT framework, which enables the systemic evaluation of various network configurations and user mobility scenarios. The results demonstrate that NNs significantly improve signal strength and expected user datarates, presenting a viable solution for enhancing urban cellular connectivity.
△ Less
Submitted 14 November, 2024;
originally announced November 2024.
-
Decoding Complexity-Rate-Quality Pareto-Front for Adaptive VVC Streaming
Authors:
Angeliki Katsenou,
Vignesh V Menon,
Adam Wieckowski,
Benjamin Bross,
Detlev Marpe
Abstract:
Pareto-front optimization is crucial for addressing the multi-objective challenges in video streaming, enabling the identification of optimal trade-offs between conflicting goals such as bitrate, video quality, and decoding complexity. This paper explores the construction of efficient bitrate ladders for adaptive Versatile Video Coding (VVC) streaming, focusing on optimizing these trade-offs. We i…
▽ More
Pareto-front optimization is crucial for addressing the multi-objective challenges in video streaming, enabling the identification of optimal trade-offs between conflicting goals such as bitrate, video quality, and decoding complexity. This paper explores the construction of efficient bitrate ladders for adaptive Versatile Video Coding (VVC) streaming, focusing on optimizing these trade-offs. We investigate various ladder construction methods based on Pareto-front optimization, including exhaustive Rate-Quality and fixed ladder approaches. We propose a joint decoding time-rate-quality Pareto-front, providing a comprehensive framework to balance bitrate, decoding time, and video quality in video streaming. This allows streaming services to tailor their encoding strategies to meet specific requirements, prioritizing low decoding latency, bandwidth efficiency, or a balanced approach, thus enhancing the overall user experience. The experimental results confirm and demonstrate these opportunities for navigating the decoding time-rate-quality space to support various use cases. For example, when prioritizing low decoding latency, the proposed method achieves decoding time reduction of 14.86% while providing Bjontegaard delta rate savings of 4.65% and 0.32dB improvement in the eXtended Peak Signal-to-Noise Ratio (XPSNR)-Rate domain over the traditional fixed ladder solution.
△ Less
Submitted 27 September, 2024;
originally announced September 2024.
-
ReLaX-VQA: Residual Fragment and Layer Stack Extraction for Enhancing Video Quality Assessment
Authors:
Xinyi Wang,
Angeliki Katsenou,
David Bull
Abstract:
With the rapid growth of User-Generated Content (UGC) exchanged between users and sharing platforms, the need for video quality assessment in the wild is increasingly evident. UGC is typically acquired using consumer devices and undergoes multiple rounds of compression (transcoding) before reaching the end user. Therefore, traditional quality metrics that employ the original content as a reference…
▽ More
With the rapid growth of User-Generated Content (UGC) exchanged between users and sharing platforms, the need for video quality assessment in the wild is increasingly evident. UGC is typically acquired using consumer devices and undergoes multiple rounds of compression (transcoding) before reaching the end user. Therefore, traditional quality metrics that employ the original content as a reference are not suitable. In this paper, we propose ReLaX-VQA, a novel No-Reference Video Quality Assessment (NR-VQA) model that aims to address the challenges of evaluating the quality of diverse video content without reference to the original uncompressed videos. ReLaX-VQA uses frame differences to select spatio-temporal fragments intelligently together with different expressions of spatial features associated with the sampled frames. These are then used to better capture spatial and temporal variabilities in the quality of neighbouring frames. Furthermore, the model enhances abstraction by employing layer-stacking techniques in deep neural network features from Residual Networks and Vision Transformers. Extensive testing across four UGC datasets demonstrates that ReLaX-VQA consistently outperforms existing NR-VQA methods, achieving an average SRCC of 0.8658 and PLCC of 0.8873. Open-source code and trained models that will facilitate further research and applications of NR-VQA can be found at https://github.com/xinyiW915/ReLaX-VQA.
△ Less
Submitted 12 March, 2025; v1 submitted 16 July, 2024;
originally announced July 2024.
-
Rate-Quality or Energy-Quality Pareto Fronts for Adaptive Video Streaming?
Authors:
Angeliki Katsenou,
Xinyi Wang,
Daniel Schien,
David Bull
Abstract:
Adaptive video streaming is a key enabler for optimising the delivery of offline encoded video content. The research focus to date has been on optimisation, based solely on rate-quality curves. This paper adds an additional dimension, the energy expenditure, and explores construction of bitrate ladders based on decoding energy-quality curves rather than the conventional rate-quality curves. Pareto…
▽ More
Adaptive video streaming is a key enabler for optimising the delivery of offline encoded video content. The research focus to date has been on optimisation, based solely on rate-quality curves. This paper adds an additional dimension, the energy expenditure, and explores construction of bitrate ladders based on decoding energy-quality curves rather than the conventional rate-quality curves. Pareto fronts are extracted from the rate-quality and energy-quality spaces to select optimal points. Bitrate ladders are constructed from these points using conventional rate-based rules together with a novel quality-based approach. Evaluation on a subset of YouTube-UGC videos encoded with x.265 shows that the energy-quality ladders reduce energy requirements by 28-31% on average at the cost of slightly higher bitrates. The results indicate that optimising based on energy-quality curves rather than rate-quality curves and using quality levels to create the rungs could potentially improve energy efficiency for a comparable quality of experience.
△ Less
Submitted 10 February, 2024;
originally announced February 2024.
-
Comparative Study of Hardware and Software Power Measurements in Video Compression
Authors:
Angeliki Katsenou,
Xinyi Wang,
Daniel Schien,
David Bull
Abstract:
The environmental impact of video streaming services has been discussed as part of the strategies towards sustainable information and communication technologies. A first step towards that is the energy profiling and assessment of energy consumption of existing video technologies. This paper presents a comprehensive study of power measurement techniques in video compression, comparing the use of ha…
▽ More
The environmental impact of video streaming services has been discussed as part of the strategies towards sustainable information and communication technologies. A first step towards that is the energy profiling and assessment of energy consumption of existing video technologies. This paper presents a comprehensive study of power measurement techniques in video compression, comparing the use of hardware and software power meters. An experimental methodology to ensure reliability of measurements is introduced. Key findings demonstrate the high correlation of hardware and software based energy measurements for two video codecs across different spatial and temporal resolutions at a lower computational overhead.
△ Less
Submitted 19 December, 2023;
originally announced December 2023.
-
UGC Quality Assessment: Exploring the Impact of Saliency in Deep Feature-Based Quality Assessment
Authors:
Xinyi Wang,
Angeliki Katsenou,
David Bull
Abstract:
The volume of User Generated Content (UGC) has increased in recent years. The challenge with this type of content is assessing its quality. So far, the state-of-the-art metrics are not exhibiting a very high correlation with perceptual quality. In this paper, we explore state-of-the-art metrics that extract/combine natural scene statistics and deep neural network features. We experiment with these…
▽ More
The volume of User Generated Content (UGC) has increased in recent years. The challenge with this type of content is assessing its quality. So far, the state-of-the-art metrics are not exhibiting a very high correlation with perceptual quality. In this paper, we explore state-of-the-art metrics that extract/combine natural scene statistics and deep neural network features. We experiment with these by introducing saliency maps to improve perceptibility. We train and test our models using public datasets, namely, YouTube-UGC and KoNViD-1k. Preliminary results indicate that high correlations are achieved by using only deep features while adding saliency is not always boosting the performance. Our results and code will be made publicly available to serve as a benchmark for the research community and can be found on our project page: https://github.com/xinyiW915/SPIE-2023-Supplementary.
△ Less
Submitted 13 August, 2023;
originally announced August 2023.
-
Subjective assessment of the impact of a content adaptive optimiser for compressing 4K HDR content with AV1
Authors:
Vibhoothi,
Angeliki Katsenou,
François Pitié,
Katarina Domijan,
Anil Kokaram
Abstract:
Since 2015 video dimensionality has expanded to higher spatial and temporal resolutions and a wider colour gamut. This High Dynamic Range (HDR) content has gained traction in the consumer space as it delivers an enhanced quality of experience. At the same time, the complexity of codecs is growing. This has driven the development of tools for content-adaptive optimisation that achieve optimal rate-…
▽ More
Since 2015 video dimensionality has expanded to higher spatial and temporal resolutions and a wider colour gamut. This High Dynamic Range (HDR) content has gained traction in the consumer space as it delivers an enhanced quality of experience. At the same time, the complexity of codecs is growing. This has driven the development of tools for content-adaptive optimisation that achieve optimal rate-distortion performance for HDR video at 4K resolution. While improvements of just a few percentage points in BD-Rate (1-5\%) are significant for the streaming media industry, the impact on subjective quality has been less studied especially for HDR/AV1. In this paper, we conduct a subjective quality assessment (42 subjects) of 4K HDR content with a per-clip optimisation strategy. We correlate these subjective scores with existing popular objective metrics used in standard development and show that some perceptual metrics correlate surprisingly well even though they are not tuned for HDR. We find that the DSQCS protocol is too insensitive to categorically compare the methods but the data allows us to make recommendations about the use of experts vs non-experts in HDR studies, and explain the subjective impact of film grain in HDR content under compression.
△ Less
Submitted 26 June, 2023;
originally announced June 2023.
-
Recommendations for Verifying HDR Subjective Testing Workflows
Authors:
Vibhoothi,
Angeliki Katsenou,
John Squires,
François Pitié,
Anil Kokaram
Abstract:
Over the past few years, there has been an increase in the demand and availability of High Dynamic Range (HDR) displays and content. To ensure the production of high-quality materials, human evaluation is required. However, ascertaining whether the full playback pipeline is indeed HDR-compliant can be challenging. In this paper, we present a set of recommendations for conformance testing to valida…
▽ More
Over the past few years, there has been an increase in the demand and availability of High Dynamic Range (HDR) displays and content. To ensure the production of high-quality materials, human evaluation is required. However, ascertaining whether the full playback pipeline is indeed HDR-compliant can be challenging. In this paper, we present a set of recommendations for conformance testing to validate various aspects of the testing workflow, including playback, displays, brightness, colours, and viewing environment. We assessed the effectiveness of HDR conversion techniques used in current standards development (3GPP) for making source materials. Additionally, we evaluate HDR display technologies, including OLED and LCD, using both consumer television and a reference monitor.
△ Less
Submitted 19 May, 2023;
originally announced May 2023.
-
Comparison of HDR quality metrics in Per-Clip Lagrangian multiplier optimisation with AV1
Authors:
Vibhoothi,
François Pitié,
Angeliki Katsenou,
Yeping Su,
Balu Adsumilli,
Anil Kokaram
Abstract:
The complexity of modern codecs along with the increased need of delivering high-quality videos at low bitrates has reinforced the idea of a per-clip tailoring of parameters for optimised rate-distortion performance. While the objective quality metrics used for Standard Dynamic Range (SDR) videos have been well studied, the transitioning of consumer displays to support High Dynamic Range (HDR) vid…
▽ More
The complexity of modern codecs along with the increased need of delivering high-quality videos at low bitrates has reinforced the idea of a per-clip tailoring of parameters for optimised rate-distortion performance. While the objective quality metrics used for Standard Dynamic Range (SDR) videos have been well studied, the transitioning of consumer displays to support High Dynamic Range (HDR) videos, poses a new challenge to rate-distortion optimisation. In this paper, we review the popular HDR metrics DeltaE100 (DE100), PSNRL100, wPSNR, and HDR-VQM. We measure the impact of employing these metrics in per-clip direct search optimisation of the rate-distortion Lagrange multiplier in AV1. We report, on 35 HDR videos, average Bjontegaard Delta Rate (BD-Rate) gains of 4.675%, 2.226%, and 7.253% in terms of DE100, PSNRL100, and HDR-VQM. We also show that the inclusion of chroma in the quality metrics has a significant impact on optimisation, which can only be partially addressed by the use of chroma offsets.
△ Less
Submitted 26 April, 2023; v1 submitted 28 March, 2023;
originally announced March 2023.
-
Direct Optimisation of $\boldsymbolλ$ for HDR Content Adaptive Transcoding in AV1
Authors:
Vibhoothi,
François Pitié,
Angeliki Katsenou,
Daniel Joseph Ringis,
Yeping Su,
Neil Birkbeck,
Jessie Lin,
Balu Adsumilli,
Anil Kokaram
Abstract:
Since the adoption of VP9 by Netflix in 2016, royalty-free coding standards continued to gain prominence through the activities of the AOMedia consortium. AV1, the latest open source standard, is now widely supported. In the early years after standardisation, HDR video tends to be under served in open source encoders for a variety of reasons including the relatively small amount of true HDR conten…
▽ More
Since the adoption of VP9 by Netflix in 2016, royalty-free coding standards continued to gain prominence through the activities of the AOMedia consortium. AV1, the latest open source standard, is now widely supported. In the early years after standardisation, HDR video tends to be under served in open source encoders for a variety of reasons including the relatively small amount of true HDR content being broadcast and the challenges in RD optimisation with that material. AV1 codec optimisation has been ongoing since 2020 including consideration of the computational load. In this paper, we explore the idea of direct optimisation of the Lagrangian $λ$ parameter used in the rate control of the encoders to estimate the optimal Rate-Distortion trade-off achievable for a High Dynamic Range signalled video clip. We show that by adjusting the Lagrange multiplier in the RD optimisation process on a frame-hierarchy basis, we are able to increase the Bjontegaard difference rate gains by more than 3.98$\times$ on average without visually affecting the quality.
△ Less
Submitted 7 October, 2022; v1 submitted 23 August, 2022;
originally announced August 2022.
-
Enhancing VMAF through New Feature Integration and Model Combination
Authors:
Fan Zhang,
Angeliki Katsenou,
Christos Bampis,
Lukas Krasula,
Zhi Li,
David Bull
Abstract:
VMAF is a machine learning based video quality assessment method, originally designed for streaming applications, which combines multiple quality metrics and video features through SVM regression. It offers higher correlation with subjective opinions compared to many conventional quality assessment methods. In this paper we propose enhancements to VMAF through the integration of new video features…
▽ More
VMAF is a machine learning based video quality assessment method, originally designed for streaming applications, which combines multiple quality metrics and video features through SVM regression. It offers higher correlation with subjective opinions compared to many conventional quality assessment methods. In this paper we propose enhancements to VMAF through the integration of new video features and alternative quality metrics (selected from a diverse pool) alongside multiple model combination. The proposed combination approach enables training on multiple databases with varying content and distortion characteristics. Our enhanced VMAF method has been evaluated on eight HD video databases, and consistently outperforms the original VMAF model (0.6.1) and other benchmark quality metrics, exhibiting higher correlation with subjective ground truth data.
△ Less
Submitted 10 March, 2021;
originally announced March 2021.
-
Encoding in the Dark Grand Challenge: An Overview
Authors:
Nantheera Anantrasirichai,
Fan Zhang,
Alexandra Malyugina,
Paul Hill,
Angeliki Katsenou
Abstract:
A big part of the video content we consume from video providers consists of genres featuring low-light aesthetics. Low light sequences have special characteristics, such as spatio-temporal varying acquisition noise and light flickering, that make the encoding process challenging. To deal with the spatio-temporal incoherent noise, higher bitrates are used to achieve high objective quality. Addition…
▽ More
A big part of the video content we consume from video providers consists of genres featuring low-light aesthetics. Low light sequences have special characteristics, such as spatio-temporal varying acquisition noise and light flickering, that make the encoding process challenging. To deal with the spatio-temporal incoherent noise, higher bitrates are used to achieve high objective quality. Additionally, the quality assessment metrics and methods have not been designed, trained or tested for this type of content. This has inspired us to trigger research in that area and propose a Grand Challenge on encoding low-light video sequences. In this paper, we present an overview of the proposed challenge, and test state-of-the-art methods that will be part of the benchmark methods at the stage of the participants' deliverable assessment. From this exploration, our results show that VVC already achieves a high performance compared to simply denoising the video source prior to encoding. Moreover, the quality of the video streams can be further improved by employing a post-processing image enhancement method.
△ Less
Submitted 7 May, 2020;
originally announced May 2020.
