-
Multi-Modal Multi-Task Federated Foundation Models for Next-Generation Extended Reality Systems: Towards Privacy-Preserving Distributed Intelligence in AR/VR/MR
Authors:
Fardis Nadimi,
Payam Abdisarabshali,
Kasra Borazjani,
Jacob Chakareski,
Seyyedali Hosseinalipour
Abstract:
Extended reality (XR) systems, which consist of virtual reality (VR), augmented reality (AR), and mixed reality (XR), offer a transformative interface for immersive, multi-modal, and embodied human-computer interaction. In this paper, we envision that multi-modal multi-task (M3T) federated foundation models (FedFMs) can offer transformative capabilities for XR systems through integrating the repre…
▽ More
Extended reality (XR) systems, which consist of virtual reality (VR), augmented reality (AR), and mixed reality (XR), offer a transformative interface for immersive, multi-modal, and embodied human-computer interaction. In this paper, we envision that multi-modal multi-task (M3T) federated foundation models (FedFMs) can offer transformative capabilities for XR systems through integrating the representational strength of M3T foundation models (FMs) with the privacy-preserving model training principles of federated learning (FL). We present a modular architecture for FedFMs, which entails different coordination paradigms for model training and aggregations. Central to our vision is the codification of XR challenges that affect the implementation of FedFMs under the SHIFT dimensions: (1) Sensor and modality diversity, (2) Hardware heterogeneity and system-level constraints, (3) Interactivity and embodied personalization, (4) Functional/task variability, and (5) Temporality and environmental variability. We illustrate the manifestation of these dimensions across a set of emerging and anticipated applications of XR systems. Finally, we propose evaluation metrics, dataset requirements, and design tradeoffs necessary for the development of resource-aware FedFMs in XR. This perspective aims to chart the technical and conceptual foundations for context-aware privacy-preserving intelligence in the next generation of XR systems.
△ Less
Submitted 12 June, 2025; v1 submitted 5 June, 2025;
originally announced June 2025.
-
Neural-Enhanced Rate Adaptation and Computation Distribution for Emerging mmWave Multi-User 3D Video Streaming Systems
Authors:
Babak Badnava,
Jacob Chakareski,
Morteza Hashemi
Abstract:
We investigate multitask edge-user communication-computation resource allocation for $360^\circ$ video streaming in an edge-computing enabled millimeter wave (mmWave) multi-user virtual reality system. To balance the communication-computation trade-offs that arise herein, we formulate a video quality maximization problem that integrates interdependent multitask/multi-user action spaces and rebuffe…
▽ More
We investigate multitask edge-user communication-computation resource allocation for $360^\circ$ video streaming in an edge-computing enabled millimeter wave (mmWave) multi-user virtual reality system. To balance the communication-computation trade-offs that arise herein, we formulate a video quality maximization problem that integrates interdependent multitask/multi-user action spaces and rebuffering time/quality variation constraints. We formulate a deep reinforcement learning framework for \underline{m}ulti-\underline{t}ask \underline{r}ate adaptation and \underline{c}omputation distribution (MTRC) to solve the problem of interest. Our solution does not rely on a priori knowledge about the environment and uses only prior video streaming statistics (e.g., throughput, decoding time, and transmission delay), and content information, to adjust the assigned video bitrates and computation distribution, as it observes the induced streaming performance online. Moreover, to capture the task interdependence in the environment, we leverage neural network cascades to extend our MTRC method to two novel variants denoted as R1C2 and C1R2. We train all three methods with real-world mmWave network traces and $360^\circ$ video datasets to evaluate their performance in terms of expected quality of experience (QoE), viewport peak signal-to-noise ratio (PSNR), rebuffering time, and quality variation. We outperform state-of-the-art rate adaptation algorithms, with C1R2 showing best results and achieving $5.21-6.06$ dB PSNR gains, $2.18-2.70$x rebuffering time reduction, and $4.14-4.50$ dB quality variation reduction.
△ Less
Submitted 19 May, 2025;
originally announced May 2025.
-
Learning Driven Elastic Task Multi-Connectivity Immersive Computing Systems
Authors:
Babak Badnava,
Jacob Chakareski,
Morteza Hashemi
Abstract:
In virtual reality (VR) environments, computational tasks exhibit an elastic nature, meaning they can dynamically adjust based on various user and system constraints. This elasticity is essential for maintaining immersive experiences; however, it also introduces challenges for communication and computing in VR systems. In this paper, we investigate elastic task offloading for multi-user edge-compu…
▽ More
In virtual reality (VR) environments, computational tasks exhibit an elastic nature, meaning they can dynamically adjust based on various user and system constraints. This elasticity is essential for maintaining immersive experiences; however, it also introduces challenges for communication and computing in VR systems. In this paper, we investigate elastic task offloading for multi-user edge-computing-enabled VR systems with multi-connectivity, aiming to maximize the computational energy-efficiency (computational throughput per unit of energy consumed). To balance the induced communication, computation, energy consumption, and quality of experience trade-offs due to the elasticity of VR tasks, we formulate a constrained stochastic computational energy-efficiency optimization problem that integrates the multi-connectivity/multi-user action space and the elastic nature of VR computational tasks. We formulate a centralized phasic policy gradient (CPPG) framework to solve the problem of interest online, using only prior elastic task offloading statistics (energy consumption, response time, and transmission time), and task information (i.e., task size and computational intensity), while observing the induced system performance (energy consumption and latency). We further extend our approach to decentralized learning by formulating an independent phasic policy gradient (IPPG) method and a decentralized shared multi-armed bandit (DSMAB) method. We train our methods with real-world 4G, 5G, and WiGig network traces and 360 video datasets to evaluate their performance in terms of response time, energy efficiency, scalability, and delivered quality of experience. We also provide a comprehensive analysis of task size and its effect on offloading policy and system performance. In particular, we show that CPPG reduces latency by 28% and energy consumption by 78% compared to IPPG.
△ Less
Submitted 19 May, 2025;
originally announced May 2025.
-
Multi-Task Decision-Making for Multi-User 360 Video Processing over Wireless Networks
Authors:
Babak Badnava,
Jacob Chakareski,
Morteza Hashemi
Abstract:
We study a multi-task decision-making problem for 360 video processing in a wireless multi-user virtual reality (VR) system that includes an edge computing unit (ECU) to deliver 360 videos to VR users and offer computing assistance for decoding/rendering of video frames. However, this comes at the expense of increased data volume and required bandwidth. To balance this trade-off, we formulate a co…
▽ More
We study a multi-task decision-making problem for 360 video processing in a wireless multi-user virtual reality (VR) system that includes an edge computing unit (ECU) to deliver 360 videos to VR users and offer computing assistance for decoding/rendering of video frames. However, this comes at the expense of increased data volume and required bandwidth. To balance this trade-off, we formulate a constrained quality of experience (QoE) maximization problem in which the rebuffering time and quality variation between video frames are bounded by user and video requirements. To solve the formulated multi-user QoE maximization, we leverage deep reinforcement learning (DRL) for multi-task rate adaptation and computation distribution (MTRC). The proposed MTRC approach does not rely on any predefined assumption about the environment and relies on video playback statistics (i.e., past throughput, decoding time, transmission time, etc.), video information, and the resulting performance to adjust the video bitrate and computation distribution. We train MTRC with real-world wireless network traces and 360 video datasets to obtain evaluation results in terms of the average QoE, peak signal-to-noise ratio (PSNR), rebuffering time, and quality variation. Our results indicate that the MTRC improves the users' QoE compared to state-of-the-art rate adaptation algorithm. Specifically, we show a 5.97 dB to 6.44 dB improvement in PSNR, a 1.66X to 4.23X improvement in rebuffering time, and a 4.21 dB to 4.35 dB improvement in quality variation.
△ Less
Submitted 3 July, 2024;
originally announced July 2024.
-
BONES: Near-Optimal Neural-Enhanced Video Streaming
Authors:
Lingdong Wang,
Simran Singh,
Jacob Chakareski,
Mohammad Hajiesmaili,
Ramesh K. Sitaraman
Abstract:
Accessing high-quality video content can be challenging due to insufficient and unstable network bandwidth. Recent advances in neural enhancement have shown promising results in improving the quality of degraded videos through deep learning. Neural-Enhanced Streaming (NES) incorporates this new approach into video streaming, allowing users to download low-quality video segments and then enhance th…
▽ More
Accessing high-quality video content can be challenging due to insufficient and unstable network bandwidth. Recent advances in neural enhancement have shown promising results in improving the quality of degraded videos through deep learning. Neural-Enhanced Streaming (NES) incorporates this new approach into video streaming, allowing users to download low-quality video segments and then enhance them to obtain high-quality content without violating the playback of the video stream. We introduce BONES, an NES control algorithm that jointly manages the network and computational resources to maximize the quality of experience (QoE) of the user. BONES formulates NES as a Lyapunov optimization problem and solves it in an online manner with near-optimal performance, making it the first NES algorithm to provide a theoretical performance guarantee. Comprehensive experimental results indicate that BONES increases QoE by 5\% to 20\% over state-of-the-art algorithms with minimal overhead. Our code is available at https://github.com/UMass-LIDS/bones.
△ Less
Submitted 10 April, 2024; v1 submitted 15 October, 2023;
originally announced October 2023.
-
CU-Net: Real-Time High-Fidelity Color Upsampling for Point Clouds
Authors:
Lingdong Wang,
Mohammad Hajiesmaili,
Jacob Chakareski,
Ramesh K. Sitaraman
Abstract:
Point cloud upsampling is essential for high-quality augmented reality, virtual reality, and telepresence applications, due to the capture, processing, and communication limitations of existing technologies. Although geometry upsampling to densify a point cloud's coordinates has been well studied, the upsampling of the color attributes has been largely overlooked. In this paper, we propose CU-Net,…
▽ More
Point cloud upsampling is essential for high-quality augmented reality, virtual reality, and telepresence applications, due to the capture, processing, and communication limitations of existing technologies. Although geometry upsampling to densify a point cloud's coordinates has been well studied, the upsampling of the color attributes has been largely overlooked. In this paper, we propose CU-Net, the first deep-learning point cloud color upsampling model that enables low latency and high visual fidelity operation. CU-Net achieves linear time and space complexity by leveraging a feature extractor based on sparse convolution and a color prediction module based on neural implicit function. Therefore, CU-Net is theoretically guaranteed to be more efficient than most existing methods with quadratic complexity. Experimental results demonstrate that CU-Net can colorize a photo-realistic point cloud with nearly a million points in real time, while having notably better visual performance than baselines. Besides, CU-Net can adapt to arbitrary upsampling ratios and unseen objects without retraining. Our source code is available at https://github.com/UMass-LIDS/cunet.
△ Less
Submitted 16 November, 2022; v1 submitted 12 September, 2022;
originally announced September 2022.
-
Towards Enabling Next Generation Societal Virtual Reality Applications for Virtual Human Teleportation
Authors:
Jacob Chakareski,
Mahmudur Khan,
Murat Yuksel
Abstract:
Virtual reality (VR) is an emerging technology of great societal potential. Some of its most exciting and promising use cases include remote scene content and untethered lifelike navigation. This article first highlights the relevance of such future societal applications and the challenges ahead towards enabling them. It then provides a broad and contextual high-level perspective of several emergi…
▽ More
Virtual reality (VR) is an emerging technology of great societal potential. Some of its most exciting and promising use cases include remote scene content and untethered lifelike navigation. This article first highlights the relevance of such future societal applications and the challenges ahead towards enabling them. It then provides a broad and contextual high-level perspective of several emerging technologies and unconventional techniques and argues that only by their synergistic integration can the fundamental performance bottlenecks of hyper-intensive computation, ultra-high data rate, and ultra-low latency be overcome to enable untethered and lifelike VR-based remote scene immersion. A novel future system concept is introduced that embodies this holistic integration, unified with a rigorous analysis, to capture the fundamental synergies and interplay between communications, computation, and signal scalability that arise in this context, and advance its performance at the same time. Several representative results highlighting these trade-offs and the benefits of the envisioned system are presented at the end.
△ Less
Submitted 9 August, 2022;
originally announced August 2022.
-
Decentralized Collaborative Video Caching in 5G Small-Cell Base Station Cellular Networks
Authors:
Shadab Mahboob,
Koushik Kar,
Jacob Chakareski
Abstract:
We consider the problem of video caching across a set of 5G small-cell base stations (SBS) connected to each other over a high-capacity short-delay back-haul link, and linked to a remote server over a long-delay connection. Even though the problem of minimizing the overall video delivery delay is NP-hard, the Collaborative Caching Algorithm (CCA) that we present can efficiently compute a solution…
▽ More
We consider the problem of video caching across a set of 5G small-cell base stations (SBS) connected to each other over a high-capacity short-delay back-haul link, and linked to a remote server over a long-delay connection. Even though the problem of minimizing the overall video delivery delay is NP-hard, the Collaborative Caching Algorithm (CCA) that we present can efficiently compute a solution close to the optimal, where the degree of sub-optimality depends on the worst case video-to-cache size ratio. The algorithm is naturally amenable to distributed implementation that requires zero explicit coordination between the SBSs, and runs in $O(N + K \log K)$ time, where $N$ is the number of SBSs (caches) and $K$ the maximum number of videos. We extend CCA to an online setting where the video popularities are not known a priori but are estimated over time through a limited amount of periodic information sharing between SBSs. We demonstrate that our algorithm closely approaches the optimal integral caching solution as the cache size increases. Moreover, via simulations carried out on real video access traces, we show that our algorithm effectively uses the SBS caches to reduce the video delivery delay and conserve the remote server's bandwidth, and that it outperforms two other reference caching methods adapted to our system setting.
△ Less
Submitted 1 September, 2021;
originally announced September 2021.
-
A Review of AI-enabled Routing Protocols for UAV Networks: Trends, Challenges, and Future Outlook
Authors:
Arnau Rovira-Sugranes,
Abolfazl Razi,
Fatemeh Afghah,
Jacob Chakareski
Abstract:
Unmanned Aerial Vehicles (UAVs), as a recently emerging technology, enabled a new breed of unprecedented applications in different domains. This technology's ongoing trend is departing from large remotely-controlled drones to networks of small autonomous drones to collectively complete intricate tasks time and cost-effectively. An important challenge is developing efficient sensing, communication,…
▽ More
Unmanned Aerial Vehicles (UAVs), as a recently emerging technology, enabled a new breed of unprecedented applications in different domains. This technology's ongoing trend is departing from large remotely-controlled drones to networks of small autonomous drones to collectively complete intricate tasks time and cost-effectively. An important challenge is developing efficient sensing, communication, and control algorithms that can accommodate the requirements of highly dynamic UAV networks with heterogeneous mobility levels. Recently, the use of Artificial Intelligence (AI) in learning-based networking has gained momentum to harness the learning power of cognizant nodes to make more intelligent networking decisions by integrating computational intelligence into UAV networks. An important example of this trend is developing learning-powered routing protocols, where machine learning methods are used to model and predict topology evolution, channel status, traffic mobility, and environmental factors for enhanced routing.
This paper reviews AI-enabled routing protocols designed primarily for aerial networks, including topology-predictive and self-adaptive learning-based routing algorithms, with an emphasis on accommodating highly-dynamic network topology. To this end, we justify the importance and adaptation of AI into UAV network communications. We also address, with an AI emphasis, the closely related topics of mobility and networking models for UAV networks, simulation tools and public datasets, and relations to UAV swarming, which serve to choose the right algorithm for each scenario. We conclude by presenting future trends, and the remaining challenges in AI-based UAV networking, for different aspects of routing, connectivity, topology control, security and privacy, energy efficiency, and spectrum sharing.
△ Less
Submitted 8 November, 2021; v1 submitted 2 April, 2021;
originally announced April 2021.
-
Wildfire Monitoring in Remote Areas using Autonomous Unmanned Aerial Vehicles
Authors:
Fatemeh Afghah,
Abolfazl Razi,
Jacob Chakareski,
Jonathan Ashdown
Abstract:
In this paper, we propose a drone-based wildfire monitoring system for remote and hard-to-reach areas. This system utilizes autonomous unmanned aerial vehicles (UAVs) with the main advantage of providing on-demand monitoring service faster than the current approaches of using satellite images, manned aircraft and remotely controlled drones. Furthermore, using autonomous drones facilitates minimizi…
▽ More
In this paper, we propose a drone-based wildfire monitoring system for remote and hard-to-reach areas. This system utilizes autonomous unmanned aerial vehicles (UAVs) with the main advantage of providing on-demand monitoring service faster than the current approaches of using satellite images, manned aircraft and remotely controlled drones. Furthermore, using autonomous drones facilitates minimizing human intervention in risky wildfire zones. In particular, to develop a fully autonomous system, we propose a distributed leader-follower coalition formation model to cluster a set of drones into multiple coalitions that collectively cover the designated monitoring field. The coalition leader is a drone %with longer communication range that employs observer drones potentially with different sensing and imaging %actuation capabilities to hover in circular paths and collect imagery information from the impacted areas. The objectives of the proposed system include i) to cover the entire fire zone with a minimum number of drones, and ii) to minimize the energy consumption and latency of the available drones to fly to the fire zone. Simulation results confirm that the performance of the proposed system -- without the need for inter-coalition communications -- approaches that of a centrally-optimized system.
△ Less
Submitted 15 April, 2019;
originally announced May 2019.
-
Visible Light Communication for Next Generation Untethered Virtual Reality Systems
Authors:
Mahmudur Khan,
Jacob Chakareski
Abstract:
Virtual and augmented reality (VR/AR) systems are emerging technologies requiring data rates of multiple Gbps. Existing high quality VR headsets require connections through HDMI cables to a computer rendering rich graphic contents to meet the extremely high data transfer rate requirement. Such a cable connection limits the VR user's mobility and interferes with the VR experience. Current wireless…
▽ More
Virtual and augmented reality (VR/AR) systems are emerging technologies requiring data rates of multiple Gbps. Existing high quality VR headsets require connections through HDMI cables to a computer rendering rich graphic contents to meet the extremely high data transfer rate requirement. Such a cable connection limits the VR user's mobility and interferes with the VR experience. Current wireless technologies such as WiFi cannot support the multi-Gbps graphics data transfer. Instead, we propose to use visible light communication (VLC) for establishing high speed wireless links between a rendering computer and a VR headset. But, VLC transceivers are highly directional with narrow beams and require constant maintenance of line-of-sight (LOS) alignment between the transmitter and the receiver. Thus, we present a novel multi-detector hemispherical VR headset design to tackle the beam misalignment problem caused by the VR user's random head orientation. We provide detailed analysis on how the number of detectors on the headset can be minimized while maintaining the required beam alignment and providing high quality VR experience.
△ Less
Submitted 7 April, 2019;
originally announced April 2019.
-
Accelerated Structure-Aware Reinforcement Learning for Delay-Sensitive Energy Harvesting Wireless Sensors
Authors:
Nikhilesh Sharma,
Nicholas Mastronarde,
Jacob Chakareski
Abstract:
We investigate an energy-harvesting wireless sensor transmitting latency-sensitive data over a fading channel. The sensor injects captured data packets into its transmission queue and relies on ambient energy harvested from the environment to transmit them. We aim to find the optimal scheduling policy that decides whether or not to transmit the queue's head-of-line packet at each transmission oppo…
▽ More
We investigate an energy-harvesting wireless sensor transmitting latency-sensitive data over a fading channel. The sensor injects captured data packets into its transmission queue and relies on ambient energy harvested from the environment to transmit them. We aim to find the optimal scheduling policy that decides whether or not to transmit the queue's head-of-line packet at each transmission opportunity such that the expected packet queuing delay is minimized given the available harvested energy. No prior knowledge of the stochastic processes that govern the channel, captured data, or harvested energy dynamics are assumed, thereby necessitating the use of online learning to optimize the scheduling policy. We formulate this scheduling problem as a Markov decision process (MDP) and analyze the structural properties of its optimal value function. In particular, we show that it is non-decreasing and has increasing differences in the queue backlog and that it is non-increasing and has increasing differences in the battery state. We exploit this structure to formulate a novel accelerated reinforcement learning (RL) algorithm to solve the scheduling problem online at a much faster learning rate, while limiting the induced computational complexity. Our experiments demonstrate that the proposed algorithm closely approximates the performance of an optimal offline solution that requires a priori knowledge of the channel, captured data, and harvested energy dynamics. Simultaneously, by leveraging the value function's structure, our approach achieves competitive performance relative to a state-of-the-art RL algorithm, at potentially orders of magnitude lower complexity. Finally, considerable performance gains are demonstrated over the well-known and widely used Q-learning algorithm.
△ Less
Submitted 5 May, 2019; v1 submitted 22 July, 2018;
originally announced July 2018.
-
Structural Properties of Optimal Transmission Policies for Delay-Sensitive Energy Harvesting Wireless Sensors
Authors:
Nikhilesh Sharma,
Nicholas Mastronarde,
Jacob Chakareski
Abstract:
We consider an energy harvesting sensor transmitting latency-sensitive data over a fading channel. We aim to find the optimal transmission scheduling policy that minimizes the packet queuing delay given the available harvested energy. We formulate the problem as a Markov decision process (MDP) over a state-space spanned by the transmitter's buffer, battery, and channel states, and analyze the stru…
▽ More
We consider an energy harvesting sensor transmitting latency-sensitive data over a fading channel. We aim to find the optimal transmission scheduling policy that minimizes the packet queuing delay given the available harvested energy. We formulate the problem as a Markov decision process (MDP) over a state-space spanned by the transmitter's buffer, battery, and channel states, and analyze the structural properties of the resulting optimal value function, which quantifies the long-run performance of the optimal scheduling policy. We show that the optimal value function (i) is non-decreasing and has increasing differences in the queue backlog; (ii) is non-increasing and has increasing differences in the battery state; and (iii) is submodular in the buffer and battery states. Our numerical results confirm these properties and demonstrate that the optimal scheduling policy outperforms a so-called greedy policy in terms of sensor outages, buffer overflows, energy efficiency, and queuing delay.
△ Less
Submitted 26 March, 2018;
originally announced March 2018.
-
Viewport-Driven Rate-Distortion Optimized 360° Video Streaming
Authors:
Jacob Chakareski,
Ridvan Aksu,
Xavier Corbillon,
Gwendal Simon,
Viswanathan Swaminathan
Abstract:
The growing popularity of virtual and augmented reality communications and 360° video streaming is moving video communication systems into much more dynamic and resource-limited operating settings. The enormous data volume of 360° videos requires an efficient use of network bandwidth to maintain the desired quality of experience for the end user. To this end, we propose a framework for viewport-dr…
▽ More
The growing popularity of virtual and augmented reality communications and 360° video streaming is moving video communication systems into much more dynamic and resource-limited operating settings. The enormous data volume of 360° videos requires an efficient use of network bandwidth to maintain the desired quality of experience for the end user. To this end, we propose a framework for viewport-driven rate-distortion optimized 360° video streaming that integrates the user view navigation pattern and the spatiotemporal rate-distortion characteristics of the 360° video content to maximize the delivered user quality of experience for the given network/system resources. The framework comprises a methodology for constructing dynamic heat maps that capture the likelihood of navigating different spatial segments of a 360° video over time by the user, an analysis and characterization of its spatiotemporal rate-distortion characteristics that leverage preprocessed spatial tilling of the 360° view sphere, and an optimization problem formulation that characterizes the delivered user quality of experience given the user navigation patterns, 360° video encoding decisions, and the available system/network resources. Our experimental results demonstrate the advantages of our framework over the conventional approach of streaming a monolithic uniformly encoded 360° video and a state-of-the-art reference method. Considerable video quality gains of 4 - 5 dB are demonstrated in the case of two popular 4K 360° videos.
△ Less
Submitted 21 March, 2018;
originally announced March 2018.
-
A Coalition Formation Approach to Coordinated Task Allocation in Heterogeneous UAV Networks
Authors:
Fatemeh Afghah,
Mohammad Zaeri-Amirani,
Abolfazl Razi,
Jacob Chakareski,
Elizabeth Bentley
Abstract:
The problem of adversary target detection and the subsequent task completion using a heterogeneous network of resource-constrained UAVs is considered. No prior knowledge about locations and required resources to identify these targets is available to the UAVs. In the proposed leader-follower coalition formation model, the UAV that first locates a target serves as the coalition leader and selects a…
▽ More
The problem of adversary target detection and the subsequent task completion using a heterogeneous network of resource-constrained UAVs is considered. No prior knowledge about locations and required resources to identify these targets is available to the UAVs. In the proposed leader-follower coalition formation model, the UAV that first locates a target serves as the coalition leader and selects a group of follower UAVs to complete the task associated with the identified target. The goal of the coalition formation is to complete the designated tasks with minimal resource utilization. Another role of coalition members is to make the ground station aware of the detected adversary target by forwarding its signal to the station via a distributed cooperative relaying scheme. We also propose a reputation-based mechanism for coalition formation to monitor the cooperative behavior of the UAVs over the course of time and exclude potentially untrustworthy UAVs. Simulation results show the efficiency of the proposed method in forming optimal coalitions compared to alternative methods.
△ Less
Submitted 1 November, 2017;
originally announced November 2017.
-
On The Continuous Coverage Problem for a Swarm of UAVs
Authors:
Hazim Shakhatreh,
Abdallah Khreishah,
Jacob Chakareski,
Haythem Bany Salameh,
Issa Khalil
Abstract:
Unmanned aerial vehicles (UAVs) can be used to provide wireless network and remote surveillance coverage for disaster-affected areas. During such a situation, the UAVs need to return periodically to a charging station for recharging, due to their limited battery capacity. We study the problem of minimizing the number of UAVs required for a continuous coverage of a given area, given the recharging…
▽ More
Unmanned aerial vehicles (UAVs) can be used to provide wireless network and remote surveillance coverage for disaster-affected areas. During such a situation, the UAVs need to return periodically to a charging station for recharging, due to their limited battery capacity. We study the problem of minimizing the number of UAVs required for a continuous coverage of a given area, given the recharging requirement. We prove that this problem is NP-complete. Due to its intractability, we study partitioning the coverage graph into cycles that start at the charging station. We first characterize the minimum number of UAVs to cover such a cycle based on the charging time, the traveling time, and the number of subareas to be covered by the cycle. Based on this analysis, we then develop an efficient algorithm, the cycles with limited energy algorithm. The straightforward method to continuously cover a given area is to split it into N subareas and cover it by N cycles using N additional UAVs. Our simulation results examine the importance of critical system parameters: the energy capacity of the UAVs, the number of subareas in the covered area, and the UAV charging and traveling times.We demonstrate that the cycles with limited energy algorithm requires 69%-94% fewer additional UAVs relative to the straightforward method, as the energy capacity of the UAVs is increased, and 67%-71% fewer additional UAVs, as the number of subareas is increased.
△ Less
Submitted 27 May, 2017;
originally announced May 2017.
-
Aerial UAV-IoT Sensing for Ubiquitous Immersive Communication and Virtual Human Teleportation
Authors:
Jacob Chakareski
Abstract:
We consider UAV IoT aerial sensing that delivers multiple VR/AR immersive communication sessions to remote users. The UAV swarm is spatially distributed over a wide area of interest, and each UAV captures a viewpoint of the scene below it. The remote users are interested in visual immersive navigation of specific subareas/scenes of interest, reconstructed on their respective VR/AR devices from the…
▽ More
We consider UAV IoT aerial sensing that delivers multiple VR/AR immersive communication sessions to remote users. The UAV swarm is spatially distributed over a wide area of interest, and each UAV captures a viewpoint of the scene below it. The remote users are interested in visual immersive navigation of specific subareas/scenes of interest, reconstructed on their respective VR/AR devices from the captured data. The reconstruction quality of the immersive scene representations at the users will depend on the sampling/sensing rates associated with each UAV. There is a limit on the aggregate amount of data that the UAV swarm can sample and send towards the users, stemming from physical/transmission capacity constraints. Similarly, each VR/AR application has minimum reconstruction quality requirements for its own session. We propose an optimization framework that makes three contributions in this context. First, we select the optimal sampling rates to be used by each UAV, such that the system and application constraints are not exceed, while the priority weighted reconstruction quality across all VR/AR sessions is maximized. Then, we design an optimal scalable source-channel signal representation that instills into the captured data inherent rate adaptivity, unequal error protection, and minimum required redundancy. Finally, the UAV transmission efficiency is enhanced by the use of small-form-factor multi-beam directional antennas and optimal power/link scheduling across the scalable signal representation layers. Our experiments demonstrate competitive advantages over conventional methods for visual sensing. This is a first-of-its-kind study of an emerging application of prospectively broad societal impact.
△ Less
Submitted 21 July, 2017; v1 submitted 12 March, 2017;
originally announced March 2017.
-
Viewport-Adaptive Navigable 360-Degree Video Delivery
Authors:
Xavier Corbillon,
Gwendal Simon,
Alisa Devlic,
Jacob Chakareski
Abstract:
The delivery and display of 360-degree videos on Head-Mounted Displays (HMDs) presents many technical challenges. 360-degree videos are ultra high resolution spherical videos, which contain an omnidirectional view of the scene. However only a portion of this scene is displayed on the HMD. Moreover, HMD need to respond in 10 ms to head movements, which prevents the server to send only the displayed…
▽ More
The delivery and display of 360-degree videos on Head-Mounted Displays (HMDs) presents many technical challenges. 360-degree videos are ultra high resolution spherical videos, which contain an omnidirectional view of the scene. However only a portion of this scene is displayed on the HMD. Moreover, HMD need to respond in 10 ms to head movements, which prevents the server to send only the displayed video part based on client feedback. To reduce the bandwidth waste, while still providing an immersive experience, a viewport-adaptive 360-degree video streaming system is proposed. The server prepares multiple video representations, which differ not only by their bit-rate, but also by the qualities of different scene regions. The client chooses a representation for the next segment such that its bit-rate fits the available throughput and a full quality region matches its viewing. We investigate the impact of various spherical-to-plane projections and quality arrangements on the video quality displayed to the user, showing that the cube map layout offers the best quality for the given bit-rate budget. An evaluation with a dataset of users navigating 360-degree videos demonstrates that segments need to be short enough to enable frequent view switches.
△ Less
Submitted 10 May, 2017; v1 submitted 26 September, 2016;
originally announced September 2016.
-
Joint Caching, Routing, and Channel Assignment for Collaborative Small-Cell Cellular Networks
Authors:
Abdallah Khreishah,
Jacob Chakareski,
Ammar Gharaibeh
Abstract:
We consider joint caching, routing, and channel assignment for video delivery over coordinated small-cell cellular systems of the future Internet. We formulate the problem of maximizing the throughput of the system as a linear program in which the number of variables is very large. To address channel interference, our formulation incorporates the conflict graph that arises when wireless links inte…
▽ More
We consider joint caching, routing, and channel assignment for video delivery over coordinated small-cell cellular systems of the future Internet. We formulate the problem of maximizing the throughput of the system as a linear program in which the number of variables is very large. To address channel interference, our formulation incorporates the conflict graph that arises when wireless links interfere with each other due to simultaneous transmission. We utilize the column generation method to solve the problem by breaking it into a restricted master subproblem that involves a select subset of variables and a collection of pricing subproblems that select the new variable to be introduced into the restricted master problem, if that leads to a better objective function value. To control the complexity of the column generation optimization further, due to the exponential number of independent sets that arise from the conflict graph, we introduce an approximation algorithm that computes a solution that is within $ε$ to optimality, at much lower complexity. Our framework demonstrates considerable gains in average transmission rate at which the video data can be delivered to the users, over the state-of-the-art Femtocaching system, of up to 46%. These operational gains in system performance map to analogous gains in video application quality, thereby enhancing the user experience considerably.
△ Less
Submitted 30 May, 2016;
originally announced May 2016.
-
Matrix Factorization-Based Clustering Of Image Features For Bandwidth-Constrained Information Retrieval
Authors:
Jacob Chakareski,
Immanuel Manohar,
Shantanu Rane
Abstract:
We consider the problem of accurately and efficiently querying a remote server to retrieve information about images captured by a mobile device. In addition to reduced transmission overhead and computational complexity, the retrieval protocol should be robust to variations in the image acquisition process, such as translation, rotation, scaling, and sensor-related differences. We propose to extrac…
▽ More
We consider the problem of accurately and efficiently querying a remote server to retrieve information about images captured by a mobile device. In addition to reduced transmission overhead and computational complexity, the retrieval protocol should be robust to variations in the image acquisition process, such as translation, rotation, scaling, and sensor-related differences. We propose to extract scale-invariant image features and then perform clustering to reduce the number of features needed for image matching. Principal Component Analysis (PCA) and Non-negative Matrix Factorization (NMF) are investigated as candidate clustering approaches. The image matching complexity at the database server is quadratic in the (small) number of clusters, not in the (very large) number of image features. We employ an image-dependent information content metric to approximate the model order, i.e., the number of clusters, needed for accurate matching, which is preferable to setting the model order using trial and error. We show how to combine the hypotheses provided by PCA and NMF factor loadings, thereby obtaining more accurate retrieval than using either approach alone. In experiments on a database of urban images, we obtain a top-1 retrieval accuracy of 89% and a top-3 accuracy of 92.5%.
△ Less
Submitted 6 May, 2016;
originally announced May 2016.
-
ComReg: A Complex Network Approach to Prioritize Test Cases for Regression Testing
Authors:
Imrul Kayes,
Jacob Chakareski
Abstract:
Regression testing is performed to provide confidence that changes in a part of software do not affect other parts of the software. An execution of all existing test cases is the best way to re-establish this confidence. However, regression testing is an expensive process---there might be insufficient resources (e.g., time, workforce) to allow for the re-execution of all test cases. Regression tes…
▽ More
Regression testing is performed to provide confidence that changes in a part of software do not affect other parts of the software. An execution of all existing test cases is the best way to re-establish this confidence. However, regression testing is an expensive process---there might be insufficient resources (e.g., time, workforce) to allow for the re-execution of all test cases. Regression test prioritization techniques attempt to re-order a regression test suite based on some criteria so that highest priority test cases are executed earlier.
In this study, we want to prioritize test cases for regression testing based on the dependency network of faults. In software testing, it is common that some faults are consequences of other faults (leading faults). Moreover, dependent faults can be removed if and only if the leading faults have been removed. Our goal is to prioritize test cases so that test cases that exposed leading faults (the most central faults in the fault dependency network) in the system testing phase, are executed first in regression testing.
We present ComReg, a test case prioritization technique based on the dependency network of faults. We model a fault dependency network as a directed graph and identify leading faults to prioritize test cases for regression testing. We use a centrality aggregation technique which considers six network representative centrality metrics to identify leading faults in the fault dependency network. We also discuss the use of fault communities to select an arbitrary percentage of the test cases from a prioritized regression test suite. We conduct a case study that evaluates the effectiveness and applicability of the proposed method.
△ Less
Submitted 26 November, 2014; v1 submitted 17 November, 2013;
originally announced November 2013.
-
Product Backlog Rating: A Case Study On Measuring Test Quality In Scrum
Authors:
Imrul Kayes,
Mithun Sarker,
Jacob Chakareski
Abstract:
Agile software development methodologies focus on software projects which are behind schedule or highly likely to have a problematic development phase. In the last decade, Agile methods have transformed from cult techniques to mainstream methodologies. Scrum, an Agile software development method, has been widely adopted due to its adaptive nature.
This paper presents a metric that measures the q…
▽ More
Agile software development methodologies focus on software projects which are behind schedule or highly likely to have a problematic development phase. In the last decade, Agile methods have transformed from cult techniques to mainstream methodologies. Scrum, an Agile software development method, has been widely adopted due to its adaptive nature.
This paper presents a metric that measures the quality of the testing process in a Scrum process. As product quality and process quality correlate, improved test quality can ensure high quality products. Also, gaining experience from eight years of successful Scrum implementation at SoftwarePeople, we describe the Scrum process emphasizing the testing process. We propose a metric Product Backlog Rating (PBR) to assess the testing process in Scrum. PBR considers the complexity of the features to be developed in an iteration of Scrum, assesses test ratings and offers a numerical score of the testing process. This metric is able to provide a comprehensive overview of the testing process over the development cycle of a product.
We present a case study which shows how the metric is used at SoftwarePeople. The case study explains some features that have been developed in a Sprint in terms of feature complexity and potential test assessment difficulties and shows how PBR is calculated during the Sprint. We propose a test process assessment metric that provides insights into the Scrum testing process. However, the metric needs further evaluation considering associated resources (e.g., quality assurance engineers, the length of the Scrum cycle).
△ Less
Submitted 25 November, 2014; v1 submitted 9 October, 2013;
originally announced October 2013.
-
A Poisson Hidden Markov Model for Multiview Video Traffic
Authors:
Lorenzo Rossi,
Jacob Chakareski,
Pascal Frossard,
Stefania Colonnese
Abstract:
Multiview video has recently emerged as a means to improve user experience in novel multimedia services. We propose a new stochastic model to characterize the traffic generated by a Multiview Video Coding (MVC) variable bit rate source. To this aim, we resort to a Poisson Hidden Markov Model (P-HMM), in which the first (hidden) layer represents the evolution of the video activity and the second la…
▽ More
Multiview video has recently emerged as a means to improve user experience in novel multimedia services. We propose a new stochastic model to characterize the traffic generated by a Multiview Video Coding (MVC) variable bit rate source. To this aim, we resort to a Poisson Hidden Markov Model (P-HMM), in which the first (hidden) layer represents the evolution of the video activity and the second layer represents the frame sizes of the multiple encoded views. We propose a method for estimating the model parameters in long MVC sequences. We then present extensive numerical simulations assessing the model's ability to produce traffic with realistic characteristics for a general class of MVC sequences. We then extend our framework to network applications where we show that our model is able to accurately describe the sender and receiver buffers behavior in MVC transmission. Finally, we derive a model of user behavior for interactive view selection, which, in conjunction with our traffic model, is able to accurately predict actual network load in interactive multiview services.
△ Less
Submitted 2 January, 2013;
originally announced January 2013.
-
A note on the data-driven capacity of P2P networks
Authors:
Jacob Chakareski,
Pascal Frossard,
Hervé Kerivin,
Jimmy Leblet,
Gwendal Simon
Abstract:
We consider two capacity problems in P2P networks. In the first one, the nodes have an infinite amount of data to send and the goal is to optimally allocate their uplink bandwidths such that the demands of every peer in terms of receiving data rate are met. We solve this problem through a mapping from a node-weighted graph featuring two labels per node to a max flow problem on an edge-weighted b…
▽ More
We consider two capacity problems in P2P networks. In the first one, the nodes have an infinite amount of data to send and the goal is to optimally allocate their uplink bandwidths such that the demands of every peer in terms of receiving data rate are met. We solve this problem through a mapping from a node-weighted graph featuring two labels per node to a max flow problem on an edge-weighted bipartite graph. In the second problem under consideration, the resource allocation is driven by the availability of the data resource that the peers are interested in sharing. That is a node cannot allocate its uplink resources unless it has data to transmit first. The problem of uplink bandwidth allocation is then equivalent to constructing a set of directed trees in the overlay such that the number of nodes receiving the data is maximized while the uplink capacities of the peers are not exceeded. We show that the problem is NP-complete, and provide a linear programming decomposition decoupling it into a master problem and multiple slave subproblems that can be resolved in polynomial time. We also design a heuristic algorithm in order to compute a suboptimal solution in a reasonable time. This algorithm requires only a local knowledge from nodes, so it should support distributed implementations.
We analyze both problems through a series of simulation experiments featuring different network sizes and network densities. On large networks, we compare our heuristic and its variants with a genetic algorithm and show that our heuristic computes the better resource allocation. On smaller networks, we contrast these performances to that of the exact algorithm and show that resource allocation fulfilling a large part of the peer can be found, even for hard configuration where no resources are in excess.
△ Less
Submitted 16 September, 2009;
originally announced September 2009.
