-
Video Summarisation with Incident and Context Information using Generative AI
Authors:
Ulindu De Silva,
Leon Fernando,
Kalinga Bandara,
Rashmika Nawaratne
Abstract:
The proliferation of video content production has led to vast amounts of data, posing substantial challenges in terms of analysis efficiency and resource utilization. Addressing this issue calls for the development of robust video analysis tools. This paper proposes a novel approach leveraging Generative Artificial Intelligence (GenAI) to facilitate streamlined video analysis. Our tool aims to del…
▽ More
The proliferation of video content production has led to vast amounts of data, posing substantial challenges in terms of analysis efficiency and resource utilization. Addressing this issue calls for the development of robust video analysis tools. This paper proposes a novel approach leveraging Generative Artificial Intelligence (GenAI) to facilitate streamlined video analysis. Our tool aims to deliver tailored textual summaries of user-defined queries, offering a focused insight amidst extensive video datasets. Unlike conventional frameworks that offer generic summaries or limited action recognition, our method harnesses the power of GenAI to distil relevant information, enhancing analysis precision and efficiency. Employing YOLO-V8 for object detection and Gemini for comprehensive video and text analysis, our solution achieves heightened contextual accuracy. By combining YOLO with Gemini, our approach furnishes textual summaries extracted from extensive CCTV footage, enabling users to swiftly navigate and verify pertinent events without the need for exhaustive manual review. The quantitative evaluation revealed a similarity of 72.8%, while the qualitative assessment rated an accuracy of 85%, demonstrating the capability of the proposed method.
△ Less
Submitted 8 January, 2025;
originally announced January 2025.
-
Test-Time Optimization for Domain Adaptive Open Vocabulary Segmentation
Authors:
Ulindu De Silva,
Didula Samaraweera,
Sasini Wanigathunga,
Kavindu Kariyawasam,
Kanchana Ranasinghe,
Muzammal Naseer,
Ranga Rodrigo
Abstract:
We present Seg-TTO, a novel framework for zero-shot, open-vocabulary semantic segmentation (OVSS), designed to excel in specialized domain tasks. While current open-vocabulary approaches show impressive performance on standard segmentation benchmarks under zero-shot settings, they fall short of supervised counterparts on highly domain-specific datasets. We focus on segmentation-specific test-time…
▽ More
We present Seg-TTO, a novel framework for zero-shot, open-vocabulary semantic segmentation (OVSS), designed to excel in specialized domain tasks. While current open-vocabulary approaches show impressive performance on standard segmentation benchmarks under zero-shot settings, they fall short of supervised counterparts on highly domain-specific datasets. We focus on segmentation-specific test-time optimization to address this gap. Segmentation requires an understanding of multiple concepts within a single image while retaining the locality and spatial structure of representations. We propose a novel self-supervised objective adhering to these requirements and use it to align the model parameters with input images at test time. In the textual modality, we learn multiple embeddings for each category to capture diverse concepts within an image, while in the visual modality, we calculate pixel-level losses followed by embedding aggregation operations specific to preserving spatial structure. Our resulting framework termed Seg-TTO is a plug-and-play module. We integrate Seg-TTO with three state-of-the-art OVSS approaches and evaluate across 22 challenging OVSS tasks covering a range of specialized domains. Our Seg-TTO demonstrates clear performance improvements (up to 27% mIoU increase on some datasets) establishing new state-of-the-art. Our code and models will be released publicly.
△ Less
Submitted 8 March, 2025; v1 submitted 8 January, 2025;
originally announced January 2025.
-
Large Language Models for Video Surveillance Applications
Authors:
Ulindu De Silva,
Leon Fernando,
Billy Lau Pik Lik,
Zann Koh,
Sam Conrad Joyce,
Belinda Yuen,
Chau Yuen
Abstract:
The rapid increase in video content production has resulted in enormous data volumes, creating significant challenges for efficient analysis and resource management. To address this, robust video analysis tools are essential. This paper presents an innovative proof of concept using Generative Artificial Intelligence (GenAI) in the form of Vision Language Models to enhance the downstream video anal…
▽ More
The rapid increase in video content production has resulted in enormous data volumes, creating significant challenges for efficient analysis and resource management. To address this, robust video analysis tools are essential. This paper presents an innovative proof of concept using Generative Artificial Intelligence (GenAI) in the form of Vision Language Models to enhance the downstream video analysis process. Our tool generates customized textual summaries based on user-defined queries, providing focused insights within extensive video datasets. Unlike traditional methods that offer generic summaries or limited action recognition, our approach utilizes Vision Language Models to extract relevant information, improving analysis precision and efficiency. The proposed method produces textual summaries from extensive CCTV footage, which can then be stored for an indefinite time in a very small storage space compared to videos, allowing users to quickly navigate and verify significant events without exhaustive manual review. Qualitative evaluations result in 80% and 70% accuracy in temporal and spatial quality and consistency of the pipeline respectively.
△ Less
Submitted 6 January, 2025;
originally announced January 2025.
-
A System Level Performance Evaluation for Superconducting Digital Systems
Authors:
Joyjit Kundu,
Debjyoti Bhattacharjee,
Nathan Josephsen,
Ankit Pokhrel,
Udara De Silva,
Wenzhe Guo,
Steven Van Winckel,
Steven Brebels,
Manu Perumkunnil,
Quentin Herr,
Anna Herr
Abstract:
Superconducting Digital (SCD) technology offers significant potential for enhancing the performance of next generation large scale compute workloads. By leveraging advanced lithography and a 300 mm platform, SCD devices can reduce energy consumption and boost computational power. This paper presents a cross-layer modeling approach to evaluate the system-level performance benefits of SCD architectu…
▽ More
Superconducting Digital (SCD) technology offers significant potential for enhancing the performance of next generation large scale compute workloads. By leveraging advanced lithography and a 300 mm platform, SCD devices can reduce energy consumption and boost computational power. This paper presents a cross-layer modeling approach to evaluate the system-level performance benefits of SCD architectures for Large Language Model (LLM) training and inference. Our findings, based on experimental data and Pulse Conserving Logic (PCL) design principles, demonstrate substantial performance gain in both training and inference. We are, thus, able to convincingly show that the SCD technology can address memory and interconnect limitations of present day solutions for next-generation compute systems.
△ Less
Submitted 13 November, 2024;
originally announced November 2024.
-
The Continuous Electron Beam Accelerator Facility at 12 GeV
Authors:
P. A. Adderley,
S. Ahmed,
T. Allison,
R. Bachimanchi,
K. Baggett,
M. BastaniNejad,
B. Bevins,
M. Bevins,
M. Bickley,
R. M. Bodenstein,
S. A. Bogacz,
M. Bruker,
A. Burrill,
L. Cardman,
J. Creel,
Y. -C. Chao,
G. Cheng,
G. Ciovati,
S. Chattopadhyay,
J. Clark,
W. A. Clemens,
G. Croke,
E. Daly,
G. K. Davis,
J. Delayen
, et al. (114 additional authors not shown)
Abstract:
This review paper describes the energy-upgraded CEBAF accelerator. This superconducting linac has achieved 12 GeV beam energy by adding 11 new high-performance cryomodules containing eighty-eight superconducting cavities that have operated CW at an average accelerating gradient of 20 MV/m. After reviewing the attributes and performance of the previous 6 GeV CEBAF accelerator, we discuss the upgrad…
▽ More
This review paper describes the energy-upgraded CEBAF accelerator. This superconducting linac has achieved 12 GeV beam energy by adding 11 new high-performance cryomodules containing eighty-eight superconducting cavities that have operated CW at an average accelerating gradient of 20 MV/m. After reviewing the attributes and performance of the previous 6 GeV CEBAF accelerator, we discuss the upgraded CEBAF accelerator system in detail with particular attention paid to the new beam acceleration systems. In addition to doubling the acceleration in each linac, the upgrade included improving the beam recirculation magnets, adding more helium cooling capacity to allow the newly installed modules to run cold, adding a new experimental hall, and improving numerous other accelerator components. We review several of the techniques deployed to operate and analyze the accelerator performance, and document system operating experience and performance. In the final portion of the document, we present much of the current planning regarding projects to improve accelerator performance and enhance operating margins, and our plans for ensuring CEBAF operates reliably into the future. For the benefit of potential users of CEBAF, the performance and quality measures for beam delivered to each of the experimental halls is summarized in the appendix.
△ Less
Submitted 29 August, 2024;
originally announced August 2024.
-
A Modular 1D-CNN Architecture for Real-time Digital Pre-distortion
Authors:
Udara De Silva,
Toshiaki Koike-Akino,
Rui Ma,
Ao Yamashita,
Hideyuki Nakamizo
Abstract:
This study reports a novel hardware-friendly modular architecture for implementing one dimensional convolutional neural network (1D-CNN) digital predistortion (DPD) technique to linearize RF power amplifier (PA) real-time.The modular nature of our design enables DPD system adaptation for variable resource and timing constraints.Our work also presents a co-simulation architecture to verify the DPD…
▽ More
This study reports a novel hardware-friendly modular architecture for implementing one dimensional convolutional neural network (1D-CNN) digital predistortion (DPD) technique to linearize RF power amplifier (PA) real-time.The modular nature of our design enables DPD system adaptation for variable resource and timing constraints.Our work also presents a co-simulation architecture to verify the DPD performance with an actual power amplifier hardware-in-the-loop.The experimental results with 100 MHz signals show that the proposed 1D-CNN obtains superior performance compared with other neural network architectures for real-time DPD application.
△ Less
Submitted 18 November, 2021;
originally announced November 2021.
-
A Passive STAR Microwave Circuit for 1-3 GHz Self-Interference Cancellation
Authors:
Udara De Silva,
Sravan Pulipati,
Satheesh Bojja Venkatakrishnan,
Shubhendu Bhardwaj,
Arjuna Madanayake
Abstract:
Simultaneous transmit and receive (STAR) allows full-duplex operation of a radio, which leads to doubled capacity for a given bandwidth. A circulator with high-isolation between transmit and receive ports, and low-loss from the antenna to receive port is typically required for achieving STAR. Conventional circulators do not offer wideband performance. Although wideband circulators have been propos…
▽ More
Simultaneous transmit and receive (STAR) allows full-duplex operation of a radio, which leads to doubled capacity for a given bandwidth. A circulator with high-isolation between transmit and receive ports, and low-loss from the antenna to receive port is typically required for achieving STAR. Conventional circulators do not offer wideband performance. Although wideband circulators have been proposed using parametric, switched delay-line/capacitor, and N-path filter techniques using custom integrated circuits, these magnet-free devices have non-linearity, noise, aliasing, and switching noise injection issues. In this paper, a STAR front-end based on passive linear microwave circuit is proposed. Here, a dummy antenna located inside a miniature RF-silent absorption chamber allows circulator-free STAR using simple COTS components. The proposed approach is highly-linear, free from noise, does not require switching or parametric modulation circuits, and has virtually unlimited bandwidth only set by the performance of COTS passive microwave components. The trade-off is relatively large size of the miniature RF-shielded chamber, making this suitable for base-station side applications. Preliminary results show the measured performance of Tx/Rx isolation between 25-60 dB in the 1.0-3.0 GHz range, and 50-60 dB for the 2.4-2.7 GHz range.
△ Less
Submitted 17 August, 2020; v1 submitted 3 August, 2020;
originally announced August 2020.
-
A Direct- Conversion Digital Beamforming Array Receiver with 800 MHz Channel Bandwidth at 28 GHz using Xilinx RF SoC
Authors:
Sravan Pulipati,
Viduneth Ariyarathna,
Udara De Silva,
Najath Akram,
Elias Alwan,
Arjuna Madanayake,
Soumyajit Mandal,
Theodore S. Rappaport
Abstract:
This paper discusses early results associated with a fully-digital direct-conversion array receiver at 28~GHz. The proposed receiver makes use of commercial off-the-shelf (COTS) electronics, including the receiver chain. The design consists of a custom 28~GHz patch antenna sub-array providing gain in the elevation plane, with azimuthal plane beamforming provided by real-time digital signal process…
▽ More
This paper discusses early results associated with a fully-digital direct-conversion array receiver at 28~GHz. The proposed receiver makes use of commercial off-the-shelf (COTS) electronics, including the receiver chain. The design consists of a custom 28~GHz patch antenna sub-array providing gain in the elevation plane, with azimuthal plane beamforming provided by real-time digital signal processing (DSP) algorithms running on a Xilinx Radio Frequency System on Chip (RF SoC). The proposed array receiver employs element-wise fully-digital array processing that supports ADC sample rates up to 2~GS/second and up to 1~GHz of operating bandwidth per antenna. The RF mixed-signal data conversion circuits and DSP algorithms operate on a single-chip RF SoC solution installed on the Xilinx ZCU1275 prototyping platform.
△ Less
Submitted 20 November, 2019;
originally announced November 2019.
-
Determination of the magnetic field dependence of the surface resistance of superconductors from cavity tests
Authors:
J. R. Delayen,
H. Park,
S. U. De Silva,
G. Ciovati,
Z. Li
Abstract:
We present a general method to derive the magnetic field dependence of the surface resistance of superconductors from the Q-curves obtained during the cryogenic tests of cavities. The results are applied to coaxial half-wave cavities, TM-like "elliptical" accelerating cavities, and cavities of more complicated geometries.
We present a general method to derive the magnetic field dependence of the surface resistance of superconductors from the Q-curves obtained during the cryogenic tests of cavities. The results are applied to coaxial half-wave cavities, TM-like "elliptical" accelerating cavities, and cavities of more complicated geometries.
△ Less
Submitted 14 December, 2018;
originally announced December 2018.
-
Deploying an Information Centric Smart Lighting System in the Wild
Authors:
Upeka De Silva,
Adisorn Lertsinsrubtavee,
Arjuna Sathiaseelan,
Carlos Molina-Jimenez,
Kanchana Kanchanasut
Abstract:
In this paper, we present a NDN based smart home lighting solution where lights are automatically controlled in near real time based on occupancy and daylight. We implemented a reliable solution using NDN architecture exploiting the primitive NDN features in push based data dissemination, multicast forwarding through name prefixes and Interest filtering in application layer. Performance was evalua…
▽ More
In this paper, we present a NDN based smart home lighting solution where lights are automatically controlled in near real time based on occupancy and daylight. We implemented a reliable solution using NDN architecture exploiting the primitive NDN features in push based data dissemination, multicast forwarding through name prefixes and Interest filtering in application layer. Performance was evaluated benchmarking with respect to a cloud based approach in terms of message delivery latency. Scalability of the solution was also analyzed presenting an analysis on FIB scalability based on Interest filtering. Finally, from this study, we recommend and highlight a few requirements that could improve NDN for sustainable IoT applications.
△ Less
Submitted 19 July, 2016;
originally announced July 2016.
