-
LangDA: Building Context-Awareness via Language for Domain Adaptive Semantic Segmentation
Authors:
Chang Liu,
Bavesh Balaji,
Saad Hossain,
C Thomas,
Kwei-Herng Lai,
Raviteja Vemulapalli,
Alexander Wong,
Sirisha Rambhatla
Abstract:
Unsupervised domain adaptation for semantic segmentation (DASS) aims to transfer knowledge from a label-rich source domain to a target domain with no labels. Two key approaches in DASS are (1) vision-only approaches using masking or multi-resolution crops, and (2) language-based approaches that use generic class-wise prompts informed by target domain (e.g. "a {snowy} photo of a {class}"). However,…
▽ More
Unsupervised domain adaptation for semantic segmentation (DASS) aims to transfer knowledge from a label-rich source domain to a target domain with no labels. Two key approaches in DASS are (1) vision-only approaches using masking or multi-resolution crops, and (2) language-based approaches that use generic class-wise prompts informed by target domain (e.g. "a {snowy} photo of a {class}"). However, the former is susceptible to noisy pseudo-labels that are biased to the source domain. The latter does not fully capture the intricate spatial relationships of objects -- key for dense prediction tasks. To this end, we propose LangDA. LangDA addresses these challenges by, first, learning contextual relationships between objects via VLM-generated scene descriptions (e.g. "a pedestrian is on the sidewalk, and the street is lined with buildings."). Second, LangDA aligns the entire image features with text representation of this context-aware scene caption and learns generalized representations via text. With this, LangDA sets the new state-of-the-art across three DASS benchmarks, outperforming existing methods by 2.6%, 1.4% and 3.9%.
△ Less
Submitted 16 March, 2025;
originally announced March 2025.
-
Reinforcement Learning based dynamic weighing of Ensemble Models for Time Series Forecasting
Authors:
Satheesh K. Perepu,
Bala Shyamala Balaji,
Hemanth Kumar Tanneru,
Sudhakar Kathari,
Vivek Shankar Pinnamaraju
Abstract:
Ensemble models are powerful model building tools that are developed with a focus to improve the accuracy of model predictions. They find applications in time series forecasting in varied scenarios including but not limited to process industries, health care, and economics where a single model might not provide optimal performance. It is known that if models selected for data modelling are distinc…
▽ More
Ensemble models are powerful model building tools that are developed with a focus to improve the accuracy of model predictions. They find applications in time series forecasting in varied scenarios including but not limited to process industries, health care, and economics where a single model might not provide optimal performance. It is known that if models selected for data modelling are distinct (linear/non-linear, static/dynamic) and independent (minimally correlated models), the accuracy of the predictions is improved. Various approaches suggested in the literature to weigh the ensemble models use a static set of weights. Due to this limitation, approaches using a static set of weights for weighing ensemble models cannot capture the dynamic changes or local features of the data effectively. To address this issue, a Reinforcement Learning (RL) approach to dynamically assign and update weights of each of the models at different time instants depending on the nature of data and the individual model predictions is proposed in this work. The RL method implemented online, essentially learns to update the weights and reduce the errors as the time progresses. Simulation studies on time series data showed that the dynamic weighted approach using RL learns the weight better than existing approaches. The accuracy of the proposed method is compared with an existing approach of online Neural Network tuning quantitatively through normalized mean square error(NMSE) values.
△ Less
Submitted 20 August, 2020;
originally announced August 2020.
-
Scaling Configuration of Energy Harvesting Sensors with Reinforcement Learning
Authors:
Francesco Fraternali,
Bharathan Balaji,
Rajesh Gupta
Abstract:
With the advent of the Internet of Things (IoT), an increasing number of energy harvesting methods are being used to supplement or supplant battery based sensors. Energy harvesting sensors need to be configured according to the application, hardware, and environmental conditions to maximize their usefulness. As of today, the configuration of sensors is either manual or heuristics based, requiring…
▽ More
With the advent of the Internet of Things (IoT), an increasing number of energy harvesting methods are being used to supplement or supplant battery based sensors. Energy harvesting sensors need to be configured according to the application, hardware, and environmental conditions to maximize their usefulness. As of today, the configuration of sensors is either manual or heuristics based, requiring valuable domain expertise. Reinforcement learning (RL) is a promising approach to automate configuration and efficiently scale IoT deployments, but it is not yet adopted in practice. We propose solutions to bridge this gap: reduce the training phase of RL so that nodes are operational within a short time after deployment and reduce the computational requirements to scale to large deployments. We focus on configuration of the sampling rate of indoor solar panel based energy harvesting sensors. We created a simulator based on 3 months of data collected from 5 sensor nodes subject to different lighting conditions. Our simulation results show that RL can effectively learn energy availability patterns and configure the sampling rate of the sensor nodes to maximize the sensing data while ensuring that energy storage is not depleted. The nodes can be operational within the first day by using our methods. We show that it is possible to reduce the number of RL policies by using a single policy for nodes that share similar lighting conditions.
△ Less
Submitted 27 November, 2018;
originally announced November 2018.
-
Object Recognition and Identification Using ESM Data
Authors:
E. Taghavi,
D. Song,
R. Tharmarasa,
T. Kirubarajan,
Anne-Claire Boury-Brisset,
Bhashyam Balaji
Abstract:
Recognition and identification of unknown targets is a crucial task in surveillance and security systems. Electronic Support Measures (ESM) are one of the most effective sensors for identification, especially for maritime and air--to--ground applications. In typical surveillance systems multiple ESM sensors are usually deployed along with kinematic sensors like radar. Different ESM sensors may pro…
▽ More
Recognition and identification of unknown targets is a crucial task in surveillance and security systems. Electronic Support Measures (ESM) are one of the most effective sensors for identification, especially for maritime and air--to--ground applications. In typical surveillance systems multiple ESM sensors are usually deployed along with kinematic sensors like radar. Different ESM sensors may produce different types of reports ready to be sent to the fusion center. The focus of this paper is to develop a new architecture for target recognition and identification when non--homogeneous ESM and possibly kinematic reports are received at the fusion center. The new fusion architecture is evaluated using simulations to show the benefit of utilizing different ESM reports such as attributes and signal level ESM data.
△ Less
Submitted 22 March, 2016;
originally announced July 2016.
-
Intent Inference and Syntactic Tracking with GMTI Measurements
Authors:
Alex Wang,
Vikram Krishnamurthy,
Bhashyam Balaji
Abstract:
In conventional target tracking systems, human operators use the estimated target tracks to make higher level inference of the target behaviour/intent. This paper develops syntactic filtering algorithms that assist human operators by extracting spatial patterns from target tracks to identify suspicious/anomalous spatial trajectories. The targets' spatial trajectories are modeled by a stochastic co…
▽ More
In conventional target tracking systems, human operators use the estimated target tracks to make higher level inference of the target behaviour/intent. This paper develops syntactic filtering algorithms that assist human operators by extracting spatial patterns from target tracks to identify suspicious/anomalous spatial trajectories. The targets' spatial trajectories are modeled by a stochastic context free grammar (SCFG) and a switched mode state space model. Bayesian filtering algorithms for stochastic context free grammars are presented for extracting the syntactic structure and illustrated for a ground moving target indicator (GMTI) radar example. The performance of the algorithms is tested with the experimental data collected using DRDC Ottawa's X-band Wideband Experimental Airborne Radar (XWEAR).
△ Less
Submitted 21 April, 2011;
originally announced April 2011.
