-
Premier-TACO is a Few-Shot Policy Learner: Pretraining Multitask Representation via Temporal Action-Driven Contrastive Loss
Authors:
Ruijie Zheng,
Yongyuan Liang,
Xiyao Wang,
Shuang Ma,
Hal Daumé III,
Huazhe Xu,
John Langford,
Praveen Palanisamy,
Kalyan Shankar Basu,
Furong Huang
Abstract:
We present Premier-TACO, a multitask feature representation learning approach designed to improve few-shot policy learning efficiency in sequential decision-making tasks. Premier-TACO leverages a subset of multitask offline datasets for pretraining a general feature representation, which captures critical environmental dynamics and is fine-tuned using minimal expert demonstrations. It advances the…
▽ More
We present Premier-TACO, a multitask feature representation learning approach designed to improve few-shot policy learning efficiency in sequential decision-making tasks. Premier-TACO leverages a subset of multitask offline datasets for pretraining a general feature representation, which captures critical environmental dynamics and is fine-tuned using minimal expert demonstrations. It advances the temporal action contrastive learning (TACO) objective, known for state-of-the-art results in visual control tasks, by incorporating a novel negative example sampling strategy. This strategy is crucial in significantly boosting TACO's computational efficiency, making large-scale multitask offline pretraining feasible. Our extensive empirical evaluation in a diverse set of continuous control benchmarks including Deepmind Control Suite, MetaWorld, and LIBERO demonstrate Premier-TACO's effectiveness in pretraining visual representations, significantly enhancing few-shot imitation learning of novel tasks. Our code, pretraining data, as well as pretrained model checkpoints will be released at https://github.com/PremierTACO/premier-taco. Our project webpage is at https://premiertaco.github.io.
△ Less
Submitted 23 May, 2024; v1 submitted 9 February, 2024;
originally announced February 2024.
-
Autonomous Advanced Aerial Mobility -- An End-to-end Autonomy Framework for UAVs and Beyond
Authors:
Sakshi Mishra,
Praveen Palanisamy
Abstract:
Developing aerial robots that can both safely navigate and execute assigned mission without any human intervention - i.e., fully autonomous aerial mobility of passengers and goods - is the larger vision that guides the research, design, and development efforts in the aerial autonomy space. However, it is highly challenging to concurrently operationalize all types of aerial vehicles that are operat…
▽ More
Developing aerial robots that can both safely navigate and execute assigned mission without any human intervention - i.e., fully autonomous aerial mobility of passengers and goods - is the larger vision that guides the research, design, and development efforts in the aerial autonomy space. However, it is highly challenging to concurrently operationalize all types of aerial vehicles that are operating fully autonomously sharing the airspace. Full autonomy of the aerial transportation sector includes several aspects, such as design of the technology that powers the vehicles, operations of multi-agent fleets, and process of certification that meets stringent safety requirements of aviation sector. Thereby, Autonomous Advanced Aerial Mobility is still a vague term and its consequences for researchers and professionals are ambiguous. To address this gap, we present a comprehensive perspective on the emerging field of autonomous advanced aerial mobility, which involves the use of unmanned aerial vehicles (UAVs) and electric vertical takeoff and landing (eVTOL) aircraft for various applications, such as urban air mobility, package delivery, and surveillance. The article proposes a scalable and extensible autonomy framework consisting of four main blocks: sensing, perception, planning, and controls. Furthermore, the article discusses the challenges and opportunities in multi-agent fleet operations and management, as well as the testing, validation, and certification aspects of autonomous aerial systems. Finally, the article explores the potential of monolithic models for aerial autonomy and analyzes their advantages and limitations. The perspective aims to provide a holistic picture of the autonomous advanced aerial mobility field and its future directions.
△ Less
Submitted 2 December, 2023; v1 submitted 8 November, 2023;
originally announced November 2023.
-
Scalable Modular Synthetic Data Generation for Advancing Aerial Autonomy
Authors:
Mehrnaz Sabet,
Praveen Palanisamy,
Sakshi Mishra
Abstract:
One major barrier to advancing aerial autonomy has been collecting large-scale aerial datasets for training machine learning models. Due to costly and time-consuming real-world data collection through deploying drones, there has been an increasing shift towards using synthetic data for training models in drone applications. However, to increase widespread generalization and transferring models to…
▽ More
One major barrier to advancing aerial autonomy has been collecting large-scale aerial datasets for training machine learning models. Due to costly and time-consuming real-world data collection through deploying drones, there has been an increasing shift towards using synthetic data for training models in drone applications. However, to increase widespread generalization and transferring models to real-world, increasing the diversity of simulation environments to train a model over all the varieties and augmenting the training data, has been proved to be essential. Current synthetic aerial data generation tools either lack data augmentation or rely heavily on manual workload or real samples for configuring and generating diverse realistic simulation scenes for data collection. These dependencies limit scalability of the data generation workflow. Accordingly, there is a major challenge in balancing generalizability and scalability in synthetic data generation. To address these gaps, we introduce a scalable Aerial Synthetic Data Augmentation (ASDA) framework tailored to aerial autonomy applications. ASDA extends a central data collection engine with two scriptable pipelines that automatically perform scene and data augmentations to generate diverse aerial datasets for different training tasks. ASDA improves data generation workflow efficiency by providing a unified prompt-based interface over integrated pipelines for flexible control. The procedural generative approach of our data augmentation is performant and adaptable to different simulation environments, training tasks and data collection needs. We demonstrate the effectiveness of our method in automatically generating diverse datasets and show its potential for downstream performance optimization.
△ Less
Submitted 25 May, 2023; v1 submitted 9 November, 2022;
originally announced November 2022.
-
Multi-Agent Connected Autonomous Driving using Deep Reinforcement Learning
Authors:
Praveen Palanisamy
Abstract:
The capability to learn and adapt to changes in the driving environment is crucial for developing autonomous driving systems that are scalable beyond geo-fenced operational design domains. Deep Reinforcement Learning (RL) provides a promising and scalable framework for developing adaptive learning based solutions. Deep RL methods usually model the problem as a (Partially Observable) Markov Decisio…
▽ More
The capability to learn and adapt to changes in the driving environment is crucial for developing autonomous driving systems that are scalable beyond geo-fenced operational design domains. Deep Reinforcement Learning (RL) provides a promising and scalable framework for developing adaptive learning based solutions. Deep RL methods usually model the problem as a (Partially Observable) Markov Decision Process in which an agent acts in a stationary environment to learn an optimal behavior policy. However, driving involves complex interaction between multiple, intelligent (artificial or human) agents in a highly non-stationary environment. In this paper, we propose the use of Partially Observable Markov Games(POSG) for formulating the connected autonomous driving problems with realistic assumptions. We provide a taxonomy of multi-agent learning environments based on the nature of tasks, nature of agents and the nature of the environment to help in categorizing various autonomous driving problems that can be addressed under the proposed formulation. As our main contributions, we provide MACAD-Gym, a Multi-Agent Connected, Autonomous Driving agent learning platform for furthering research in this direction. Our MACAD-Gym platform provides an extensible set of Connected Autonomous Driving (CAD) simulation environments that enable the research and development of Deep RL- based integrated sensing, perception, planning and control algorithms for CAD systems with unlimited operational design domain under realistic, multi-agent settings. We also share the MACAD-Agents that were trained successfully using the MACAD-Gym platform to learn control policies for multiple vehicle agents in a partially observable, stop-sign controlled, 3-way urban intersection environment with raw (camera) sensor observations.
△ Less
Submitted 11 November, 2019;
originally announced November 2019.
-
An Integrated Multi-Time-Scale Modeling for Solar Irradiance Forecasting Using Deep Learning
Authors:
Sakshi Mishra,
Praveen Palanisamy
Abstract:
For short-term solar irradiance forecasting, the traditional point forecasting methods are rendered less useful due to the non-stationary characteristic of solar power. The amount of operating reserves required to maintain reliable operation of the electric grid rises due to the variability of solar energy. The higher the uncertainty in the generation, the greater the operating-reserve requirement…
▽ More
For short-term solar irradiance forecasting, the traditional point forecasting methods are rendered less useful due to the non-stationary characteristic of solar power. The amount of operating reserves required to maintain reliable operation of the electric grid rises due to the variability of solar energy. The higher the uncertainty in the generation, the greater the operating-reserve requirements, which translates to an increased cost of operation. In this research work, we propose a unified architecture for multi-time-scale predictions for intra-day solar irradiance forecasting using recurrent neural networks (RNN) and long-short-term memory networks (LSTMs). This paper also lays out a framework for extending this modeling approach to intra-hour forecasting horizons thus, making it a multi-time-horizon forecasting approach, capable of predicting intra-hour as well as intra-day solar irradiance. We develop an end-to-end pipeline to effectuate the proposed architecture. The performance of the prediction model is tested and validated by the methodical implementation. The robustness of the approach is demonstrated with case studies conducted for geographically scattered sites across the United States. The predictions demonstrate that our proposed unified architecture-based approach is effective for multi-time-scale solar forecasts and achieves a lower root-mean-square prediction error when benchmarked against the best-performing methods documented in the literature that use separate models for each time-scale during the day. Our proposed method results in a 71.5% reduction in the mean RMSE averaged across all the test sites compared to the ML-based best-performing method reported in the literature. Additionally, the proposed method enables multi-time-horizon forecasts with real-time inputs, which have a significant potential for practical industry applications in the evolving grid.
△ Less
Submitted 1 August, 2023; v1 submitted 7 May, 2019;
originally announced May 2019.
-
Learning On-Road Visual Control for Self-Driving Vehicles with Auxiliary Tasks
Authors:
Yilun Chen,
Praveen Palanisamy,
Priyantha Mudalige,
Katharina Muelling,
John M. Dolan
Abstract:
A safe and robust on-road navigation system is a crucial component of achieving fully automated vehicles. NVIDIA recently proposed an End-to-End algorithm that can directly learn steering commands from raw pixels of a front camera by using one convolutional neural network. In this paper, we leverage auxiliary information aside from raw images and design a novel network structure, called Auxiliary…
▽ More
A safe and robust on-road navigation system is a crucial component of achieving fully automated vehicles. NVIDIA recently proposed an End-to-End algorithm that can directly learn steering commands from raw pixels of a front camera by using one convolutional neural network. In this paper, we leverage auxiliary information aside from raw images and design a novel network structure, called Auxiliary Task Network (ATN), to help boost the driving performance while maintaining the advantage of minimal training data and an End-to-End training method. In this network, we introduce human prior knowledge into vehicle navigation by transferring features from image recognition tasks. Image semantic segmentation is applied as an auxiliary task for navigation. We consider temporal information by introducing an LSTM module and optical flow to the network. Finally, we combine vehicle kinematics with a sensor fusion step. We discuss the benefits of our method over state-of-the-art visual navigation methods both in the Udacity simulation environment and on the real-world Comma.ai dataset.
△ Less
Submitted 19 December, 2018;
originally announced December 2018.
-
Multi-time-horizon Solar Forecasting Using Recurrent Neural Network
Authors:
Sakshi Mishra,
Praveen Palanisamy
Abstract:
The non-stationarity characteristic of the solar power renders traditional point forecasting methods to be less useful due to large prediction errors. This results in increased uncertainties in the grid operation, thereby negatively affecting the reliability and increased cost of operation. This research paper proposes a unified architecture for multi-time-horizon predictions for short and long-te…
▽ More
The non-stationarity characteristic of the solar power renders traditional point forecasting methods to be less useful due to large prediction errors. This results in increased uncertainties in the grid operation, thereby negatively affecting the reliability and increased cost of operation. This research paper proposes a unified architecture for multi-time-horizon predictions for short and long-term solar forecasting using Recurrent Neural Networks (RNN). The paper describes an end-to-end pipeline to implement the architecture along with the methods to test and validate the performance of the prediction model. The results demonstrate that the proposed method based on the unified architecture is effective for multi-horizon solar forecasting and achieves a lower root-mean-squared prediction error compared to the previous best-performing methods which use one model for each time-horizon. The proposed method enables multi-horizon forecasts with real-time inputs, which have a high potential for practical applications in the evolving smart grid.
△ Less
Submitted 14 July, 2018;
originally announced July 2018.
-
A Novel Sparse recovery based DOA estimation algorithm by relaxing the RIP constraint
Authors:
Abhishek Aich,
P. Palanisamy
Abstract:
Direction of Arrival (DOA) estimation of mixed uncorrelated and coherent sources is a long existing challenge in array signal processing. Application of compressive sensing to array signal processing has opened up an exciting class of algorithms. The authors investigated the application of orthogonal matching pursuit (OMP) for direction of Arrival (DOA) estimation for different scenarios, especial…
▽ More
Direction of Arrival (DOA) estimation of mixed uncorrelated and coherent sources is a long existing challenge in array signal processing. Application of compressive sensing to array signal processing has opened up an exciting class of algorithms. The authors investigated the application of orthogonal matching pursuit (OMP) for direction of Arrival (DOA) estimation for different scenarios, especially to tackle the case of coherent sources and observed inconsistencies in the results. In this paper, a modified OMP algorithm is proposed to overcome these deficiencies by exploiting maximum variance based criterion using only one snapshot. This criterion relaxes the imposed restricted isometry property (RIP) on the measurement matrix to obtain the sources and hence, reduces the sparsity of the input vector to the local OMP algorithm. Moreover, it also tackles sources irrespective of their coherency. The condition for the weak-1 RIP on decreased sparsity is derived and it is shown that how the algorithm gives better result than the OMP algorithm. With an addition to this, a simple method is also presented to calculate source distance from the reference point in a uniform linear sensor array. Numerical analysis demonstrates the effectiveness of the proposed algorithm.
△ Less
Submitted 7 September, 2018; v1 submitted 25 July, 2017;
originally announced July 2017.
-
A novel CS Beamformer root-MUSIC algorithm and its subspace deviation analysis
Authors:
Abhishek Aich,
P. Palanisamy
Abstract:
Subspace based techniques for direction of arrival (DOA) estimation need large amount of snapshots to detect source directions accurately. This poses a problem in the form of computational burden on practical applications. The introduction of compressive sensing (CS) to solve this issue has become a norm in the last decade. In this paper, a novel CS beamformer root-MUSIC algorithm is presented wit…
▽ More
Subspace based techniques for direction of arrival (DOA) estimation need large amount of snapshots to detect source directions accurately. This poses a problem in the form of computational burden on practical applications. The introduction of compressive sensing (CS) to solve this issue has become a norm in the last decade. In this paper, a novel CS beamformer root-MUSIC algorithm is presented with a revised optimal measurement matrix bound. With regards to this algorithm, the effect of signal subspace deviation under low snapshot scenario (e.g. target tracking) is analysed. The CS beamformer greatly reduces computational complexity without affecting resolution of the algorithm, works on par with root-MUSIC under low snapshot scenario and also, gives an option of non-uniform linear array sensors unlike the case of root-MUSIC algorithm. The effectiveness of the algorithm is demonstrated with simulations under various scenarios.
△ Less
Submitted 27 September, 2017; v1 submitted 25 July, 2017;
originally announced July 2017.
-
On-Grid DOA Estimation Method Using Orthogonal Matching Pursuit
Authors:
Abhishek Aich,
P. Palanisamy
Abstract:
Direction of Arrival (DOA) estimation of multiple narrow-band coherent or partially coherent sources is a major challenge in array signal processing. Though many subspace- based algorithms are available in literature, none of them tackle the problem of resolving coherent sources directly, e.g. without modifying the sample data covariance matrix. Compressive Sensing (CS) based sparse recovery algor…
▽ More
Direction of Arrival (DOA) estimation of multiple narrow-band coherent or partially coherent sources is a major challenge in array signal processing. Though many subspace- based algorithms are available in literature, none of them tackle the problem of resolving coherent sources directly, e.g. without modifying the sample data covariance matrix. Compressive Sensing (CS) based sparse recovery algorithms are being applied as a novel technique to this area. In this paper, we introduce Orthogonal Matching Pursuit (OMP) to the DOA estimation problem. We demonstrate how a DOA estimation problem can be modelled for sparse recovery problem and then exploited using OMP to obtain the set of DOAs. Moreover, this algorithm uses only one snapshot to obtain the results. The simulation results demonstrate the validity and advantages of using OMP algorithm over the existing subspace- based algorithms.
△ Less
Submitted 25 January, 2018; v1 submitted 29 March, 2017;
originally announced May 2017.
-
On application of OMP and CoSaMP algorithms for DOA estimation problem
Authors:
Abhishek Aich,
P. Palanisamy
Abstract:
Remarkable properties of Compressed sensing (CS) has led researchers to utilize it in various other fields where a solution to an underdetermined system of linear equations is needed. One such application is in the area of array signal processing e.g. in signal denoising and Direction of Arrival (DOA) estimation. From the two prominent categories of CS recovery algorithms, namely convex optimizati…
▽ More
Remarkable properties of Compressed sensing (CS) has led researchers to utilize it in various other fields where a solution to an underdetermined system of linear equations is needed. One such application is in the area of array signal processing e.g. in signal denoising and Direction of Arrival (DOA) estimation. From the two prominent categories of CS recovery algorithms, namely convex optimization algorithms and greedy sparse approximation algorithms, we investigate the application of greedy sparse approximation algorithms to estimate DOA in the uniform linear array (ULA) environment. We conduct an empirical investigation into the behavior of the two state-of-the-art greedy algorithms: OMP and CoSaMP. This investigation takes into account the various scenarios such as varying degrees of noise level and coherency between the sources. We perform simulations to demonstrate the performances of these algorithms and give a brief analysis of the results.
△ Less
Submitted 25 January, 2018; v1 submitted 23 March, 2017;
originally announced April 2017.
