Showing 1–10 of 10 results for author: Krishnaswamy, H

Towards a Multi-Agent Vision-Language System for Zero-Shot Novel Hazardous Object Detection for Autonomous Driving Safety

Authors: Shashank Shriram, Srinivasa Perisetla, Aryan Keskar, Harsha Krishnaswamy, Tonko Emil Westerhof Bossen, Andreas Møgelmose, Ross Greer

Abstract: Detecting anomalous hazards in visual data, particularly in video streams, is a critical challenge in autonomous driving. Existing models often struggle with unpredictable, out-of-label hazards due to their reliance on predefined object categories. In this paper, we propose a multimodal approach that integrates vision-language reasoning with zero-shot object detection to improve hazard identificat… ▽ More Detecting anomalous hazards in visual data, particularly in video streams, is a critical challenge in autonomous driving. Existing models often struggle with unpredictable, out-of-label hazards due to their reliance on predefined object categories. In this paper, we propose a multimodal approach that integrates vision-language reasoning with zero-shot object detection to improve hazard identification and explanation. Our pipeline consists of a Vision-Language Model (VLM), a Large Language Model (LLM), in order to detect hazardous objects within a traffic scene. We refine object detection by incorporating OpenAI's CLIP model to match predicted hazards with bounding box annotations, improving localization accuracy. To assess model performance, we create a ground truth dataset by denoising and extending the foundational COOOL (Challenge-of-Out-of-Label) anomaly detection benchmark dataset with complete natural language descriptions for hazard annotations. We define a means of hazard detection and labeling evaluation on the extended dataset using cosine similarity. This evaluation considers the semantic similarity between the predicted hazard description and the annotated ground truth for each video. Additionally, we release a set of tools for structuring and managing large-scale hazard detection datasets. Our findings highlight the strengths and limitations of current vision-language-based approaches, offering insights into future improvements in autonomous hazard detection systems. Our models, scripts, and data can be found at https://github.com/mi3labucm/COOOLER.git △ Less

Submitted 17 April, 2025; originally announced April 2025.

doScenes: An Autonomous Driving Dataset with Natural Language Instruction for Human Interaction and Vision-Language Navigation

Authors: Parthib Roy, Srinivasa Perisetla, Shashank Shriram, Harsha Krishnaswamy, Aryan Keskar, Ross Greer

Abstract: Human-interactive robotic systems, particularly autonomous vehicles (AVs), must effectively integrate human instructions into their motion planning. This paper introduces doScenes, a novel dataset designed to facilitate research on human-vehicle instruction interactions, focusing on short-term directives that directly influence vehicle motion. By annotating multimodal sensor data with natural lang… ▽ More Human-interactive robotic systems, particularly autonomous vehicles (AVs), must effectively integrate human instructions into their motion planning. This paper introduces doScenes, a novel dataset designed to facilitate research on human-vehicle instruction interactions, focusing on short-term directives that directly influence vehicle motion. By annotating multimodal sensor data with natural language instructions and referentiality tags, doScenes bridges the gap between instruction and driving response, enabling context-aware and adaptive planning. Unlike existing datasets that focus on ranking or scene-level reasoning, doScenes emphasizes actionable directives tied to static and dynamic scene objects. This framework addresses limitations in prior research, such as reliance on simulated data or predefined action sets, by supporting nuanced and flexible responses in real-world scenarios. This work lays the foundation for developing learning strategies that seamlessly integrate human instructions into autonomous systems, advancing safe and effective human-vehicle collaboration for vision-language navigation. We make our data publicly available at https://www.github.com/rossgreer/doScenes △ Less

Submitted 8 December, 2024; originally announced December 2024.

Design and Testbed Deployment of Frequency-Domain Equalization Full Duplex Radios

Authors: Manav Kohli, Mahmood Baraani Dastjerdi, Jin Zhou, Ivan Seskar, Harish Krishnaswamy, Gil Zussman, Tingjun Chen

Abstract: Full-duplex (FD) wireless can significantly enhance spectrum efficiency but requires effective self-interference (SI) cancellers. RF SI cancellation (SIC) via frequency-domain equalization (FDE), where bandpass filters channelize the SI, is suited for integrated circuits (ICs). In this paper, we explore the limits and higher layer challenges associated with using such cancellers. We evaluate the p… ▽ More Full-duplex (FD) wireless can significantly enhance spectrum efficiency but requires effective self-interference (SI) cancellers. RF SI cancellation (SIC) via frequency-domain equalization (FDE), where bandpass filters channelize the SI, is suited for integrated circuits (ICs). In this paper, we explore the limits and higher layer challenges associated with using such cancellers. We evaluate the performance of a custom FDE-based canceller using two testbeds; one with mobile FD radios and the other with upgraded, static FD radios in the PAWR COSMOS testbed. The latter is a lasting artifact for the research community, alongside a dataset containing baseband waveforms captured on the COSMOS FD radios, facilitating FD-related experimentation at the higher networking layers. We evaluate the performance of the FDE-based FD radios in both testbeds, with experiments showing 95 dB overall achieved SIC (52 dB from RF SIC) across 20 MHz bandwidth, and an average link-level FD rate gain of 1.87x. We also conduct experiments in (i) uplink-downlink networks with inter-user interference, and (ii) heterogeneous networks with half-duplex and FD users. The experimental FD gains in the two types of networks depend on the users' SNR values and the number of FD users, and are 1.14x-1.25x and 1.25x-1.73x, respectively, confirming previous analytical results. △ Less

Submitted 31 January, 2024; originally announced January 2024.

Comments: 13 pages, 22 figures. arXiv admin note: substantial text overlap with arXiv:1812.01126

CMOS Integrated Magnetless Circulators Based on Spatiotemporal Modulation Angular-Momentum Biasing

Authors: Ahmed Kord, Mykhailo Tymchenko, Dimitrios Sounas, Harish Krishnaswamy, Andrea Alù

Abstract: In this paper, we introduce the first integrated circuit (IC) implementation of spatiotemporally modulated angular-momentum (STM-AM) biased magnetless circulators. The design is based on a modified current-mode topology which is less sensitive to parasitics and relies on switched capacitors rather than varactors to achieve the desired modulation, thus reducing the circuit complexity and easing its… ▽ More In this paper, we introduce the first integrated circuit (IC) implementation of spatiotemporally modulated angular-momentum (STM-AM) biased magnetless circulators. The design is based on a modified current-mode topology which is less sensitive to parasitics and relies on switched capacitors rather than varactors to achieve the desired modulation, thus reducing the circuit complexity and easing its chip-scale realization. We analyze the presented circuit and study its performance in the presence of inevitable non-idealities using an in-house so-called composite Floquet scattering matrix (CFSM) numerical method. We also validate the analysis with simulated and measured results using a standard 180 nm CMOS technology, showing good performance. Compared to previous discrete implementations of STM-AM circulators, the presented CMOS chip reduces the form factor by at least an order of magnitude and occupies a total area of only 36 mm2. △ Less

Submitted 25 February, 2019; v1 submitted 19 February, 2019; originally announced February 2019.

Wideband Full-Duplex Wireless via Frequency-Domain Equalization: Design and Experimentation

Authors: Tingjun Chen, Mahmood Baraani Dastjerdi, Jin Zhou, Harish Krishnaswamy, Gil Zussman

Abstract: Full-duplex (FD) wireless can significantly enhance spectrum efficiency but requires tremendous amount of self-interference (SI) cancellation. Recent advances in the RFIC community enabled wideband RF SI cancellation (SIC) in integrated circuits (ICs) via frequency-domain equalization (FDE), where RF filters channelize the SI signal path. Unlike other FD implementations, that mostly rely on delay… ▽ More Full-duplex (FD) wireless can significantly enhance spectrum efficiency but requires tremendous amount of self-interference (SI) cancellation. Recent advances in the RFIC community enabled wideband RF SI cancellation (SIC) in integrated circuits (ICs) via frequency-domain equalization (FDE), where RF filters channelize the SI signal path. Unlike other FD implementations, that mostly rely on delay lines, FDE-based cancellers can be realized in small-form-factor devices. However, the fundamental limits and higher layer challenges associated with these cancellers were not explored yet. Therefore, and in order to support the integration with a software-defined radio (SDR) and to facilitate experimentation in a testbed with several nodes, we design and implement an FDE-based RF canceller on a printed circuit board (PCB). We derive and experimentally validate the PCB canceller model and present a canceller configuration scheme based on an optimization problem. We then extensively evaluate the performance of the FDE-based FD radio in the SDR testbed. Experiments show that it achieves 95dB overall SIC (52dB from RF SIC) across 20MHz bandwidth, and an average link-level FD gain of 1.87x. We also conduct experiments in: (i) uplink-downlink networks with inter-user interference, and (ii) heterogeneous networks with half-duplex and FD users. The experimental FD gains in the two types of networks confirm previous analytical results. They depend on the users' SNR values and the number of FD users, and are 1.14x-1.25x and 1.25x-1.73x, respectively. Finally, we numerically evaluate and compare the RFIC and PCB implementations and study various design tradeoffs. △ Less

Submitted 3 December, 2018; originally announced December 2018.

Comments: To appear in Proc. ACM MobiCom'19

Integrated Conductivity-Modulation-Based RF Magnetic-Free Non-Reciprocal Components: Recent Results and Benchmarking

Authors: Negar Reiskarimian, Aravind Nagulu, Tolga Dinc, Harish Krishnaswamy

Abstract: Achieving non-reciprocity and building nonreciprocal components through spatio-temporal modulation of material properties has attracted a lot of attention in the recent past as an alternative to the more traditional approach of exploiting Faraday rotation in magnetic materials. In this letter, we review recent research on spatio-temporal conductivity-modulation, which enables low-loss, small-footp… ▽ More Achieving non-reciprocity and building nonreciprocal components through spatio-temporal modulation of material properties has attracted a lot of attention in the recent past as an alternative to the more traditional approach of exploiting Faraday rotation in magnetic materials. In this letter, we review recent research on spatio-temporal conductivity-modulation, which enables low-loss, small-footprint, wide-bandwidth and high-power-handling non-reciprocal components operating from radio frequencies (RF) to millimeter-waves (mm-waves) and integrated in a CMOS platform. Four generations of non-reciprocal circulators and circulator-based systems will be reviewed. We will also discuss metrics of performance that are important for wireless applications and standards, and introduce a new antenna (ANT) interface efficiency figure of merit ($η_{ANT}$) to enable a fair comparison between various types of antenna interfaces. △ Less

Submitted 15 May, 2018; originally announced May 2018.

Comments: Submitted to IEEE Antennas AND Wireless Propagation Letters (AWPL)

Analysis and Design of Commutation-Based Circulator-Receivers for Integrated Full-Duplex Wireless

Authors: Negar Reiskarimian, Mahmood Baraani Dastjerdi, Jin Zhou, Harish Krishnaswamy

Abstract: Previously, we presented a non-magnetic, nonreciprocal N-path-filter-based circulator-receiver (circ.-RX) architecture for full-duplex (FD) wireless which merges a commutation-based linear periodically-time-varying (LPTV) non-magnetic circulator with a down-converting mixer and directly provides the baseband (BB) receiver signals at its output, while suppressing the noise contribution of one set o… ▽ More Previously, we presented a non-magnetic, nonreciprocal N-path-filter-based circulator-receiver (circ.-RX) architecture for full-duplex (FD) wireless which merges a commutation-based linear periodically-time-varying (LPTV) non-magnetic circulator with a down-converting mixer and directly provides the baseband (BB) receiver signals at its output, while suppressing the noise contribution of one set of the commutating switches. The architecture also incorporates an on-chip balance network to enhance the transmitter (TX)-receiver (RX) isolation. In this paper, we present a detailed analysis of the architecture, including a noise analysis and an analysis of the effect of the balance network. The analyses are verified by simulation and measurement results of a 65 nm CMOS 750 MHz circulator-receiver prototype. The circulator-receiver can handle up to +8 dBm of TX power, with 8 dB noise figure (NF) and 40 dB average isolation over 20 MHz RF bandwidth (BW). In conjunction with digital self-interference (SI) and its third-order intermodulation (IM3) cancellation, the FD circ.-RX demonstrates 80 dB overall SI suppression for up to +8 dBm TX average output power. The claims are also verified through an FD demonstration where a -50 dBm weak desired received signal is recovered while transmitting a 0 dBm average-power OFDM-like TX signal. △ Less

Submitted 15 May, 2018; originally announced May 2018.

Comments: Submitted to IEEE Journal of Solid-State Circuits (JSSC). This is the initial submission version. The final version has been recently accepted for publication

Non-reciprocal Components Based on Switched Transmission Lines

Authors: Aravind Nagulu, Tolga Dinc, Zhicheng Xiao, Mykhailo Tymchenko, Dimitrios Sounas, Andrea Alù, Harish Krishnaswamy

Abstract: Non-reciprocal components, such as isolators and circulators, are critical to wireless communication and radar applications. Traditionally, non-reciprocal components have been implemented using ferrite materials, which exhibit non-reciprocity under the influence of an external magnetic field. However, ferrite materials cannot be integrated into IC fabrication processes, and consequently are bulky… ▽ More Non-reciprocal components, such as isolators and circulators, are critical to wireless communication and radar applications. Traditionally, non-reciprocal components have been implemented using ferrite materials, which exhibit non-reciprocity under the influence of an external magnetic field. However, ferrite materials cannot be integrated into IC fabrication processes, and consequently are bulky and expensive. In the recent past, there has been strong interest in achieving non-reciprocity in a non-magnetic IC-compatible fashion using spatio-temporal modulation. In this paper, we present a general approach to non-reciprocity based on switched transmission lines. Switched transmission lines enable broadband, lossless and compact non-reciprocity, and a wide range of non-reciprocal functionalities, including non-reciprocal phase shifters, ultra-broadband gyrators and isolators, frequency-conversion isolators, and high-linearity/high-frequency/ultra-broadband circulators. We present a detailed theoretical analysis of the various non-idealities that impact insertion loss and provide design guidelines. The theory is validated by experimental results from discrete-component-based gyrators and isolators, and a 25GHz circulator fabricated in 45nm SOI CMOS technology. △ Less

Submitted 18 March, 2018; originally announced March 2018.

Comments: 17 pages, 24 figures, TMTT, under review

Open-Access Full-Duplex Wireless in the ORBIT Testbed

Authors: Tingjun Chen, Mahmood Baraani Dastjerdi, Guy Farkash, Jin Zhou, Harish Krishnaswamy, Gil Zussman

Abstract: In order to support experimentation with full-duplex (FD) wireless, we recently integrated an open-access FD transceiver in the ORBIT testbed. In this report, we present the design and implementation of the FD transceiver and interfaces, and provide examples and guidelines for experimentation. In particular, an ORBIT node with a National Instruments (NI)/Ettus Research Universal Software Radio Per… ▽ More In order to support experimentation with full-duplex (FD) wireless, we recently integrated an open-access FD transceiver in the ORBIT testbed. In this report, we present the design and implementation of the FD transceiver and interfaces, and provide examples and guidelines for experimentation. In particular, an ORBIT node with a National Instruments (NI)/Ettus Research Universal Software Radio Peripheral (USRP) N210 software-defined radio (SDR) was equipped with the Columbia FlexICoN Gen-1 customized RF self-interference (SI) canceller box. The RF canceller box includes an RF SI canceller that is implemented using discrete components on a printed circuit board (PCB) and achieves 40dB RF SI cancellation across 5MHz bandwidth. We provide an FD transceiver baseline program and present two example FD experiments where 90dB and 85dB overall SI cancellation is achieved for a simple waveform and PSK modulated signals across both the RF and digital domains. We also discuss potential FD wireless experiments that can be conducted based on the implemented open-access FD transceiver and baseline program. △ Less

Submitted 29 May, 2018; v1 submitted 9 January, 2018; originally announced January 2018.

Resource Allocation and Rate Gains in Practical Full-Duplex Systems

Authors: Jelena Marašević, Jin Zhou, Harish Krishnaswamy, Yuan Zhong, Gil Zussman

Abstract: Full-duplex communication has the potential to substantially increase the throughput in wireless networks. However, the benefits of full-duplex are still not well understood. In this paper, we characterize the full-duplex rate gains in both single-channel and multi-channel use cases. For the single-channel case, we quantify the rate gain as a function of the remaining self-interference and SNR val… ▽ More Full-duplex communication has the potential to substantially increase the throughput in wireless networks. However, the benefits of full-duplex are still not well understood. In this paper, we characterize the full-duplex rate gains in both single-channel and multi-channel use cases. For the single-channel case, we quantify the rate gain as a function of the remaining self-interference and SNR values. We also provide a sufficient condition under which the sum of uplink and downlink rates on a full-duplex channel is concave in the transmission power levels. Building on these results, we consider the multi-channel case. For that case, we introduce a new realistic model of a small form-factor (e.g., smartphone) full-duplex receiver and demonstrate its accuracy via measurements. We study the problem of jointly allocating power levels to different channels and selecting the frequency of maximum self-interference suppression, where the objective is maximizing the sum of the rates over uplink and downlink OFDM channels. We develop a polynomial time algorithm which is nearly optimal in practice under very mild restrictions. To reduce the running time, we develop an efficient nearly-optimal algorithm under the high SINR approximation. Finally, we demonstrate via numerical evaluations the capacity gains in the different use cases and obtain insights into the impact of the remaining self-interference and wireless channel states on the performance. △ Less

Submitted 8 June, 2016; v1 submitted 27 March, 2015; originally announced March 2015.

Comments: Partial and preliminary version appeared in ACM SIGMETRICS'15. A shorter version of v2 is to appear in IEEE/ACM Transactions on Networking