-
CAST: Contrastive Adaptation and Distillation for Semi-Supervised Instance Segmentation
Authors:
Pardis Taghavi,
Tian Liu,
Renjie Li,
Reza Langari,
Zhengzhong Tu
Abstract:
Instance segmentation demands costly per-pixel annotations and large models. We introduce CAST, a semi-supervised knowledge distillation (SSKD) framework that compresses pretrained vision foundation models (VFM) into compact experts using limited labeled and abundant unlabeled data. CAST unfolds in three stages: (1) domain adaptation of the VFM teacher(s) via self-training with contrastive pixel c…
▽ More
Instance segmentation demands costly per-pixel annotations and large models. We introduce CAST, a semi-supervised knowledge distillation (SSKD) framework that compresses pretrained vision foundation models (VFM) into compact experts using limited labeled and abundant unlabeled data. CAST unfolds in three stages: (1) domain adaptation of the VFM teacher(s) via self-training with contrastive pixel calibration, (2) distillation into a compact student via a unified multi-objective loss that couples standard supervision and pseudo-labels with our instance-aware pixel-wise contrastive term, and (3) fine-tuning on labeled data to remove residual pseudo-label bias. Central to CAST is an \emph{instance-aware pixel-wise contrastive loss} that fuses mask and class scores to mine informative negatives and enforce clear inter-instance margins. By maintaining this contrastive signal across both adaptation and distillation, we align teacher and student embeddings and fully leverage unlabeled images. On Cityscapes and ADE20K, our ~11X smaller student surpasses its adapted VFM teacher(s) by +3.4 AP (33.9 vs. 30.5) and +1.5 AP (16.7 vs. 15.2) and outperforms state-of-the-art semi-supervised approaches.
△ Less
Submitted 7 June, 2025; v1 submitted 27 May, 2025;
originally announced May 2025.
-
SwinMTL: A Shared Architecture for Simultaneous Depth Estimation and Semantic Segmentation from Monocular Camera Images
Authors:
Pardis Taghavi,
Reza Langari,
Gaurav Pandey
Abstract:
This research paper presents an innovative multi-task learning framework that allows concurrent depth estimation and semantic segmentation using a single camera. The proposed approach is based on a shared encoder-decoder architecture, which integrates various techniques to improve the accuracy of the depth estimation and semantic segmentation task without compromising computational efficiency. Add…
▽ More
This research paper presents an innovative multi-task learning framework that allows concurrent depth estimation and semantic segmentation using a single camera. The proposed approach is based on a shared encoder-decoder architecture, which integrates various techniques to improve the accuracy of the depth estimation and semantic segmentation task without compromising computational efficiency. Additionally, the paper incorporates an adversarial training component, employing a Wasserstein GAN framework with a critic network, to refine model's predictions. The framework is thoroughly evaluated on two datasets - the outdoor Cityscapes dataset and the indoor NYU Depth V2 dataset - and it outperforms existing state-of-the-art methods in both segmentation and depth estimation tasks. We also conducted ablation studies to analyze the contributions of different components, including pre-training strategies, the inclusion of critics, the use of logarithmic depth scaling, and advanced image augmentations, to provide a better understanding of the proposed framework. The accompanying source code is accessible at \url{https://github.com/PardisTaghavi/SwinMTL}.
△ Less
Submitted 15 March, 2024;
originally announced March 2024.
-
Real-Time Semantic Segmentation using Hyperspectral Images for Mapping Unstructured and Unknown Environments
Authors:
Anthony Medellin,
Anant Bhamri,
Reza Langari,
Swaminathan Gopalswamy
Abstract:
Autonomous navigation in unstructured off-road environments is greatly improved by semantic scene understanding. Conventional image processing algorithms are difficult to implement and lack robustness due to a lack of structure and high variability across off-road environments. The use of neural networks and machine learning can overcome the previous challenges but they require large labeled data…
▽ More
Autonomous navigation in unstructured off-road environments is greatly improved by semantic scene understanding. Conventional image processing algorithms are difficult to implement and lack robustness due to a lack of structure and high variability across off-road environments. The use of neural networks and machine learning can overcome the previous challenges but they require large labeled data sets for training. In our work we propose the use of hyperspectral images for real-time pixel-wise semantic classification and segmentation, without the need of any prior training data. The resulting segmented image is processed to extract, filter, and approximate objects as polygons, using a polygon approximation algorithm. The resulting polygons are then used to generate a semantic map of the environment. Using our framework. we show the capability to add new semantic classes in run-time for classification. The proposed methodology is also shown to operate in real-time and produce outputs at a frequency of 1Hz, using high resolution hyperspectral images.
△ Less
Submitted 27 March, 2023;
originally announced March 2023.
-
Separable Tendon-Driven Robotic Manipulator with a Long, Flexible, Passive Proximal Section
Authors:
Christian DeBuys,
Florin C. Ghesu,
Jagadeesan Jayender,
Reza Langari,
Young-Ho Kim
Abstract:
This work tackles practical issues which arise when using a tendon-driven robotic manipulator (TDRM) with a long, flexible, passive proximal section in medical applications. Tendon-driven devices are preferred in medicine for their improved outcomes via minimally invasive procedures, but TDRMs come with unique challenges such as sterilization and reuse, simultaneous control of tendons, hysteresis…
▽ More
This work tackles practical issues which arise when using a tendon-driven robotic manipulator (TDRM) with a long, flexible, passive proximal section in medical applications. Tendon-driven devices are preferred in medicine for their improved outcomes via minimally invasive procedures, but TDRMs come with unique challenges such as sterilization and reuse, simultaneous control of tendons, hysteresis in the tendon-sheath mechanism, and unmodeled effects of the proximal section shape. A separable TDRM which overcomes difficulties in actuation and sterilization is introduced, in which the body containing the electronics is reusable and the remainder is disposable. An open-loop redundant controller which resolves the redundancy in the kinematics is developed. Simple linear hysteresis compensation and re-tension compensation based on the physical properties of the device are proposed. The controller and compensation methods are evaluated on a testbed for a straight proximal section, a curved proximal section at various static angles, and a proximal section which dynamically changes angles; and overall, distal tip error was reduced.
△ Less
Submitted 18 April, 2023; v1 submitted 31 December, 2022;
originally announced January 2023.
-
Design and validation of zero-slack separable manipulator for Intracardiac Echocardiography
Authors:
Christian DeBuy,
Florin Ghesu,
Reza Langari,
Young-Ho Kim
Abstract:
Clinicians require substantial training and experience to become comfortable with steering Intracardiac echocardiography (ICE) catheter to localize and measure the area of treatment to watch for complications while device catheters are deployed in another access. Thus, it is reasonable that a robotic-assist system to hold and actively manipulate the ICE catheter could ease the workload of the phys…
▽ More
Clinicians require substantial training and experience to become comfortable with steering Intracardiac echocardiography (ICE) catheter to localize and measure the area of treatment to watch for complications while device catheters are deployed in another access. Thus, it is reasonable that a robotic-assist system to hold and actively manipulate the ICE catheter could ease the workload of the physician. Existing commercially-available robotic systems and research prototypes all use existing commercially available ICE catheters based on multiple tendon-sheath mechanism (TSM). To motorize the existing TSM-based ICE catheter, the actuators interface with the outer handle knobs to manipulate four internal tendons. However, in practice, the actuators are located at a sterile, safe place far away from the ICE handle. Thus, to interface with knobs, there exist multiple coupled gear structures between two, leading to a highly nonlinear behavior (e.g. various slack, elasticity) alongside hysteresis phenomena in TSM.
Since ICE catheters are designed for single use, the expensive actuators need to be located in a safe place so as to be reusable. Moreover, these actuators should interface as directly as possible with the tendons for accurate tip controls. In this paper, we introduce a separable ICE catheter robot with four tendon actuation: one part reusable and another disposable. Moreover, we propose a practical model and calibration method for our proposed mechanism so that four tendons are actuated simultaneously allowing for precise tip control and mitigating issues with conventional devices such as dead-zone and hysteresis with simple linear compensation. We consider an open-loop controller since many available ICE catheters are used without position-tracking sensors at the tip due to costs and single use
△ Less
Submitted 1 April, 2022;
originally announced April 2022.
-
A Stackelberg Game Theoretic Model of Lane-Merging
Authors:
Jehong Yoo,
Reza Langari
Abstract:
Merging in the form of a mandatory lane-change is an important issue in transportation research. Even when safely completed, merging may disturb the mainline traffic and reduce the efficiency or capacity of the roadway. In this paper, we consider a Stackelberg game-theoretic driver behavior model where the so-called utilities or payoffs reflect the merging vehicle's aggressiveness as it pertains t…
▽ More
Merging in the form of a mandatory lane-change is an important issue in transportation research. Even when safely completed, merging may disturb the mainline traffic and reduce the efficiency or capacity of the roadway. In this paper, we consider a Stackelberg game-theoretic driver behavior model where the so-called utilities or payoffs reflect the merging vehicle's aggressiveness as it pertains the decision to merge as the situation stands or to accelerate, decelerate prior to the actual lane-change maneuver. The interaction of the merging vehicle with the mainline traffic is also considered whereby the combination of aggressiveness of the respective vehicles leads to both longitudinal and lateral disturbances to the mainline flow as well as subsequent reduction in the roadway throughput. The present study shows in semi-quantitative form that this impact depends on the level of aggressiveness of the merging and mainline vehicles, leading to the potential use of this model in traffic flow analysis and autonomous driving.
△ Less
Submitted 21 March, 2020;
originally announced March 2020.
-
A Game-Theoretic Model of Human Driving and Application to Discretionary Lane-Changes
Authors:
Jehong Yoo,
Reza Langari
Abstract:
In this paper we consider the application of Stackelberg game theory to model discretionary lane-changing in lightly congested highway setting. The fundamental intent of this model, which is parameterized to capture driver disposition (aggressiveness or inattentiveness), is to help with the development of decision-making strategies for autonomous vehicles in ways that are mindful of how human driv…
▽ More
In this paper we consider the application of Stackelberg game theory to model discretionary lane-changing in lightly congested highway setting. The fundamental intent of this model, which is parameterized to capture driver disposition (aggressiveness or inattentiveness), is to help with the development of decision-making strategies for autonomous vehicles in ways that are mindful of how human drivers perform the same function on the road (on which have reported elsewhere.) This paper, however, focuses only on the model development and the respective qualitative assessment. This is accomplished in unit test simulations as well as in bulk mode (i.e. using the Monte Carlo methodology), via a limited traffic micro-simulation compared against the NHTSA 100-Car Naturalistic Driving Safety data. In particular, a qualitative comparison shows the relative consistency of the proposed model with human decision-making in terms of producing qualitatively similar proportions of crashes and near crashes as a function of driver inattentiveness (or aggressiveness). While this result by itself does not offer a true quantitative validation of the proposed model, it does demonstrate the utility of the proposed approach in modeling discretionary lane-changing and may therefore be of use in autonomous driving in a manner that is consistent with human decision making on the road.
△ Less
Submitted 21 March, 2020;
originally announced March 2020.
-
A Computational Approach for Human-like Motion Generation in Upper Limb Exoskeletons Supporting Scapulohumeral Rhythms
Authors:
Rana Soltani-Zarrin,
Amin Zeiaee,
Reza Langari,
Reza Tafreshi
Abstract:
This paper proposes a computational approach for generation of reference path for upper-limb exoskeletons considering the scapulohumeral rhythms of the shoulder. The proposed method can be used in upper-limb exoskeletons with 3 Degrees of Freedom (DoF) in shoulder and 1 DoF in elbow, which are capable of supporting shoulder girdle. The developed computational method is based on Central Nervous Sys…
▽ More
This paper proposes a computational approach for generation of reference path for upper-limb exoskeletons considering the scapulohumeral rhythms of the shoulder. The proposed method can be used in upper-limb exoskeletons with 3 Degrees of Freedom (DoF) in shoulder and 1 DoF in elbow, which are capable of supporting shoulder girdle. The developed computational method is based on Central Nervous System (CNS) governing rules. Existing computational reference generation methods are based on the assumption of fixed shoulder center during motions. This assumption can be considered valid for reaching movements with limited range of motion (RoM). However, most upper limb motions such as Activities of Daily Living (ADL) include large scale inward and outward reaching motions, during which the center of shoulder joint moves significantly. The proposed method generates the reference motion based on a simple model of human arm and a transformation can be used to map the developed motion for other exoskeleton with different kinematics. Comparison of the model outputs with experimental results of healthy subjects performing ADL, show that the proposed model is able to reproduce human-like motions.
△ Less
Submitted 18 December, 2017; v1 submitted 6 December, 2017;
originally announced December 2017.
-
A Systematic Approach For Kinematic Design Of Upper Limb Rehabilitation Exoskeletons
Authors:
Rana Soltani-Zarrin,
Amin Zeiaee,
Reza Langari,
Richard Malak
Abstract:
Kinematic structure of an exoskeleton is the most fundamental block of its design and is determinant of many functional capabilities of it. Although numerous upper limb rehabilitation devices have been designed in the recent years, there is not a framework that can systematically guide the kinematic design procedure. Additionally, diversity of currently available devices and the many minute detail…
▽ More
Kinematic structure of an exoskeleton is the most fundamental block of its design and is determinant of many functional capabilities of it. Although numerous upper limb rehabilitation devices have been designed in the recent years, there is not a framework that can systematically guide the kinematic design procedure. Additionally, diversity of currently available devices and the many minute details incorporated to address certain design requirements hinders pinpointing the core kinematics of the available devices to compare them against each other. This makes the review of literature for identifying drawbacks of the state of the art systems a challenging and puzzling task. In fact, lack of a unifying framework makes designing rehabilitation devices an intuitive process and prone to biases from currently available designs. This research work proposes a systematic approach for kinematic design of upper limb rehabilitation exoskeletons based on conceptual design techniques. Having defined a solution neutral problem statement based on the characteristics of an ideal device, the main functionality of the system is divided into smaller functional units via the Functional Decomposition Method. Various directions for concept generation are explored and finally, it has been shown that a vast majority of the current exoskeleton designs fit within the proposed design framework and the defined functionalities.
△ Less
Submitted 12 December, 2017; v1 submitted 6 December, 2017;
originally announced December 2017.
-
Cleverarm: A Novel Exoskeleton For Rehabilitation Of Upper Limb Impairments
Authors:
Rana Soltani-Zarrin,
Amin Zeiaee,
Andrew Eib,
Reza Langari,
Reza Tafreshi
Abstract:
CLEVERarm (Compact, Low-weight, Ergonomic, Virtual and Augmented Reality Enhanced Rehabilitation arm) is a novel exoskeleton with eight degrees of freedom supporting the motion of shoulder girdle, glenohumeral joint, elbow and wrist. Of the eight degrees of freedom of the exoskeleton, six are active and the two degrees of freedom supporting the motion of wrist are passive. This paper briefly outli…
▽ More
CLEVERarm (Compact, Low-weight, Ergonomic, Virtual and Augmented Reality Enhanced Rehabilitation arm) is a novel exoskeleton with eight degrees of freedom supporting the motion of shoulder girdle, glenohumeral joint, elbow and wrist. Of the eight degrees of freedom of the exoskeleton, six are active and the two degrees of freedom supporting the motion of wrist are passive. This paper briefly outlines the design of CLEVERarm and its control architectures.
△ Less
Submitted 18 December, 2017; v1 submitted 6 December, 2017;
originally announced December 2017.
-
Challenges and Opportunities in Exoskeleton-based Rehabilitation
Authors:
Rana Soltani-Zarrin,
Amin Zeiaee,
Reza Langari,
Reza Tafreshi
Abstract:
Robotic systems are increasingly used in rehabilitation to provide high intensity training for patients with motor impairment. The results of controlled trials involving human subjects confirm the effectiveness of robot-enhanced methods and prove them to be marginally superior over standard manual therapy in some cases. Although very promising, this line of research is still in its infancy and fur…
▽ More
Robotic systems are increasingly used in rehabilitation to provide high intensity training for patients with motor impairment. The results of controlled trials involving human subjects confirm the effectiveness of robot-enhanced methods and prove them to be marginally superior over standard manual therapy in some cases. Although very promising, this line of research is still in its infancy and further studies are required to fully understand the potential benefits of using robotic devices such as exoskeletons. Exoskeletons have been widely studied due to their capability in providing more control over paretic limb as well as the complexities involved in their design and control. This paper briefly discusses the main challenges in development of rehabilitation exoskeletons and elaborates more on how some of these issues are addressed in the design of CLEVERarm, a recently developed upper limb rehabilitation exoskeleton. The paper is concluded with several remarks on the current challenges in wide-spread use of exoskeletons in medical facilities, and a vision for the future of these technologies in rehabilitation medicine.
△ Less
Submitted 18 December, 2017; v1 submitted 26 November, 2017;
originally announced November 2017.
