Showing 1–26 of 26 results for author: Banerjee, N

VirtLab: An AI-Powered System for Flexible, Customizable, and Large-scale Team Simulations

Authors: Mohammed Almutairi, Charles Chiang, Haoze Guo, Matthew Belcher, Nandini Banerjee, Maria Milkowski, Svitlana Volkova, Daniel Nguyen, Tim Weninger, Michael Yankoski, Trenton W. Ford, Diego Gomez-Zara

Abstract: Simulating how team members collaborate within complex environments using Agentic AI is a promising approach to explore hypotheses grounded in social science theories and study team behaviors. We introduce VirtLab, a user-friendly, customizable, multi-agent, and scalable team simulation system that enables testing teams with LLM-based agents in spatial and temporal settings. This system addresses… ▽ More Simulating how team members collaborate within complex environments using Agentic AI is a promising approach to explore hypotheses grounded in social science theories and study team behaviors. We introduce VirtLab, a user-friendly, customizable, multi-agent, and scalable team simulation system that enables testing teams with LLM-based agents in spatial and temporal settings. This system addresses the current frameworks' design and technical limitations that do not consider flexible simulation scenarios and spatial settings. VirtLab contains a simulation engine and a web interface that enables both technical and non-technical users to formulate, run, and analyze team simulations without programming. We demonstrate the system's utility by comparing ground truth data with simulated scenarios. △ Less

Submitted 6 August, 2025; originally announced August 2025.

Comments: 5 pages, 2 figures, UIST 2025

Predicting 3D Motion from 2D Video for Behavior-Based VR Biometrics

Authors: Mingjun Li, Natasha Kholgade Banerjee, Sean Banerjee

Abstract: Critical VR applications in domains such as healthcare, education, and finance that use traditional credentials, such as PIN, password, or multi-factor authentication, stand the chance of being compromised if a malicious person acquires the user credentials or if the user hands over their credentials to an ally. Recently, a number of approaches on user authentication have emerged that use motions… ▽ More Critical VR applications in domains such as healthcare, education, and finance that use traditional credentials, such as PIN, password, or multi-factor authentication, stand the chance of being compromised if a malicious person acquires the user credentials or if the user hands over their credentials to an ally. Recently, a number of approaches on user authentication have emerged that use motions of VR head-mounted displays (HMDs) and hand controllers during user interactions in VR to represent the user's behavior as a VR biometric signature. One of the fundamental limitations of behavior-based approaches is that current on-device tracking for HMDs and controllers lacks capability to perform tracking of full-body joint articulation, losing key signature data encapsulated by the user articulation. In this paper, we propose an approach that uses 2D body joints, namely shoulder, elbow, wrist, hip, knee, and ankle, acquired from the right side of the participants using an external 2D camera. Using a Transformer-based deep neural network, our method uses the 2D data of body joints that are not tracked by the VR device to predict past and future 3D tracks of the right controller, providing the benefit of augmenting 3D knowledge in authentication. Our approach provides a minimum equal error rate (EER) of 0.025, and a maximum EER drop of 0.040 over prior work that uses single-unit 3D trajectory as the input. △ Less

Submitted 4 February, 2025; originally announced February 2025.

Comments: IEEE AIxVR 2025: 7th International Conference on Artificial Intelligence & extended and Virtual Reality

Directed Token Sliding

Authors: Niranka Banerjee, Christian Engels, Duc A. Hoang

Abstract: Reconfiguration problems involve determining whether two given configurations can be transformed into each other under specific rules. The Token Sliding problem asks whether, given two different set of tokens on vertices of a graph $G$, we can transform one into the other by sliding tokens step-by-step along edges of $G$ such that each resulting set of tokens forms an independent set in $G$. Recen… ▽ More Reconfiguration problems involve determining whether two given configurations can be transformed into each other under specific rules. The Token Sliding problem asks whether, given two different set of tokens on vertices of a graph $G$, we can transform one into the other by sliding tokens step-by-step along edges of $G$ such that each resulting set of tokens forms an independent set in $G$. Recently, Ito et al. [MFCS 2022] introduced a directed variant of this problem. They showed that for general oriented graphs (i.e., graphs where no pair of vertices can have directed edges in both directions), the problem remains $\mathsf{PSPACE}$-complete, and is solvable in polynomial time on oriented trees. In this paper, we further investigate the Token Sliding problem on various oriented graph classes. We show that the problem remains $\mathsf{PSPACE}$-complete for oriented planar graphs, split graphs, bipartite graphs and bounded treewidth graphs. Additionally, we present polynomial-time algorithms for solving the problem on oriented cycles and cographs. △ Less

Submitted 25 November, 2024; originally announced November 2024.

Distance Recoloring

Authors: Niranka Banerjee, Christian Engels, Duc A. Hoang

Abstract: For integers $d \geq 1$ and $k \geq d+1$, the \textsc{Distance Coloring} problem asks if a given graph $G$ has a $(d, k)$-coloring, i.e., a coloring of the vertices of $G$ by $k$ colors such that any two vertices within distance $d$ from each other have different colors. In particular, the well-known \textsc{Coloring} problem is a special case of \textsc{Distance Coloring} when $d = 1$. For intege… ▽ More For integers $d \geq 1$ and $k \geq d+1$, the \textsc{Distance Coloring} problem asks if a given graph $G$ has a $(d, k)$-coloring, i.e., a coloring of the vertices of $G$ by $k$ colors such that any two vertices within distance $d$ from each other have different colors. In particular, the well-known \textsc{Coloring} problem is a special case of \textsc{Distance Coloring} when $d = 1$. For integers $d \geq 2$ and $k \geq d+1$, the \textsc{$(d, k)$-Coloring Reconfiguration} problem asks if there is a way to change the color of one vertex at a time, starting from a $(d, k)$-coloring $α$ of a graph $G$ to reach another $(d, k)$-coloring $β$ of $G$, such that all intermediate colorings are also $(d, k)$-colorings. We show that even for planar, bipartite, and $2$-degenerate graphs, \textsc{$(d, k)$-Coloring Reconfiguration} remains $\mathsf{PSPACE}$-complete for $d \geq 2$ and $k = Ω(d^2)$ via a reduction from the well-known \textsc{Sliding Tokens} problem. Additionally, on split graphs, there is an interesting dichotomy: the problem is $\mathsf{PSPACE}$-complete when $d = 2$ and $k$ is large but can be solved efficiently when $d \geq 3$ and $k \geq d+1$. For chordal graphs, we show that the problem is $\mathsf{PSPACE}$-complete for even values of $d \geq 2$. Finally, we design a quadratic-time algorithm to solve the problem on paths for any $d \geq 2$ and $k \geq d+1$. △ Less

Submitted 6 March, 2025; v1 submitted 19 February, 2024; originally announced February 2024.

Comments: 31 pages, 8 figures, v3: revision of v2, update results on split graphs and chordal graphs

Using Motion Forecasting for Behavior-Based Virtual Reality (VR) Authentication

Authors: Mingjun Li, Natasha Kholgade Banerjee, Sean Banerjee

Abstract: Task-based behavioral biometric authentication of users interacting in virtual reality (VR) environments enables seamless continuous authentication by using only the motion trajectories of the person's body as a unique signature. Deep learning-based approaches for behavioral biometrics show high accuracy when using complete or near complete portions of the user trajectory, but show lower performan… ▽ More Task-based behavioral biometric authentication of users interacting in virtual reality (VR) environments enables seamless continuous authentication by using only the motion trajectories of the person's body as a unique signature. Deep learning-based approaches for behavioral biometrics show high accuracy when using complete or near complete portions of the user trajectory, but show lower performance when using smaller segments from the start of the task. Thus, any systems designed with existing techniques are vulnerable while waiting for future segments of motion trajectories to become available. In this work, we present the first approach that predicts future user behavior using Transformer-based forecasting and using the forecasted trajectory to perform user authentication. Our work leverages the notion that given the current trajectory of a user in a task-based environment we can predict the future trajectory of the user as they are unlikely to dramatically shift their behavior since it would preclude the user from successfully completing their task goal. Using the publicly available 41-subject ball throwing dataset of Miller et al. we show improvement in user authentication when using forecasted data. When compared to no forecasting, our approach reduces the authentication equal error rate (EER) by an average of 23.85% and a maximum reduction of 36.14%. △ Less

Submitted 29 January, 2024; originally announced January 2024.

Comments: AIxVR 2024 Best Paper Award

Evaluating Deep Networks for Detecting User Familiarity with VR from Hand Interactions

Authors: Mingjun Li, Numan Zafar, Natasha Kholgade Banerjee, Sean Banerjee

Abstract: As VR devices become more prevalent in the consumer space, VR applications are likely to be increasingly used by users unfamiliar with VR. Detecting the familiarity level of a user with VR as an interaction medium provides the potential of providing on-demand training for acclimatization and prevents the user from being burdened by the VR environment in accomplishing their tasks. In this work, we… ▽ More As VR devices become more prevalent in the consumer space, VR applications are likely to be increasingly used by users unfamiliar with VR. Detecting the familiarity level of a user with VR as an interaction medium provides the potential of providing on-demand training for acclimatization and prevents the user from being burdened by the VR environment in accomplishing their tasks. In this work, we present preliminary results of using deep classifiers to conduct automatic detection of familiarity with VR by using hand tracking of the user as they interact with a numeric passcode entry panel to unlock a VR door. We use a VR door as we envision it to the first point of entry to collaborative virtual spaces, such as meeting rooms, offices, or clinics. Users who are unfamiliar with VR will have used their hands to open doors with passcode entry panels in the real world. Thus, while the user may not be familiar with VR, they would be familiar with the task of opening the door. Using a pilot dataset consisting of 7 users familiar with VR, and 7 not familiar with VR, we acquire highest accuracy of 88.03\% when 6 test users, 3 familiar and 3 not familiar, are evaluated with classifiers trained using data from the remaining 8 users. Our results indicate potential for using user movement data to detect familiarity for the simple yet important task of secure passcode-based access. △ Less

Submitted 27 January, 2024; originally announced January 2024.

Comments: AIxVR 2024 poster paper

HOH: Markerless Multimodal Human-Object-Human Handover Dataset with Large Object Count

Authors: Noah Wiederhold, Ava Megyeri, DiMaggio Paris, Sean Banerjee, Natasha Kholgade Banerjee

Abstract: We present the HOH (Human-Object-Human) Handover Dataset, a large object count dataset with 136 objects, to accelerate data-driven research on handover studies, human-robot handover implementation, and artificial intelligence (AI) on handover parameter estimation from 2D and 3D data of person interactions. HOH contains multi-view RGB and depth data, skeletons, fused point clouds, grasp type and ha… ▽ More We present the HOH (Human-Object-Human) Handover Dataset, a large object count dataset with 136 objects, to accelerate data-driven research on handover studies, human-robot handover implementation, and artificial intelligence (AI) on handover parameter estimation from 2D and 3D data of person interactions. HOH contains multi-view RGB and depth data, skeletons, fused point clouds, grasp type and handedness labels, object, giver hand, and receiver hand 2D and 3D segmentations, giver and receiver comfort ratings, and paired object metadata and aligned 3D models for 2,720 handover interactions spanning 136 objects and 20 giver-receiver pairs-40 with role-reversal-organized from 40 participants. We also show experimental results of neural networks trained using HOH to perform grasp, orientation, and trajectory prediction. As the only fully markerless handover capture dataset, HOH represents natural human-human handover interactions, overcoming challenges with markered datasets that require specific suiting for body tracking, and lack high-resolution hand tracking. To date, HOH is the largest handover dataset in number of objects, participants, pairs with role reversal accounted for, and total interactions captured. △ Less

Submitted 3 May, 2024; v1 submitted 1 October, 2023; originally announced October 2023.

Comments: NeurIPS 2023 Datasets and Benchmarks

The Complexity of Distance-$r$ Dominating Set Reconfiguration

Authors: Niranka Banerjee, Duc A. Hoang

Abstract: For a fixed integer $r \geq 1$, a distance-$r$ dominating set (D$r$DS) of a graph $G = (V, E)$ is a vertex subset $D \subseteq V$ such that every vertex in $V$ is within distance $r$ from some member of $D$. Given two D$r$DSs $D_s, D_t$ of $G$, the Distance-$r$ Dominating Set Reconfiguration (D$r$DSR) problem asks if there is a sequence of D$r$DSs that transforms $D_s$ into $D_t$ (or vice versa) s… ▽ More For a fixed integer $r \geq 1$, a distance-$r$ dominating set (D$r$DS) of a graph $G = (V, E)$ is a vertex subset $D \subseteq V$ such that every vertex in $V$ is within distance $r$ from some member of $D$. Given two D$r$DSs $D_s, D_t$ of $G$, the Distance-$r$ Dominating Set Reconfiguration (D$r$DSR) problem asks if there is a sequence of D$r$DSs that transforms $D_s$ into $D_t$ (or vice versa) such that each intermediate member is obtained from its predecessor by applying a given reconfiguration rule exactly once. The problem for $r = 1$ has been well-studied in the literature. We consider D$r$DSR for $r \geq 2$ under two well-known reconfiguration rules: Token Jumping ($\mathsf{TJ}$, which involves replacing a member of the current D$r$DS by a non-member) and Token Sliding ($\mathsf{TS}$, which involves replacing a member of the current D$r$DS by an adjacent non-member). It is known that under any of $\mathsf{TS}$ and $\mathsf{TJ}$, the problem on split graphs is $\mathtt{PSPACE}$-complete for $r = 1$. We show that for $r \geq 2$, the problem is in $\mathtt{P}$, resulting in an interesting complexity dichotomy. Along the way, we prove some non-trivial bounds on the length of a shortest reconfiguration sequence on split graphs when $r = 2$ which may be of independent interest. Additionally, we design a linear-time algorithm under $\mathsf{TJ}$ on trees. On the negative side, we show that D$r$DSR for $r \geq 1$ on planar graphs of maximum degree three and bounded bandwidth is $\mathtt{PSPACE}$-complete, improving the degree bound of previously known results. We also show that the known $\mathtt{PSPACE}$-completeness results under $\mathsf{TS}$ and $\mathsf{TJ}$ for $r = 1$ on bipartite graphs and chordal graphs can be extended for $r \geq 2$. △ Less

Submitted 2 February, 2024; v1 submitted 29 September, 2023; originally announced October 2023.

Comments: 32 pages, 10 figures, v2: revise some incorrect proofs in v1, add new results for split graphs and planar graphs, add minor modifications. The HTML version of the abstract was shorten to fit the requirement of arXiv

MSC Class: 05C85

RESTORE: Graph Embedding Assessment Through Reconstruction

Authors: Hong Yung Yip, Chidaksh Ravuru, Neelabha Banerjee, Shashwat Jha, Amit Sheth, Aman Chadha, Amitava Das

Abstract: Following the success of Word2Vec embeddings, graph embeddings (GEs) have gained substantial traction. GEs are commonly generated and evaluated extrinsically on downstream applications, but intrinsic evaluations of the original graph properties in terms of topological structure and semantic information have been lacking. Understanding these will help identify the deficiency of the various families… ▽ More Following the success of Word2Vec embeddings, graph embeddings (GEs) have gained substantial traction. GEs are commonly generated and evaluated extrinsically on downstream applications, but intrinsic evaluations of the original graph properties in terms of topological structure and semantic information have been lacking. Understanding these will help identify the deficiency of the various families of GE methods when vectorizing graphs in terms of preserving the relevant knowledge or learning incorrect knowledge. To address this, we propose RESTORE, a framework for intrinsic GEs assessment through graph reconstruction. We show that reconstructing the original graph from the underlying GEs yields insights into the relative amount of information preserved in a given vector form. We first introduce the graph reconstruction task. We generate GEs from three GE families based on factorization methods, random walks, and deep learning (with representative algorithms from each family) on the CommonSense Knowledge Graph (CSKG). We analyze their effectiveness in preserving the (a) topological structure of node-level graph reconstruction with an increasing number of hops and (b) semantic information on various word semantic and analogy tests. Our evaluations show deep learning-based GE algorithm (SDNE) is overall better at preserving (a) with a mean average precision (mAP) of 0.54 and 0.35 for 2 and 3-hop reconstruction respectively, while the factorization-based algorithm (HOPE) is better at encapsulating (b) with an average Euclidean distance of 0.14, 0.17, and 0.11 for 1, 2, and 3-hop reconstruction respectively. The modest performance of these GEs leaves room for further research avenues on better graph representation learning. △ Less

Submitted 5 September, 2023; v1 submitted 28 August, 2023; originally announced August 2023.

Pix2Repair: Implicit Shape Restoration from Images

Authors: Xinchao Song, Nikolas Lamb, Sean Banerjee, Natasha Kholgade Banerjee

Abstract: We present Pix2Repair, an automated shape repair approach that generates restoration shapes from images to repair fractured objects. Prior repair approaches require a high-resolution watertight 3D mesh of the fractured object as input. Input 3D meshes must be obtained using expensive 3D scanners, and scanned meshes require manual cleanup, limiting accessibility and scalability. Pix2Repair takes an… ▽ More We present Pix2Repair, an automated shape repair approach that generates restoration shapes from images to repair fractured objects. Prior repair approaches require a high-resolution watertight 3D mesh of the fractured object as input. Input 3D meshes must be obtained using expensive 3D scanners, and scanned meshes require manual cleanup, limiting accessibility and scalability. Pix2Repair takes an image of the fractured object as input and automatically generates a 3D printable restoration shape. We contribute a novel shape function that deconstructs a latent code representing the fractured object into a complete shape and a break surface. We also introduce Fantastic Breaks Imaged, the first large-scale dataset of 11,653 real-world images of fractured objects for training and evaluating image-based shape repair approaches. Our dataset contains images of objects from Fantastic Breaks, complete with rich annotations. We show restorations for real fractures from our dataset, and for synthetic fractures from the Geometric Breaks and Breaking Bad datasets. Our approach outperforms shape completion approaches adapted for shape repair in terms of chamfer distance, normal consistency, and percent restorations generated. △ Less

Submitted 20 December, 2023; v1 submitted 29 May, 2023; originally announced May 2023.

FVCARE:Formal Verification of Security Primitives in Resilient Embedded SoCs

Authors: Avani Dave, Nilanjan Banerjee, Chintan Patel

Abstract: With the increased utilization, the small embedded and IoT devices have become an attractive target for sophisticated attacks that can exploit the devices security critical information and data in malevolent activities. Secure boot and Remote Attestation (RA) techniques verifies the integrity of the devices software state at boot-time and runtime. Correct implementation and formal verification of… ▽ More With the increased utilization, the small embedded and IoT devices have become an attractive target for sophisticated attacks that can exploit the devices security critical information and data in malevolent activities. Secure boot and Remote Attestation (RA) techniques verifies the integrity of the devices software state at boot-time and runtime. Correct implementation and formal verification of these security primitives provide strong security guarantees and enhance user confidence. The formal verification of these security primitives is considered challenging, as it involves complex hardware software interactions, semantics gaps and requires bit-precise reasoning. To address these challenges, this paper presents FVCARE an end to end system co-verification framework. It also defines the security properties for resilient small embedded systems. FVCARE divides the end to end system co verification problem into two modules: 1) verifying the (bit precise) initial system settings, registers, and access control policies by hardware verification techniques, and 2) verifying the system specification, security properties, and functional correctness using source-level software abstraction of the hardware. The evaluation of proposed techniques on SRACARE based systems demonstrates its efficacy in security co verification. △ Less

Submitted 22 April, 2023; originally announced April 2023.

Fantastic Breaks: A Dataset of Paired 3D Scans of Real-World Broken Objects and Their Complete Counterparts

Authors: Nikolas Lamb, Cameron Palmer, Benjamin Molloy, Sean Banerjee, Natasha Kholgade Banerjee

Abstract: Automated shape repair approaches currently lack access to datasets that describe real-world damaged geometry. We present Fantastic Breaks (and Where to Find Them: https://terascale-all-sensing-research-studio.github.io/FantasticBreaks), a dataset containing scanned, waterproofed, and cleaned 3D meshes for 150 broken objects, paired and geometrically aligned with complete counterparts. Fantastic B… ▽ More Automated shape repair approaches currently lack access to datasets that describe real-world damaged geometry. We present Fantastic Breaks (and Where to Find Them: https://terascale-all-sensing-research-studio.github.io/FantasticBreaks), a dataset containing scanned, waterproofed, and cleaned 3D meshes for 150 broken objects, paired and geometrically aligned with complete counterparts. Fantastic Breaks contains class and material labels, proxy repair parts that join to broken meshes to generate complete meshes, and manually annotated fracture boundaries. Through a detailed analysis of fracture geometry, we reveal differences between Fantastic Breaks and synthetic fracture datasets generated using geometric and physics-based methods. We show experimental shape repair evaluation with Fantastic Breaks using multiple learning-based approaches pre-trained with synthetic datasets and re-trained with subset of Fantastic Breaks. △ Less

Submitted 1 May, 2023; v1 submitted 24 March, 2023; originally announced March 2023.

Comments: To be published at CVPR 2023

Simultaneous prediction of hand gestures, handedness, and hand keypoints using thermal images

Authors: Sichao Li, Sean Banerjee, Natasha Kholgade Banerjee, Soumyabrata Dey

Abstract: Hand gesture detection is a well-explored area in computer vision with applications in various forms of Human-Computer Interactions. In this work, we propose a technique for simultaneous hand gesture classification, handedness detection, and hand keypoints localization using thermal data captured by an infrared camera. Our method uses a novel deep multi-task learning architecture that includes sha… ▽ More Hand gesture detection is a well-explored area in computer vision with applications in various forms of Human-Computer Interactions. In this work, we propose a technique for simultaneous hand gesture classification, handedness detection, and hand keypoints localization using thermal data captured by an infrared camera. Our method uses a novel deep multi-task learning architecture that includes shared encoderdecoder layers followed by three branches dedicated for each mentioned task. We performed extensive experimental validation of our model on an in-house dataset consisting of 24 users data. The results confirm higher than 98 percent accuracy for gesture classification, handedness detection, and fingertips localization, and more than 91 percent accuracy for wrist points localization. △ Less

Submitted 2 March, 2023; originally announced March 2023.

Comments: ICDEC 2022

DeepJoin: Learning a Joint Occupancy, Signed Distance, and Normal Field Function for Shape Repair

Authors: Nikolas Lamb, Sean Banerjee, Natasha Kholgade Banerjee

Abstract: We introduce DeepJoin, an automated approach to generate high-resolution repairs for fractured shapes using deep neural networks. Existing approaches to perform automated shape repair operate exclusively on symmetric objects, require a complete proxy shape, or predict restoration shapes using low-resolution voxels which are too coarse for physical repair. We generate a high-resolution restoration… ▽ More We introduce DeepJoin, an automated approach to generate high-resolution repairs for fractured shapes using deep neural networks. Existing approaches to perform automated shape repair operate exclusively on symmetric objects, require a complete proxy shape, or predict restoration shapes using low-resolution voxels which are too coarse for physical repair. We generate a high-resolution restoration shape by inferring a corresponding complete shape and a break surface from an input fractured shape. We present a novel implicit shape representation for fractured shape repair that combines the occupancy function, signed distance function, and normal field. We demonstrate repairs using our approach for synthetically fractured objects from ShapeNet, 3D scans from the Google Scanned Objects dataset, objects in the style of ancient Greek pottery from the QP Cultural Heritage dataset, and real fractured objects. We outperform three baseline approaches in terms of chamfer distance and normal consistency. Unlike existing approaches and restorations using subtraction, DeepJoin restorations do not exhibit surface artifacts and join closely to the fractured region of the fractured shape. Our code is available at: https://github.com/Terascale-All-sensing-Research-Studio/DeepJoin. △ Less

Submitted 22 November, 2022; originally announced November 2022.

Comments: To be published at SIGGRAPH Asia 2022 (Journal)

DeepMend: Learning Occupancy Functions to Represent Shape for Repair

Authors: Nikolas Lamb, Sean Banerjee, Natasha Kholgade Banerjee

Abstract: We present DeepMend, a novel approach to reconstruct restorations to fractured shapes using learned occupancy functions. Existing shape repair approaches predict low-resolution voxelized restorations, or require symmetries or access to a pre-existing complete oracle. We represent the occupancy of a fractured shape as the conjunction of the occupancy of an underlying complete shape and the fracture… ▽ More We present DeepMend, a novel approach to reconstruct restorations to fractured shapes using learned occupancy functions. Existing shape repair approaches predict low-resolution voxelized restorations, or require symmetries or access to a pre-existing complete oracle. We represent the occupancy of a fractured shape as the conjunction of the occupancy of an underlying complete shape and the fracture surface, which we model as functions of latent codes using neural networks. Given occupancy samples from an input fractured shape, we estimate latent codes using an inference loss augmented with novel penalty terms that avoid empty or voluminous restorations. We use inferred codes to reconstruct the restoration shape. We show results with simulated fractures on synthetic and real-world scanned objects, and with scanned real fractured mugs. Compared to the existing voxel approach and two baseline methods, our work shows state-of-the-art results in accuracy and avoiding restoration artifacts over non-fracture regions of the fractured shape. △ Less

Submitted 11 October, 2022; originally announced October 2022.

Comments: To be published at ECCV 2020 (poster)

CARE: Lightweight Attack Resilient Secure Boot Architecturewith Onboard Recovery for RISC-V based SOC

Authors: Avani Dave, Nilanjan Banerjee, Chintan Patel

Abstract: Recent technological advancements have proliferated the use of small embedded devices for collecting, processing, and transferring the security-critical information. The Internet of Things (IoT) has enabled remote access and control of these network-connected devices. Consequently, an attacker can exploit security vulnerabilities and compromise these devices. In this context, the secure boot becom… ▽ More Recent technological advancements have proliferated the use of small embedded devices for collecting, processing, and transferring the security-critical information. The Internet of Things (IoT) has enabled remote access and control of these network-connected devices. Consequently, an attacker can exploit security vulnerabilities and compromise these devices. In this context, the secure boot becomes a useful security mechanism to verify the integrity and authenticity of the software state of the devices. However, the current secure boot schemes focus on detecting the presence of potential malware on the device but not on disinfecting and restoring the soft-ware to a benign state. This manuscript presents CARE- the first secure boot framework that provides detection, resilience, and onboard recovery mechanism for the com-promised devices. The framework uses a prototype hybrid CARE: Code Authentication and Resilience Engine to verify the software state and restore it to a benign state. It uses Physical Memory Protection (PMP) and other security enchaining techniques of RISC-V processor to pro-vide resilience from modern attacks. The state-of-the-art comparison and performance analysis results indicate that the proposed secure boot framework provides a promising resilience and recovery mechanism with very little 8 % performance and resource overhead △ Less

Submitted 15 January, 2021; originally announced January 2021.

SRACARE: Secure Remote Attestation with Code Authentication and Resilience Engine

Authors: Avani Dave, Nilanjan Banerjee, Chintan Patel

Abstract: Recent technological advancements have enabled proliferated use of small embedded and IoT devices for collecting, processing, and transferring the security-critical information and user data. This exponential use has acted as a catalyst in the recent growth of sophisticated attacks such as the replay, man-in-the-middle, and malicious code modification to slink, leak, tweak or exploit the security-… ▽ More Recent technological advancements have enabled proliferated use of small embedded and IoT devices for collecting, processing, and transferring the security-critical information and user data. This exponential use has acted as a catalyst in the recent growth of sophisticated attacks such as the replay, man-in-the-middle, and malicious code modification to slink, leak, tweak or exploit the security-critical information in malevolent activities. Therefore, secure communication and software state assurance (at run-time and boot-time) of the device has emerged as open security problems. Furthermore, these devices need to have an appropriate recovery mechanism to bring them back to the known-good operational state. Previous researchers have demonstrated independent methods for attack detection and safeguard. However, the majority of them lack in providing onboard system recovery and secure communication techniques. To bridge this gap, this manuscript proposes SRACARE- a framework that utilizes the custom lightweight, secure communication protocol that performs remote/local attestation, and secure boot with an onboard resilience recovery mechanism to protect the devices from the above-mentioned attacks. The prototype employs an efficient lightweight, low-power 32-bit RISC-V processor, secure communication protocol, code authentication, and resilience engine running on the Artix 7 Field Programmable Gate Array(FPGA) board. This work presents the performance evaluation and state-of-the-art comparison results, which shows promising resilience to attacks and demonstrate the novel protection mechanism with onboard recovery. The framework achieves these with only 8 % performance overhead and a very small increase in hardware-software footprint. △ Less

Submitted 15 January, 2021; originally announced January 2021.

AI Failures: A Review of Underlying Issues

Authors: Debarag Narayan Banerjee, Sasanka Sekhar Chanda

Abstract: Instances of Artificial Intelligence (AI) systems failing to deliver consistent, satisfactory performance are legion. We investigate why AI failures occur. We address only a narrow subset of the broader field of AI Safety. We focus on AI failures on account of flaws in conceptualization, design and deployment. Other AI Safety issues like trade-offs between privacy and security or convenience, bad… ▽ More Instances of Artificial Intelligence (AI) systems failing to deliver consistent, satisfactory performance are legion. We investigate why AI failures occur. We address only a narrow subset of the broader field of AI Safety. We focus on AI failures on account of flaws in conceptualization, design and deployment. Other AI Safety issues like trade-offs between privacy and security or convenience, bad actors hacking into AI systems to create mayhem or bad actors deploying AI for purposes harmful to humanity and are out of scope of our discussion. We find that AI systems fail on account of omission and commission errors in the design of the AI system, as well as upon failure to develop an appropriate interpretation of input information. Moreover, even when there is no significant flaw in the AI software, an AI system may fail because the hardware is incapable of robust performance across environments. Finally an AI system is quite likely to fail in situations where, in effect, it is called upon to deliver moral judgments -- a capability AI does not possess. We observe certain trade-offs in measures to mitigate a subset of AI failures and provide some recommendations. △ Less

Submitted 18 July, 2020; originally announced August 2020.

Comments: 8 pages

An RNN-Survival Model to Decide Email Send Times

Authors: Harvineet Singh, Moumita Sinha, Atanu R. Sinha, Sahil Garg, Neha Banerjee

Abstract: Email communications are ubiquitous. Firms control send times of emails and thereby the instants at which emails reach recipients (it is assumed email is received instantaneously from the send time). However, they do not control the duration it takes for recipients to open emails, labeled as time-to-open. Importantly, among emails that are opened, most occur within a short window from their send t… ▽ More Email communications are ubiquitous. Firms control send times of emails and thereby the instants at which emails reach recipients (it is assumed email is received instantaneously from the send time). However, they do not control the duration it takes for recipients to open emails, labeled as time-to-open. Importantly, among emails that are opened, most occur within a short window from their send times. We posit that emails are likely to be opened sooner when send times are convenient for recipients, while for other send times, emails can get ignored. Thus, to compute appropriate send times it is important to predict times-to-open accurately. We propose a recurrent neural network (RNN) in a survival model framework to predict times-to-open, for each recipient. Using that we compute appropriate send times. We experiment on a data set of emails sent to a million customers over five months. The sequence of emails received by a person from a sender is a result of interactions with past emails from the sender, and hence contain useful signal that inform our model. This sequential dependence affords our proposed RNN-Survival (RNN-S) approach to outperform survival analysis approaches in predicting times-to-open. We show that best times to send emails can be computed accurately from predicted times-to-open. This approach allows a firm to tune send times of emails, which is in its control, to favorably influence open rates and engagement. △ Less

Submitted 21 April, 2020; originally announced April 2020.

Comments: 11 pages, 3 figures, 2 tables

View management for lifelong visual maps

Authors: Nandan Banerjee, Ryan C. Connolly, Dimitri Lisin, Jimmy Briggs, Manjunath Narayana, Mario E. Munich

Abstract: The time complexity of making observations and loop closures in a graph-based visual SLAM system is a function of the number of views stored. Clever algorithms, such as approximate nearest neighbor search, can make this function sub-linear. Despite this, over time the number of views can still grow to a point at which the speed and/or accuracy of the system becomes unacceptable, especially in comp… ▽ More The time complexity of making observations and loop closures in a graph-based visual SLAM system is a function of the number of views stored. Clever algorithms, such as approximate nearest neighbor search, can make this function sub-linear. Despite this, over time the number of views can still grow to a point at which the speed and/or accuracy of the system becomes unacceptable, especially in computation- and memory-constrained SLAM systems. However, not all views are created equal. Some views are rarely observed, because they have been created in an unusual lighting condition, or from low quality images, or in a location whose appearance has changed. These views can be removed to improve the overall performance of a SLAM system. In this paper, we propose a method for pruning views in a visual SLAM system to maintain its speed and accuracy for long term use. △ Less

Submitted 9 August, 2019; originally announced August 2019.

Comments: IEEE International Conference on Intelligent Robots and Systems (IROS), 2019

Estimating Buildings' Parameters over Time Including Prior Knowledge

Authors: Nilavra Pathak, James Foulds, Nirmalya Roy, Nilanjan Banerjee, Ryan Robucci

Abstract: Modeling buildings' heat dynamics is a complex process which depends on various factors including weather, building thermal capacity, insulation preservation, and residents' behavior. Gray-box models offer a causal inference of those dynamics expressed in few parameters specific to built environments. These parameters can provide compelling insights into the characteristics of building artifacts a… ▽ More Modeling buildings' heat dynamics is a complex process which depends on various factors including weather, building thermal capacity, insulation preservation, and residents' behavior. Gray-box models offer a causal inference of those dynamics expressed in few parameters specific to built environments. These parameters can provide compelling insights into the characteristics of building artifacts and have various applications such as forecasting HVAC usage, indoor temperature control monitoring of built environments, etc. In this paper, we present a systematic study of modeling buildings' thermal characteristics and thus derive the parameters of built conditions with a Bayesian approach. We build a Bayesian state-space model that can adapt and incorporate buildings' thermal equations and propose a generalized solution that can easily adapt prior knowledge regarding the parameters. We show that a faster approximate approach using variational inference for parameter estimation can provide similar parameters as that of a more time-consuming Markov Chain Monte Carlo (MCMC) approach. We perform extensive evaluations on two datasets to understand the generative process and show that the Bayesian approach is more interpretable. We further study the effects of prior selection for the model parameters and transfer learning, where we learn parameters from one season and use them to fit the model in the other. We perform extensive evaluations on controlled and real data traces to enumerate buildings' parameter within a 95% credible interval. △ Less

Submitted 18 February, 2019; v1 submitted 8 January, 2019; originally announced January 2019.

Comments: 11 pages with reference

Unsupervised Learning for Trustworthy IoT

Authors: Nikhil Banerjee, Thanassis Giannetsos, Emmanouil Panaousis, Clive Cheong Took

Abstract: The advancement of Internet-of-Things (IoT) edge devices with various types of sensors enables us to harness diverse information with Mobile Crowd-Sensing applications (MCS). This highly dynamic setting entails the collection of ubiquitous data traces, originating from sensors carried by people, introducing new information security challenges; one of them being the preservation of data trustworthi… ▽ More The advancement of Internet-of-Things (IoT) edge devices with various types of sensors enables us to harness diverse information with Mobile Crowd-Sensing applications (MCS). This highly dynamic setting entails the collection of ubiquitous data traces, originating from sensors carried by people, introducing new information security challenges; one of them being the preservation of data trustworthiness. What is needed in these settings is the timely analysis of these large datasets to produce accurate insights on the correctness of user reports. Existing data mining and other artificial intelligence methods are the most popular to gain hidden insights from IoT data, albeit with many challenges. In this paper, we first model the cyber trustworthiness of MCS reports in the presence of intelligent and colluding adversaries. We then rigorously assess, using real IoT datasets, the effectiveness and accuracy of well-known data mining algorithms when employed towards IoT security and privacy. By taking into account the spatio-temporal changes of the underlying phenomena, we demonstrate how concept drifts can masquerade the existence of attackers and their impact on the accuracy of both the clustering and classification processes. Our initial set of results clearly show that these unsupervised learning algorithms are prone to adversarial infection, thus, magnifying the need for further research in the field by leveraging a mix of advanced machine learning models and mathematical optimization techniques. △ Less

Submitted 25 May, 2018; originally announced May 2018.

Comments: 9 pages, 9 figures, 2018 IEEE International Conference on Fuzzy Systems

Improved Space efficient linear time algorithms for BFS, DFS and applications

Authors: Niranka Banerjee, Sankardeep Chakraborty, Venkatesh Raman, Srinivasa Rao Satti

Abstract: Recent work by Elmasry et al. (STACS 2015) and Asano et al. (ISAAC 2014), reconsidered classical fundamental graph algorithms focusing on improving the space complexity. We continue this line of work focusing on space. Our first result is a simple data structure that can maintain any subset $S$ of a universe of $n$ elements using $n+o(n)$ bits and support in constant time, apart from the standard… ▽ More Recent work by Elmasry et al. (STACS 2015) and Asano et al. (ISAAC 2014), reconsidered classical fundamental graph algorithms focusing on improving the space complexity. We continue this line of work focusing on space. Our first result is a simple data structure that can maintain any subset $S$ of a universe of $n$ elements using $n+o(n)$ bits and support in constant time, apart from the standard insert, delete and membership queries, the operation {\it findany} that finds and returns any element of the set (or outputs that the set is empty). Using this we give a BFS implementation that takes $O(m+n)$ time using at most $2n+o(n)$ bits. Later, we further improve the space requirement of BFS to at most $1.585n + o(n)$ bits. We demonstrate the use of our data structure by developing another data structure using it that can represent a sequence of $n$ non-negative integers $x_1, x_2, \ldots x_n$ using at most $\sum_{i=1}^n x_i + 2n + o(\sum_{i=1}^n x_i+n)$ bits and, in constant time, determine whether the $i$-th element is $0$ or decrement it otherwise. We also discuss an algorithm for finding a minimum weight spanning tree of a weighted undirected graph using at most $n+o(n)$ bits. We also provide an implementation for DFS that takes $O(m+n)$ time and $O(n \lg(m/n))$ bits. Using this DFS algorithm and other careful implementations, we can test biconnectivity, 2-edge connectivity, and determine cut vertices, bridges etc among others, essentially within the same time and space bounds required for DFS. These improve the space required for earlier implementations from $Ω(n\lg n)$ bits. △ Less

Submitted 27 July, 2017; v1 submitted 15 June, 2016; originally announced June 2016.

Comments: A preliminary version of this paper appears in the proceedings of COCOON 2016

A comparison algorithm to check LTSA Layer 1 and SCORM compliance in e-Learning sites

Authors: Souvik Sengupta, Saurabh Pal, Nilanjan Banerjee

Abstract: The success of e-Learning is largely dependent on the impact of its multimedia aided learning content on the learner over the hyper media. The e-Learning portals with different proportion of multimedia elements have different impact on the learner, as there is lack of standardization. The Learning Technology System Architecture (LTSA) Layer 1 deals with the effect of environment on the learner. Fr… ▽ More The success of e-Learning is largely dependent on the impact of its multimedia aided learning content on the learner over the hyper media. The e-Learning portals with different proportion of multimedia elements have different impact on the learner, as there is lack of standardization. The Learning Technology System Architecture (LTSA) Layer 1 deals with the effect of environment on the learner. From an information technology perspective it specifies learner interaction from the environment to the learner via multimedia content. Sharable Content Object Reference Model (SCROM) is a collection of standards and specifications for content of web-based e-learning and specifies how JavaScript API can be used to integrate content development. In this paper an examination is made on the design features of interactive multimedia components of the learning packages by creating an algorithm which will give a comparative study of multimedia component used by different learning packages. The resultant graph as output helps us to analysis to what extent any LMS compliance LTSA layer 1 and SCORM specification. △ Less

Submitted 19 January, 2012; originally announced January 2012.

Mirage: Mitigating Illicit Inventorying in a RFID Enabled Retail Environment

Authors: Jonathan White, Nilanjan Banerjee

Abstract: Given its low dollar and maintenance cost, RFID is poised to become the enabling technology for inventory control and supply chain management. However, as an outcome of its low cost, RFID based inventory control is susceptible to pernicious security and privacy threats. A deleterious attack on such a system is corporate espionage, where attackers through illicit inventorying infer sales and restoc… ▽ More Given its low dollar and maintenance cost, RFID is poised to become the enabling technology for inventory control and supply chain management. However, as an outcome of its low cost, RFID based inventory control is susceptible to pernicious security and privacy threats. A deleterious attack on such a system is corporate espionage, where attackers through illicit inventorying infer sales and restocking trends for products. In this paper, we first present plausible aftermaths of corporate espionage using real data from online sources. Second, to mitigate corporate espionage in a retail store environment, we present a simple lowcost system called Mirage. Mirage uses additional programmable low cost passive RFID tags called honeytokens to inject noise in retail store inven-torying. Using a simple history based algorithm that controls activation and de-activation of honeytokens, Mirage randomizes sales and restocking trends. We evaluate Mirage in a real warehouse environment using a commercial off-the-shelf Motorola MC9090 handheld RFID reader and over 450 Gen2 low cost RFID tags. We show that Mirage successfully flattens and randomizes sales and restocking trends while adding minimal cost to inventory control. △ Less

Submitted 29 June, 2010; originally announced June 2010.

Statistical Modeling of Pipeline Delay and Design of Pipeline under Process Variation to Enhance Yield in sub-100nm Technologies

Authors: Animesh Datta, Swarup Bhunia, Saibal Mukhopadhyay, Nilanjan Banerjee, Kaushik Roy

Abstract: Operating frequency of a pipelined circuit is determined by the delay of the slowest pipeline stage. However, under statistical delay variation in sub-100nm technology regime, the slowest stage is not readily identifiable and the estimation of the pipeline yield with respect to a target delay is a challenging problem. We have proposed analytical models to estimate yield for a pipelined design ba… ▽ More Operating frequency of a pipelined circuit is determined by the delay of the slowest pipeline stage. However, under statistical delay variation in sub-100nm technology regime, the slowest stage is not readily identifiable and the estimation of the pipeline yield with respect to a target delay is a challenging problem. We have proposed analytical models to estimate yield for a pipelined design based on delay distributions of individual pipe stages. Using the proposed models, we have shown that change in logic depth and imbalance between the stage delays can improve the yield of a pipeline. A statistical methodology has been developed to optimally design a pipeline circuit for enhancing yield. Optimization results show that, proper imbalance among the stage delays in a pipeline improves design yield by 9% for the same area and performance (and area reduction by about 8.4% under a yield constraint) over a balanced design. △ Less

Submitted 25 October, 2007; originally announced October 2007.

Comments: Submitted on behalf of EDAA (http://www.edaa.com/)

Journal ref: Dans Design, Automation and Test in Europe - DATE'05, Munich : Allemagne (2005)