-
Controlled privacy leakage propagation throughout overlapping grouped learning
Authors:
Shahrzad Kiani,
Franziska Boenisch,
Stark C. Draper
Abstract:
Federated Learning (FL) is the standard protocol for collaborative learning. In FL, multiple workers jointly train a shared model. They exchange model updates calculated on their data, while keeping the raw data itself local. Since workers naturally form groups based on common interests and privacy policies, we are motivated to extend standard FL to reflect a setting with multiple, potentially ove…
▽ More
Federated Learning (FL) is the standard protocol for collaborative learning. In FL, multiple workers jointly train a shared model. They exchange model updates calculated on their data, while keeping the raw data itself local. Since workers naturally form groups based on common interests and privacy policies, we are motivated to extend standard FL to reflect a setting with multiple, potentially overlapping groups. In this setup where workers can belong and contribute to more than one group at a time, complexities arise in understanding privacy leakage and in adhering to privacy policies. To address the challenges, we propose differential private overlapping grouped learning (DPOGL), a novel method to implement privacy guarantees within overlapping groups. Under the honest-but-curious threat model, we derive novel privacy guarantees between arbitrary pairs of workers. These privacy guarantees describe and quantify two key effects of privacy leakage in DP-OGL: propagation delay, i.e., the fact that information from one group will leak to other groups only with temporal offset through the common workers and information degradation, i.e., the fact that noise addition over model updates limits information leakage between workers. Our experiments show that applying DP-OGL enhances utility while maintaining strong privacy compared to standard FL setups.
△ Less
Submitted 5 March, 2025;
originally announced March 2025.
-
Differentially Private Federated Learning With Time-Adaptive Privacy Spending
Authors:
Shahrzad Kiani,
Nupur Kulkarni,
Adam Dziedzic,
Stark Draper,
Franziska Boenisch
Abstract:
Federated learning (FL) with differential privacy (DP) provides a framework for collaborative machine learning, enabling clients to train a shared model while adhering to strict privacy constraints. The framework allows each client to have an individual privacy guarantee, e.g., by adding different amounts of noise to each client's model updates. One underlying assumption is that all clients spend…
▽ More
Federated learning (FL) with differential privacy (DP) provides a framework for collaborative machine learning, enabling clients to train a shared model while adhering to strict privacy constraints. The framework allows each client to have an individual privacy guarantee, e.g., by adding different amounts of noise to each client's model updates. One underlying assumption is that all clients spend their privacy budgets uniformly over time (learning rounds). However, it has been shown in the literature that learning in early rounds typically focuses on more coarse-grained features that can be learned at lower signal-to-noise ratios while later rounds learn fine-grained features that benefit from higher signal-to-noise ratios. Building on this intuition, we propose a time-adaptive DP-FL framework that expends the privacy budget non-uniformly across both time and clients. Our framework enables each client to save privacy budget in early rounds so as to be able to spend more in later rounds when additional accuracy is beneficial in learning more fine-grained features. We theoretically prove utility improvements in the case that clients with stricter privacy budgets spend budgets unevenly across rounds, compared to clients with more relaxed budgets, who have sufficient budgets to distribute their spend more evenly. Our practical experiments on standard benchmark datasets support our theoretical results and show that, in practice, our algorithms improve the privacy-utility trade-offs compared to baseline schemes.
△ Less
Submitted 25 February, 2025;
originally announced February 2025.
-
Counterfactual Prediction Under Selective Confounding
Authors:
Sohaib Kiani,
Jared Barton,
Jon Sushinsky,
Lynda Heimbach,
Bo Luo
Abstract:
This research addresses the challenge of conducting interpretable causal inference between a binary treatment and its resulting outcome when not all confounders are known. Confounders are factors that have an influence on both the treatment and the outcome. We relax the requirement of knowing all confounders under desired treatment, which we refer to as Selective Confounding, to enable causal infe…
▽ More
This research addresses the challenge of conducting interpretable causal inference between a binary treatment and its resulting outcome when not all confounders are known. Confounders are factors that have an influence on both the treatment and the outcome. We relax the requirement of knowing all confounders under desired treatment, which we refer to as Selective Confounding, to enable causal inference in diverse real-world scenarios. Our proposed scheme is designed to work in situations where multiple decision-makers with different policies are involved and where there is a re-evaluation mechanism after the initial decision to ensure consistency. These assumptions are more practical to fulfill compared to the availability of all confounders under all treatments. To tackle the issue of Selective Confounding, we propose the use of dual-treatment samples. These samples allow us to employ two-step procedures, such as Regression Adjustment or Doubly-Robust, to learn counterfactual predictors. We provide both theoretical error bounds and empirical evidence of the effectiveness of our proposed scheme using synthetic and real-world child placement data. Furthermore, we introduce three evaluation methods specifically tailored to assess the performance in child placement scenarios. By emphasizing transparency and interpretability, our approach aims to provide decision-makers with a valuable tool. The source code repository of this work is located at https://github.com/sohaib730/CausalML.
△ Less
Submitted 21 October, 2023;
originally announced October 2023.
-
Hierarchical coded elastic computing
Authors:
Shahrzad Kiani,
Tharindu Adikari,
Stark C. Draper
Abstract:
Elasticity is offered by cloud service providers to exploit under-utilized computing resources. The low-cost elastic nodes can leave and join any time during the computation cycle. The possibility of elastic events occurring together with the problem of slow nodes, referred to as stragglers, increases the uncertainty of the system, leading to computation delay. Recent results have shown that coded…
▽ More
Elasticity is offered by cloud service providers to exploit under-utilized computing resources. The low-cost elastic nodes can leave and join any time during the computation cycle. The possibility of elastic events occurring together with the problem of slow nodes, referred to as stragglers, increases the uncertainty of the system, leading to computation delay. Recent results have shown that coded computing can be used to reduce the negative effect of elasticity and stragglers. In this paper, we propose two hierarchical coded elastic computing schemes that can further speed up the system by exploiting stragglers and effectively allocating tasks among available nodes. In our simulations, our scheme realizes 45% improvement in average finishing time compared to the state-of-the-art coded elastic computing scheme.
△ Less
Submitted 23 June, 2022; v1 submitted 19 June, 2022;
originally announced June 2022.
-
Successive Approximation Coding for Distributed Matrix Multiplication
Authors:
Shahrzad Kiani,
Stark C. Draper
Abstract:
Coded distributed computing was recently introduced to mitigate the effect of stragglers on distributed computing. This paper combines ideas of approximate computing with coded computing to further accelerate computation. We propose successive approximation coding (SAC) techniques that realize a tradeoff between accuracy and speed, allowing the distributed computing system to produce approximation…
▽ More
Coded distributed computing was recently introduced to mitigate the effect of stragglers on distributed computing. This paper combines ideas of approximate computing with coded computing to further accelerate computation. We propose successive approximation coding (SAC) techniques that realize a tradeoff between accuracy and speed, allowing the distributed computing system to produce approximations that increase in accuracy over time. If a sufficient number of compute nodes finish their tasks, SAC exactly recovers the desired computation. We theoretically provide design guidelines for our SAC techniques, and numerically show that SAC achieves a better accuracy-speed tradeoff in comparison with previous methods.
△ Less
Submitted 10 January, 2022;
originally announced January 2022.
-
Two Souls in an Adversarial Image: Towards Universal Adversarial Example Detection using Multi-view Inconsistency
Authors:
Sohaib Kiani,
Sana Awan,
Chao Lan,
Fengjun Li,
Bo Luo
Abstract:
In the evasion attacks against deep neural networks (DNN), the attacker generates adversarial instances that are visually indistinguishable from benign samples and sends them to the target DNN to trigger misclassifications. In this paper, we propose a novel multi-view adversarial image detector, namely Argos, based on a novel observation. That is, there exist two "souls" in an adversarial instance…
▽ More
In the evasion attacks against deep neural networks (DNN), the attacker generates adversarial instances that are visually indistinguishable from benign samples and sends them to the target DNN to trigger misclassifications. In this paper, we propose a novel multi-view adversarial image detector, namely Argos, based on a novel observation. That is, there exist two "souls" in an adversarial instance, i.e., the visually unchanged content, which corresponds to the true label, and the added invisible perturbation, which corresponds to the misclassified label. Such inconsistencies could be further amplified through an autoregressive generative approach that generates images with seed pixels selected from the original image, a selected label, and pixel distributions learned from the training data. The generated images (i.e., the "views") will deviate significantly from the original one if the label is adversarial, demonstrating inconsistencies that Argos expects to detect. To this end, Argos first amplifies the discrepancies between the visual content of an image and its misclassified label induced by the attack using a set of regeneration mechanisms and then identifies an image as adversarial if the reproduced views deviate to a preset degree. Our experimental results show that Argos significantly outperforms two representative adversarial detectors in both detection accuracy and robustness against six well-known adversarial attacks. Code is available at: https://github.com/sohaib730/Argos-Adversarial_Detection
△ Less
Submitted 11 October, 2021; v1 submitted 25 September, 2021;
originally announced September 2021.
-
Hierarchical Coded Matrix Multiplication
Authors:
Shahrzad Kiani,
Nuwan Ferdinand,
Stark C. Draper
Abstract:
In distributed computing systems slow working nodes, known as stragglers, can greatly extend finishing times. Coded computing is a technique that enables straggler-resistant computation. Most coded computing techniques presented to date provide robustness by ensuring that the time to finish depends only on a set of the fastest nodes. However, while stragglers do compute less work than non-straggle…
▽ More
In distributed computing systems slow working nodes, known as stragglers, can greatly extend finishing times. Coded computing is a technique that enables straggler-resistant computation. Most coded computing techniques presented to date provide robustness by ensuring that the time to finish depends only on a set of the fastest nodes. However, while stragglers do compute less work than non-stragglers, in real-world commercial cloud computing systems (e.g., Amazon's Elastic Compute Cloud (EC2)) the distinction is often a soft one. In this paper, we develop hierarchical coded computing that exploits the work completed by all nodes, both fast and slow, automatically integrating the potential contribution of each. We first present a conceptual framework to represent the division of work amongst nodes in coded matrix multiplication as a cuboid partitioning problem. This framework allows us to unify existing methods and motivates new techniques. We then develop three methods of hierarchical coded computing that we term bit-interleaved coded computation (BICC), multilevel coded computation (MLCC), and hybrid hierarchical coded computation (HHCC). In this paradigm, each worker is tasked with completing a sequence (a hierarchy) of ordered subtasks. The sequence of subtasks, and the complexity of each, is designed so that partial work completed by stragglers can be used, rather than ignored. We note that our methods can be used in conjunction with any coded computing method. We illustrate this by showing how we can use our methods to accelerate all previously developed coded computing techniques by enabling them to exploit stragglers. Under a widely studied statistical model of completion time, our approach realizes a $66\%$ improvement in the expected finishing time. On Amazon EC2, the gain was $27\%$ when stragglers are simulated.
△ Less
Submitted 30 January, 2021; v1 submitted 14 December, 2019;
originally announced December 2019.
-
Cuboid Partitioning for Hierarchical Coded Matrix Multiplication
Authors:
Shahrzad Kiani,
Nuwan Ferdinand,
Stark C. Draper
Abstract:
Coded matrix multiplication is a technique to enable straggler-resistant multiplication of large matrices in distributed computing systems. In this paper, we first present a conceptual framework to represent the division of work amongst processors in coded matrix multiplication as a cuboid partitioning problem. This framework allows us to unify existing methods and motivates new techniques. Buildi…
▽ More
Coded matrix multiplication is a technique to enable straggler-resistant multiplication of large matrices in distributed computing systems. In this paper, we first present a conceptual framework to represent the division of work amongst processors in coded matrix multiplication as a cuboid partitioning problem. This framework allows us to unify existing methods and motivates new techniques. Building on this framework, we apply the idea of hierarchical coding (Ferdinand & Draper, 2018) to coded matrix multiplication. The hierarchical scheme we develop is able to exploit the work completed by all processors (fast and slow), rather than ignoring the slow ones, even if the amount of work completed by stragglers is much less than that completed by the fastest workers. On Amazon EC2, we achieve a 37% improvement in average finishing time compared to non-hierarchical schemes.
△ Less
Submitted 20 July, 2019;
originally announced July 2019.
-
Hierarchical Coded Matrix Multiplication
Authors:
Shahrzad Kiani,
Nuwan Ferdinand,
Stark C. Draper
Abstract:
Slow working nodes, known as stragglers, can greatly reduce the speed of distributed computation. Coded matrix multiplication is a recently introduced technique that enables straggler-resistant distributed multiplication of large matrices. A key property is that the finishing time depends only on the work completed by a set of the fastest workers, while the work done by the slowest workers is igno…
▽ More
Slow working nodes, known as stragglers, can greatly reduce the speed of distributed computation. Coded matrix multiplication is a recently introduced technique that enables straggler-resistant distributed multiplication of large matrices. A key property is that the finishing time depends only on the work completed by a set of the fastest workers, while the work done by the slowest workers is ignored completely. This paper is motivated by the observation that in real-world commercial cloud computing systems such as Amazon's Elastic Compute Cloud (EC2) the distinction between fast and slow nodes is often a soft one. Thus, if we could also exploit the work completed by stragglers we may realize substantial performance gains. To realize such gains, in this paper we use the idea of hierarchical coding (Ferdinand and Draper, IEEE Int. Symp. Inf. Theory, 2018). We decompose the overall matrix multiplication task into a hierarchy of heterogeneously sized subtasks. The duty to complete each subtask is shared amongst all workers and each subtask is (generally) of a different complexity. The motivation for the hierarchical decomposition is the recognition that more workers will finish the first subtask than the second (or third, forth, etc.). Connecting to error correction coding, earlier subtasks can therefore be designed to be of a higher rate than later subtasks. Through this hierarchical design our scheme exploits the work completed by stragglers, rather than ignoring it, even if that amount is much less than that completed by the fastest workers. We numerically show that our method realizes a 60% improvement in the expected finishing time for a widely studied statistical model of the speed of computation and, on Amazon EC2, the gain is 35%.
△ Less
Submitted 20 July, 2019;
originally announced July 2019.
-
Exploitation of Stragglers in Coded Computation
Authors:
Shahrzad Kiani,
Nuwan Ferdinand,
Stark C. Draper
Abstract:
In cloud computing systems slow processing nodes, often referred to as "stragglers", can significantly extend the computation time. Recent results have shown that error correction coding can be used to reduce the effect of stragglers. In this work we introduce a scheme that, in addition to using error correction to distribute mixed jobs across nodes, is also able to exploit the work completed by a…
▽ More
In cloud computing systems slow processing nodes, often referred to as "stragglers", can significantly extend the computation time. Recent results have shown that error correction coding can be used to reduce the effect of stragglers. In this work we introduce a scheme that, in addition to using error correction to distribute mixed jobs across nodes, is also able to exploit the work completed by all nodes, including stragglers. We first consider vector-matrix multiplication and apply maximum distance separable (MDS) codes to small blocks of sub-matrices. The worker nodes process blocks sequentially, working block-by-block, transmitting partial per-block results to the master as they are completed. Sub-blocking allows a more continuous completion process, which thereby allows us to exploit the work of a much broader spectrum of processors and reduces computation time. We then apply this technique to matrix-matrix multiplication using product code. In this case, we show that the order of computing sub-tasks is a new degree of design freedom that can be exploited to reduce computation time further. We propose a novel approach to analyze the finishing time, which is different from typical order statistics. Simulation results show that the expected computation time decreases by a factor of at least two in compared to previous methods.
△ Less
Submitted 26 June, 2018;
originally announced June 2018.
-
Iterative method for simultaneous sparse approximation
Authors:
Sahar Sadrizadeh,
Shahrzad Kiani,
Mahdi Boloursaz,
Farokh Marvasti
Abstract:
This paper studies the problem of Simultaneous Sparse Approximation (SSA). This problem arises in many applications which work with multiple signals maintaining some degree of dependency such as radar and sensor networks. In this paper, we introduce a new method towards joint recovery of several independent sparse signals with the same support. We provide an analytical discussion on the convergenc…
▽ More
This paper studies the problem of Simultaneous Sparse Approximation (SSA). This problem arises in many applications which work with multiple signals maintaining some degree of dependency such as radar and sensor networks. In this paper, we introduce a new method towards joint recovery of several independent sparse signals with the same support. We provide an analytical discussion on the convergence of our method called Simultaneous Iterative Method with Adaptive Thresholding (SIMAT). Additionally, we compare our method with other group-sparse reconstruction techniques, i.e., Simultaneous Orthogonal Matching Pursuit (SOMP), and Block Iterative Method with Adaptive Thresholding (BIMAT) through numerical experiments. The simulation results demonstrate that SIMAT outperforms these algorithms in terms of the metrics Signal to Noise Ratio (SNR) and Success Rate (SR). Moreover, SIMAT is considerably less complicated than BIMAT, which makes it feasible for practical applications such as implementation in MIMO radar systems.
△ Less
Submitted 3 April, 2023; v1 submitted 26 July, 2017;
originally announced July 2017.
-
Heart Rate Tracking using Wrist-Type Photoplethysmographic (PPG) Signals during Physical Exercise with Simultaneous Accelerometry
Authors:
Mahdi Boloursaz Mashhadi,
Ehsan Asadi,
Mohsen Eskandari,
Shahrzad Kiani,
Farrokh Marvasti
Abstract:
This paper considers the problem of casual heart rate tracking during intensive physical exercise using simultaneous 2 channel photoplethysmographic (PPG) and 3 dimensional (3D) acceleration signals recorded from wrist. This is a challenging problem because the PPG signals recorded from wrist during exercise are contaminated by strong Motion Artifacts (MAs). In this work, a novel algorithm is prop…
▽ More
This paper considers the problem of casual heart rate tracking during intensive physical exercise using simultaneous 2 channel photoplethysmographic (PPG) and 3 dimensional (3D) acceleration signals recorded from wrist. This is a challenging problem because the PPG signals recorded from wrist during exercise are contaminated by strong Motion Artifacts (MAs). In this work, a novel algorithm is proposed which consists of two main steps of MA Cancellation and Spectral Analysis. The MA cancellation step cleanses the MA-contaminated PPG signals utilizing the acceleration data and the spectral analysis step estimates a higher resolution spectrum of the signal and selects the spectral peaks corresponding to HR. Experimental results on datasets recorded from 12 subjects during fast running at the peak speed of 15 km/hour showed that the proposed algorithm achieves an average absolute error of 1.25 beat per minute (BPM). These experimental results also confirm that the proposed algorithm keeps high estimation accuracies even in strong MA conditions.
△ Less
Submitted 12 September, 2016; v1 submitted 18 April, 2015;
originally announced April 2015.
-
An Integrated e-science Analysis Base for Computation Neuroscience Experiments and Analysis
Authors:
Kamran Munir,
Saad Liaquat Kiani,
Khawar Hasham,
Richard McClatchey,
Andrew Branson,
Jetendr Shamdasani,
the N4U Consortium
Abstract:
Recent developments in data management and imaging technologies have significantly affected diagnostic and extrapolative research in the understanding of neurodegenerative diseases. However, the impact of these new technologies is largely dependent on the speed and reliability with which the medical data can be visualised, analysed and interpreted. The EUs neuGRID for Users (N4U) is a follow-on pr…
▽ More
Recent developments in data management and imaging technologies have significantly affected diagnostic and extrapolative research in the understanding of neurodegenerative diseases. However, the impact of these new technologies is largely dependent on the speed and reliability with which the medical data can be visualised, analysed and interpreted. The EUs neuGRID for Users (N4U) is a follow-on project to neuGRID, which aims to provide an integrated environment to carry out computational neuroscience experiments. This paper reports on the design and development of the N4U Analysis Base and related Information Services, which addresses existing research and practical challenges by offering an integrated medical data analysis environment with the necessary building blocks for neuroscientists to optimally exploit neuroscience workflows, large image datasets and algorithms in order to conduct analyses. The N4U Analysis Base enables such analyses by indexing and interlinking the neuroimaging and clinical study datasets stored on the N4U Grid infrastructure, algorithms and scientific workflow definitions along with their associated provenance information.
△ Less
Submitted 24 February, 2014;
originally announced February 2014.
-
Context-Aware Service Utilisation in the Clouds and Energy Conservation
Authors:
Saad Liaquat Kiani,
Ashiq Anjum,
Nick Antonopoulos,
Michael Knappmeyer,
Nigel Baker,
Richard McClatchey
Abstract:
Ubiquitous computing environments are characterised by smart, interconnected artefacts embedded in our physical world that are projected to provide useful services to human inhabitants unobtrusively. Mobile devices are becoming the primary tools of human interaction with these embedded artefacts and utilisation of services available in smart computing environments such as clouds. Advancements in c…
▽ More
Ubiquitous computing environments are characterised by smart, interconnected artefacts embedded in our physical world that are projected to provide useful services to human inhabitants unobtrusively. Mobile devices are becoming the primary tools of human interaction with these embedded artefacts and utilisation of services available in smart computing environments such as clouds. Advancements in capabilities of mobile devices allow a number of user and environment related context consumers to be hosted on these devices. Without a coordinating component, these context consumers and providers are a potential burden on device resources; specifically the effect of uncoordinated computation and communication with cloud-enabled services can negatively impact the battery life. Therefore energy conservation is a major concern in realising the collaboration and utilisation of mobile device based context-aware applications and cloud based services. This paper presents the concept of a context-brokering component to aid in coordination and communication of context information between mobile devices and services deployed in a cloud infrastructure. A prototype context broker is experimentally analysed for effects on energy conservation when accessing and coordinating with cloud services on a smart device, with results signifying reduction in energy consumption.
△ Less
Submitted 24 February, 2012;
originally announced February 2012.
-
DIAMOnDS - DIstributed Agents for MObile & Dynamic Services
Authors:
Aamir Shafi,
Umer Farooq,
Saad Kiani,
Maria Riaz,
Anjum Shehzad,
Arshad Ali,
Iosif Legrand,
Harvey Newman
Abstract:
Distributed Services Architecture with support for mobile agents between services, offer significantly improved communication and computational flexibility. The uses of agents allow execution of complex operations that involve large amounts of data to be processed effectively using distributed resources. The prototype system Distributed Agents for Mobile and Dynamic Services (DIAMOnDS), allows a…
▽ More
Distributed Services Architecture with support for mobile agents between services, offer significantly improved communication and computational flexibility. The uses of agents allow execution of complex operations that involve large amounts of data to be processed effectively using distributed resources. The prototype system Distributed Agents for Mobile and Dynamic Services (DIAMOnDS), allows a service to send agents on its behalf, to other services, to perform data manipulation and processing. Agents have been implemented as mobile services that are discovered using the Jini Lookup mechanism and used by other services for task management and communication. Agents provide proxies for interaction with other services as well as specific GUI to monitor and control the agent activity. Thus agents acting on behalf of one service cooperate with other services to carry out a job, providing inter-operation of loosely coupled services in a semi-autonomous way. Remote file system access functionality has been incorporated by the agent framework and allows services to dynamically share and browse the file system resources of hosts, running the services. Generic database access functionality has been implemented in the mobile agent framework that allows performing complex data mining and processing operations efficiently in distributed system. A basic data searching agent is also implemented that performs a query based search in a file system. The testing of the framework was carried out on WAN by moving Connectivity Test agents between AgentStations in CERN, Switzerland and NUST, Pakistan.
△ Less
Submitted 13 June, 2003; v1 submitted 30 May, 2003;
originally announced May 2003.
