-
Generalized Multivariate Polynomial Codes for Distributed Matrix-Matrix Multiplication
Authors:
Jesús Gómez-Vilardebó,
Burak Hasırcıoğlu,
Deniz Gündüz
Abstract:
Supporting multiple partial computations efficiently at each of the workers is a keystone in distributed coded computing in order to speed up computations and to fully exploit the resources of heterogeneous workers in terms of communication, storage, or computation capabilities. Multivariate polynomial coding schemes have recently been shown to deliver faster results for distributed matrix-matrix…
▽ More
Supporting multiple partial computations efficiently at each of the workers is a keystone in distributed coded computing in order to speed up computations and to fully exploit the resources of heterogeneous workers in terms of communication, storage, or computation capabilities. Multivariate polynomial coding schemes have recently been shown to deliver faster results for distributed matrix-matrix multiplication compared to conventional univariate polynomial coding schemes by supporting multiple partial coded computations at each worker at reduced communication costs. In this work, we extend multivariate coding schemes to also support arbitrary matrix partitions. Generalized matrix partitions have been proved useful to trade-off between computation speed and communication costs in distributed (univariate) coded computing. We first formulate the computation latency-communication trade-off in terms of the computation complexity and communication overheads required by coded computing approaches as compared to a single server uncoded computing system. Then, we propose two novel multivariate coded computing schemes supporting arbitrary matrix partitions. The proposed schemes are shown to improve the studied trade-off as compared to univariate schemes.
△ Less
Submitted 22 November, 2024;
originally announced November 2024.
-
Bivariate Polynomial Codes for Secure Distributed Matrix Multiplication
Authors:
Burak Hasircioglu,
Jesus Gomez-Vilardebo,
Deniz Gunduz
Abstract:
We consider the problem of secure distributed matrix multiplication (SDMM). Coded computation has been shown to be an effective solution in distributed matrix multiplication, both providing privacy against workers and boosting the computation speed by efficiently mitigating stragglers. In this work, we present a non-direct secure extension of the recently introduced bivariate polynomial codes. Biv…
▽ More
We consider the problem of secure distributed matrix multiplication (SDMM). Coded computation has been shown to be an effective solution in distributed matrix multiplication, both providing privacy against workers and boosting the computation speed by efficiently mitigating stragglers. In this work, we present a non-direct secure extension of the recently introduced bivariate polynomial codes. Bivariate polynomial codes have been shown to be able to further speed up distributed matrix multiplication by exploiting the partial work done by the stragglers rather than completely ignoring them while reducing the upload communication cost and/or the workers' storage's capacity needs. We show that, especially for upload communication or storage constrained settings, the proposed approach reduces the average computation time of SDMM compared to its competitors in the literature.
△ Less
Submitted 6 February, 2022; v1 submitted 14 June, 2021;
originally announced June 2021.
-
Speeding Up Private Distributed Matrix Multiplication via Bivariate Polynomial Codes
Authors:
Burak Hasircioglu,
Jesus Gomez-Vilardebo,
Deniz Gunduz
Abstract:
We consider the problem of private distributed matrix multiplication under limited resources. Coded computation has been shown to be an effective solution in distributed matrix multiplication, both providing privacy against the workers and boosting the computation speed by efficiently mitigating stragglers. In this work, we propose the use of recently-introduced bivariate polynomial codes to furth…
▽ More
We consider the problem of private distributed matrix multiplication under limited resources. Coded computation has been shown to be an effective solution in distributed matrix multiplication, both providing privacy against the workers and boosting the computation speed by efficiently mitigating stragglers. In this work, we propose the use of recently-introduced bivariate polynomial codes to further speed up private distributed matrix multiplication by exploiting the partial work done by the stragglers rather than completely ignoring them. We show that the proposed approach reduces the average computation time of private distributed matrix multiplication compared to its competitors in the literature while improving the upload communication cost and the workers' storage efficiency.
△ Less
Submitted 13 July, 2021; v1 submitted 16 February, 2021;
originally announced February 2021.
-
On the Fundamental Limits of Coded Caching Systems with Restricted Demand Types
Authors:
Shuo Shao,
Jesús Gómez-Vilardebó,
Kai Zhang,
Chao Tian
Abstract:
Caching is a technique to reduce the communication load in peak hours by prefetching contents during off-peak hours. An information-theoretic framework for coded caching was introduced by Maddah-Ali and Niesen in a recent work, where it was shown that significant improvement can be obtained compared to uncoded caching. Considerable efforts have been devoted to identify the precise information-theo…
▽ More
Caching is a technique to reduce the communication load in peak hours by prefetching contents during off-peak hours. An information-theoretic framework for coded caching was introduced by Maddah-Ali and Niesen in a recent work, where it was shown that significant improvement can be obtained compared to uncoded caching. Considerable efforts have been devoted to identify the precise information-theoretic fundamental limits of the coded caching systems, however the difficulty of this task has also become clear. One of the reasons for this difficulty is that the original coded caching setting allows all possible multiple demand types during delivery, which in fact introduces tension in the coding strategy. In this paper, we seek to develop a better understanding of the fundamental limits of coded caching by investigating systems with certain demand type restrictions. We first consider the canonical three-user three-file system, and show that, contrary to popular beliefs, the worst demand type is not the one in which all three files are requested. Motivated by these findings, we focus on coded caching systems where every file must be requested by at least one user. A novel coding scheme is proposed, which can provide new operating points that are not covered by any previously known schemes.
△ Less
Submitted 4 September, 2021; v1 submitted 30 June, 2020;
originally announced June 2020.
-
Bivariate Polynomial Coding for Efficient Distributed Matrix Multiplication
Authors:
Burak Hasircioglu,
Jesus Gomez-Vilardebo,
Deniz Gunduz
Abstract:
Coded computing is an effective technique to mitigate "stragglers" in large-scale and distributed matrix multiplication. In particular, univariate polynomial codes have been shown to be effective in straggler mitigation by making the computation time depend only on the fastest workers. However, these schemes completely ignore the work done by the straggling workers resulting in a waste of computat…
▽ More
Coded computing is an effective technique to mitigate "stragglers" in large-scale and distributed matrix multiplication. In particular, univariate polynomial codes have been shown to be effective in straggler mitigation by making the computation time depend only on the fastest workers. However, these schemes completely ignore the work done by the straggling workers resulting in a waste of computational resources. To reduce the amount of work left unfinished at workers, one can further decompose the matrix multiplication task into smaller sub-tasks, and assign multiple sub-tasks to each worker, possibly heterogeneously, to better fit their particular storage and computation capacities. In this work, we propose a novel family of bivariate polynomial codes to efficiently exploit the work carried out by straggling workers. We show that bivariate polynomial codes bring significant advantages in terms of upload communication costs and storage efficiency, measured in terms of the number of sub-tasks that can be computed per worker. We propose two bivariate polynomial coding schemes. The first one exploits the fact that bivariate interpolation is always possible on a rectangular grid of evaluation points. We obtain such points at the cost of adding some redundant computations. For the second scheme, we relax the decoding constraints and require decodability for almost all choices of the evaluation points. We present interpolation sets satisfying such decodability conditions for certain storage configurations of workers. Our numerical results show that bivariate polynomial coding considerably reduces the average computation time of distributed matrix multiplication. We believe this work opens up a new class of previously unexplored coding schemes for efficient coded distributed computation.
△ Less
Submitted 18 August, 2021; v1 submitted 20 January, 2020;
originally announced January 2020.
-
Fundamental Limits of Caching: Improved Bounds with Coded Prefetching
Authors:
Jesús Gómez-Vilardebó
Abstract:
We consider a cache network in which a single server is connected to multiple users via a shared error free link. The server has access to a database with $N$ files of equal length $F$, and serves $K$ users each with a cache memory of $MF$ bits. A novel centralized coded caching scheme is proposed for scenarios with more users than files $N\leq K$ and cache capacities satisfying…
▽ More
We consider a cache network in which a single server is connected to multiple users via a shared error free link. The server has access to a database with $N$ files of equal length $F$, and serves $K$ users each with a cache memory of $MF$ bits. A novel centralized coded caching scheme is proposed for scenarios with more users than files $N\leq K$ and cache capacities satisfying $\frac{1}{K}\leq M\leq\frac{N}{K}$. The proposed scheme outperforms the best rate-memory region known in the literature if $N\leq K\leq\frac{N^{2}+1}{2}$.
△ Less
Submitted 23 May, 2017; v1 submitted 29 December, 2016;
originally announced December 2016.
-
Wireless Content Caching for Small Cell and D2D Networks
Authors:
Maria Gregori,
Jesús Gómez-Vilardebó,
Javier Matamoros,
Deniz Gündüz
Abstract:
The fifth generation wireless networks must provide fast and reliable connectivity while coping with the ongoing traffic growth. It is of paramount importance that the required resources, such as energy and bandwidth, do not scale with traffic. While the aggregate network traffic is growing at an unprecedented rate, users tend to request the same popular contents at different time instants. Theref…
▽ More
The fifth generation wireless networks must provide fast and reliable connectivity while coping with the ongoing traffic growth. It is of paramount importance that the required resources, such as energy and bandwidth, do not scale with traffic. While the aggregate network traffic is growing at an unprecedented rate, users tend to request the same popular contents at different time instants. Therefore, caching the most popular contents at the network edge is a promising solution to reduce the traffic and the energy consumption over the backhaul links. In this paper, two scenarios are considered, where caching is performed either at a small base station, or directly at the user terminals, which communicate using \ac{D2D} communications. In both scenarios, joint design of the transmission and caching policies is studied when the user demands are known in advance. This joint design offers two different caching gains, namely, the \textit{pre-downloading} and \textit{local caching gains}. It is shown that the finite cache capacity limits the attainable gains, and creates an inherent tradeoff between the two types of gains. In this context, a continuous time optimization problem is formulated to determine the optimal transmission and caching policies that minimize a generic cost function, such as energy, bandwidth, or throughput. The jointly optimal solution is obtained by demonstrating that caching files at a constant rate is optimal, which allows to reformulate the problem as a finite-dimensional convex program. The numerical results show that the proposed joint transmission and caching policy dramatically reduces the total cost, which is particularised to the total energy consumption at the \ac{MBS}, as well as to the total economical cost for the service provider, when users demand economical incentives for delivering content to other users over the D2D links.
△ Less
Submitted 14 March, 2016;
originally announced March 2016.
-
Smart Meter Privacy for Multiple Users in the Presence of an Alternative Energy Source
Authors:
Jesus Gomez-Vilardebo,
Deniz Gündüz
Abstract:
Smart meters (SMs) measure and report users' energy consumption to the utility provider (UP) in almost real-time, providing a much more detailed depiction of the consumer's energy consumption compared to their analog counterparts. This increased rate of information flow to the UP, together with its many potential benefits, raise important concerns regarding user privacy. This work investigates, fr…
▽ More
Smart meters (SMs) measure and report users' energy consumption to the utility provider (UP) in almost real-time, providing a much more detailed depiction of the consumer's energy consumption compared to their analog counterparts. This increased rate of information flow to the UP, together with its many potential benefits, raise important concerns regarding user privacy. This work investigates, from an information theoretic perspective, the privacy that can be achieved in a multi-user SM system in the presence of an alternative energy source (AES). To measure privacy, we use the mutual information rate between the users' real energy consumption profile and the SM readings that are available to the UP. The objective is to characterize the \textit{privacy-power function}, defined as the minimal information leakage rate that can be obtained with an average power limited AES. We characterize the privacy-power function in a single-letter form when the users' energy demands are assumed to be independent and identically distributed over time. Moreover, for binary and exponentially distributed energy demands, we provide an explicit characterization of the privacy-power function. For any discrete energy demands, we demonstrate that the privacy-power function can always be efficiently evaluated numerically. Finally, for continuous energy demands, we derive an explicit lower-bound on the privacy-power function, which is tight for exponentially distributed loads.
△ Less
Submitted 1 August, 2014;
originally announced August 2014.
