Title:How Compressible are Sparse Innovation Processes?

Abstract:The sparsity and compressibility of finite-dimensional signals are of great interest in fields such as compressed sensing. The notion of compressibility is also extended to infinite sequences of i.i.d. or ergodic random variables based on the observed error in their nonlinear k-term approximation. In this work, we use the entropy measure to study the compressibility of continuous-domain innovation processes (alternatively known as white noise). Specifically, we define such a measure as the entropy limit of the doubly quantized (time and amplitude) process by identifying divergent terms and extracting the convergent part. While the converging part determines the introduced entropy, the diverging terms provide a tool to compare the compressibility of various innovation processes. In particular, we study stable, and impulsive Poisson innovation processes representing various type of distributions. Our results recognize Poisson innovations as the most compressible one with an entropy measure far below that of stable innovations. While this result departs from the previous knowledge regarding the compressibility of fat-tailed distributions, our entropy measure ranks stable innovations according to their tail decay.