Showing 1–21 of 21 results for author: Lim, F

Adapting Large Language Models for Parameter-Efficient Log Anomaly Detection

Authors: Ying Fu Lim, Jiawen Zhu, Guansong Pang

Abstract: Log Anomaly Detection (LAD) seeks to identify atypical patterns in log data that are crucial to assessing the security and condition of systems. Although Large Language Models (LLMs) have shown tremendous success in various fields, the use of LLMs in enabling the detection of log anomalies is largely unexplored. This work aims to fill this gap. Due to the prohibitive costs involved in fully fine-t… ▽ More Log Anomaly Detection (LAD) seeks to identify atypical patterns in log data that are crucial to assessing the security and condition of systems. Although Large Language Models (LLMs) have shown tremendous success in various fields, the use of LLMs in enabling the detection of log anomalies is largely unexplored. This work aims to fill this gap. Due to the prohibitive costs involved in fully fine-tuning LLMs, we explore the use of parameter-efficient fine-tuning techniques (PEFTs) for adapting LLMs to LAD. To have an in-depth exploration of the potential of LLM-driven LAD, we present a comprehensive investigation of leveraging two of the most popular PEFTs -- Low-Rank Adaptation (LoRA) and Representation Fine-tuning (ReFT) -- to tap into three prominent LLMs of varying size, including RoBERTa, GPT-2, and Llama-3, for parameter-efficient LAD. Comprehensive experiments on four public log datasets are performed to reveal important insights into effective LLM-driven LAD in several key perspectives, including the efficacy of these PEFT-based LLM-driven LAD methods, their stability, sample efficiency, robustness w.r.t. unstable logs, and cross-dataset generalization. Code is available at https://github.com/mala-lab/LogADReft. △ Less

Submitted 11 March, 2025; originally announced March 2025.

Comments: 12 pages, 5 figures, accepted by PAKDD 2025 special session

Operations & Supply Chain Management: Principles and Practice

Authors: Fotios Petropoulos, Henk Akkermans, O. Zeynep Aksin, Imran Ali, Mohamed Zied Babai, Ana Barbosa-Povoa, Olga Battaïa, Maria Besiou, Nils Boysen, Stephen Brammer, Alistair Brandon-Jones, Dirk Briskorn, Tyson R. Browning, Paul Buijs, Piera Centobelli, Andrea Chiarini, Paul Cousins, Elizabeth A. Cudney, Andrew Davies, Steven J. Day, René de Koster, Rommert Dekker, Juliano Denicol, Mélanie Despeisse, Stephen M. Disney , et al. (68 additional authors not shown)

Abstract: Operations and Supply Chain Management (OSCM) has continually evolved, incorporating a broad array of strategies, frameworks, and technologies to address complex challenges across industries. This encyclopedic article provides a comprehensive overview of contemporary strategies, tools, methods, principles, and best practices that define the field's cutting-edge advancements. It also explores the d… ▽ More Operations and Supply Chain Management (OSCM) has continually evolved, incorporating a broad array of strategies, frameworks, and technologies to address complex challenges across industries. This encyclopedic article provides a comprehensive overview of contemporary strategies, tools, methods, principles, and best practices that define the field's cutting-edge advancements. It also explores the diverse environments where OSCM principles have been effectively implemented. The article is meant to be read in a nonlinear fashion. It should be used as a point of reference or first-port-of-call for a diverse pool of readers: academics, researchers, students, and practitioners. △ Less

Submitted 20 February, 2025; originally announced March 2025.

Battling Botpoop using GenAI for Higher Education: A Study of a Retrieval Augmented Generation Chatbots Impact on Learning

Authors: Maung Thway, Jose Recatala-Gomez, Fun Siong Lim, Kedar Hippalgaonkar, Leonard W. T. Ng

Abstract: Generative artificial intelligence (GenAI) and large language models (LLMs) have simultaneously opened new avenues for enhancing human learning and increased the prevalence of poor-quality information in student response - termed Botpoop. This study introduces Professor Leodar, a custom-built, Singlish-speaking Retrieval Augmented Generation (RAG) chatbot designed to enhance educational while redu… ▽ More Generative artificial intelligence (GenAI) and large language models (LLMs) have simultaneously opened new avenues for enhancing human learning and increased the prevalence of poor-quality information in student response - termed Botpoop. This study introduces Professor Leodar, a custom-built, Singlish-speaking Retrieval Augmented Generation (RAG) chatbot designed to enhance educational while reducing Botpoop. Deployed at Nanyang Technological University, Singapore, Professor Leodar offers a glimpse into the future of AI-assisted learning, offering personalized guidance, 24/7 availability, and contextually relevant information. Through a mixed-methods approach, we examine the impact of Professor Leodar on learning, engagement, and exam preparedness, with 97.1% of participants reporting positive experiences. These findings help define possible roles of AI in education and highlight the potential of custom GenAI chatbots. Our combination of chatbot development, in-class deployment and outcomes study offers a benchmark for GenAI educational tools and is a stepping stone for redefining the interplay between AI and human learning. △ Less

Submitted 21 June, 2024; v1 submitted 11 June, 2024; originally announced June 2024.

Comments: 13 pages, 5 figures, SI with Annexes A, B and C upon request

Efficiently Distilling LLMs for Edge Applications

Authors: Achintya Kundu, Fabian Lim, Aaron Chew, Laura Wynter, Penny Chong, Rhui Dih Lee

Abstract: Supernet training of LLMs is of great interest in industrial applications as it confers the ability to produce a palette of smaller models at constant cost, regardless of the number of models (of different size / latency) produced. We propose a new method called Multistage Low-rank Fine-tuning of Super-transformers (MLFS) for parameter-efficient supernet training. We show that it is possible to ob… ▽ More Supernet training of LLMs is of great interest in industrial applications as it confers the ability to produce a palette of smaller models at constant cost, regardless of the number of models (of different size / latency) produced. We propose a new method called Multistage Low-rank Fine-tuning of Super-transformers (MLFS) for parameter-efficient supernet training. We show that it is possible to obtain high-quality encoder models that are suitable for commercial edge applications, and that while decoder-only models are resistant to a comparable degree of compression, decoders can be effectively sliced for a significant reduction in training time. △ Less

Submitted 1 April, 2024; originally announced April 2024.

Comments: This paper has been accepted for publication in NAACL 2024 (Industry Track)

Visually Improved Erosion Algorithm for the Procedural Generation of Tile-based Terrain

Authors: Fong Yuan Lim, Yu Wei Tan, Anand Bhojan

Abstract: Procedural terrain generation is the process of generating a digital representation of terrain using a computer program or procedure, with little to no human guidance. This paper proposes a procedural terrain generation algorithm based on a graph representation of fluvial erosion that offers several novel improvements over existing algorithms. Namely, the use of a height constraint map with two ty… ▽ More Procedural terrain generation is the process of generating a digital representation of terrain using a computer program or procedure, with little to no human guidance. This paper proposes a procedural terrain generation algorithm based on a graph representation of fluvial erosion that offers several novel improvements over existing algorithms. Namely, the use of a height constraint map with two types of locally defined constraint strengths; the ability to specify a realistic erosion strength via level of rainfall; and the ability to carve realistic gorges. These novelties allow it to generate more varied and realistic terrain by integrating additional parameters and simulation processes, while being faster and offering more flexibility and ease of use to terrain designers due to the nature and intuitiveness of these new parameters and processes. This paper additionally reviews some common metrics used to evaluate terrain generators, and suggests a completely new one that contributes to a more holistic evaluation. △ Less

Submitted 26 October, 2022; originally announced October 2022.

ACM Class: I.3.5

Order Constraints in Optimal Transport

Authors: Fabian Lim, Laura Wynter, Shiau Hong Lim

Abstract: Optimal transport is a framework for comparing measures whereby a cost is incurred for transporting one measure to another. Recent works have aimed to improve optimal transport plans through the introduction of various forms of structure. We introduce novel order constraints into the optimal transport formulation to allow for the incorporation of structure. We define an efficient method for obtain… ▽ More Optimal transport is a framework for comparing measures whereby a cost is incurred for transporting one measure to another. Recent works have aimed to improve optimal transport plans through the introduction of various forms of structure. We introduce novel order constraints into the optimal transport formulation to allow for the incorporation of structure. We define an efficient method for obtaining explainable solutions to the new formulation that scales far better than standard approaches. The theoretical properties of the method are provided. We demonstrate experimentally that order constraints improve explainability using the e-SNLI (Stanford Natural Language Inference) dataset that includes human-annotated rationales as well as on several image color transfer examples. △ Less

Submitted 28 June, 2022; v1 submitted 14 October, 2021; originally announced October 2021.

Comments: To appear in Proceedings of ICML 2022. Main Paper + Supplementary

Handling Background Noise in Neural Speech Generation

Authors: Tom Denton, Alejandro Luebs, Felicia S. C. Lim, Andrew Storus, Hengchin Yeh, W. Bastiaan Kleijn, Jan Skoglund

Abstract: Recent advances in neural-network based generative modeling of speech has shown great potential for speech coding. However, the performance of such models drops when the input is not clean speech, e.g., in the presence of background noise, preventing its use in practical applications. In this paper we examine the reason and discuss methods to overcome this issue. Placing a denoising preprocessing… ▽ More Recent advances in neural-network based generative modeling of speech has shown great potential for speech coding. However, the performance of such models drops when the input is not clean speech, e.g., in the presence of background noise, preventing its use in practical applications. In this paper we examine the reason and discuss methods to overcome this issue. Placing a denoising preprocessing stage when extracting features and target clean speech during training is shown to be the best performing strategy. △ Less

Submitted 23 February, 2021; originally announced February 2021.

Comments: 5 pages, 3 figures, presented at the Asilomar Conference on Signals, Systems, and Computers 2020

Generative Speech Coding with Predictive Variance Regularization

Authors: W. Bastiaan Kleijn, Andrew Storus, Michael Chinen, Tom Denton, Felicia S. C. Lim, Alejandro Luebs, Jan Skoglund, Hengchin Yeh

Abstract: The recent emergence of machine-learning based generative models for speech suggests a significant reduction in bit rate for speech codecs is possible. However, the performance of generative models deteriorates significantly with the distortions present in real-world input signals. We argue that this deterioration is due to the sensitivity of the maximum likelihood criterion to outliers and the in… ▽ More The recent emergence of machine-learning based generative models for speech suggests a significant reduction in bit rate for speech codecs is possible. However, the performance of generative models deteriorates significantly with the distortions present in real-world input signals. We argue that this deterioration is due to the sensitivity of the maximum likelihood criterion to outliers and the ineffectiveness of modeling a sum of independent signals with a single autoregressive model. We introduce predictive-variance regularization to reduce the sensitivity to outliers, resulting in a significant increase in performance. We show that noise reduction to remove unwanted signals can significantly increase performance. We provide extensive subjective performance evaluations that show that our system based on generative modeling provides state-of-the-art coding performance at 3 kb/s for real-world speech signals at reasonable computational complexity. △ Less

Submitted 18 February, 2021; originally announced February 2021.

MSC Class: 94 ACM Class: I.m

ViSQOL v3: An Open Source Production Ready Objective Speech and Audio Metric

Authors: Michael Chinen, Felicia S. C. Lim, Jan Skoglund, Nikita Gureev, Feargus O'Gorman, Andrew Hines

Abstract: Estimation of perceptual quality in audio and speech is possible using a variety of methods. The combined v3 release of ViSQOL and ViSQOLAudio (for speech and audio, respectively,) provides improvements upon previous versions, in terms of both design and usage. As an open source C++ library or binary with permissive licensing, ViSQOL can now be deployed beyond the research context into production… ▽ More Estimation of perceptual quality in audio and speech is possible using a variety of methods. The combined v3 release of ViSQOL and ViSQOLAudio (for speech and audio, respectively,) provides improvements upon previous versions, in terms of both design and usage. As an open source C++ library or binary with permissive licensing, ViSQOL can now be deployed beyond the research context into production usage. The feedback from internal production teams at Google has helped to improve this new release, and serves to show cases where it is most applicable, as well as to highlight limitations. The new model is benchmarked against real-world data for evaluation purposes. The trends and direction of future work is discussed. △ Less

Submitted 20 April, 2020; originally announced April 2020.

Comments: 2020 Twelfth International Conference on Quality of Multimedia Experience (QoMEX)

Low Bit-Rate Speech Coding with VQ-VAE and a WaveNet Decoder

Authors: Cristina Gârbacea, Aäron van den Oord, Yazhe Li, Felicia S C Lim, Alejandro Luebs, Oriol Vinyals, Thomas C Walters

Abstract: In order to efficiently transmit and store speech signals, speech codecs create a minimally redundant representation of the input signal which is then decoded at the receiver with the best possible perceptual quality. In this work we demonstrate that a neural network architecture based on VQ-VAE with a WaveNet decoder can be used to perform very low bit-rate speech coding with high reconstruction… ▽ More In order to efficiently transmit and store speech signals, speech codecs create a minimally redundant representation of the input signal which is then decoded at the receiver with the best possible perceptual quality. In this work we demonstrate that a neural network architecture based on VQ-VAE with a WaveNet decoder can be used to perform very low bit-rate speech coding with high reconstruction quality. A prosody-transparent and speaker-independent model trained on the LibriSpeech corpus coding audio at 1.6 kbps exhibits perceptual quality which is around halfway between the MELP codec at 2.4 kbps and AMR-WB codec at 23.05 kbps. In addition, when training on high-quality recorded speech with the test speaker included in the training set, a model coding speech at 1.6 kbps produces output of similar perceptual quality to that generated by AMR-WB at 23.05 kbps. △ Less

Submitted 14 October, 2019; originally announced October 2019.

Comments: ICASSP 2019

Journal ref: ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 735-739. IEEE, 2019

Generative Speech Enhancement Based on Cloned Networks

Authors: Michael Chinen, W. Bastiaan Kleijn, Felicia S. C. Lim, Jan Skoglund

Abstract: We propose to implement speech enhancement by the regeneration of clean speech from a salient representation extracted from the noisy signal. The network that extracts salient features is trained using a set of weight-sharing clones of the extractor network. The clones receive mel-frequency spectra of different noisy versions of the same speech signal as input. By encouraging the outputs of the cl… ▽ More We propose to implement speech enhancement by the regeneration of clean speech from a salient representation extracted from the noisy signal. The network that extracts salient features is trained using a set of weight-sharing clones of the extractor network. The clones receive mel-frequency spectra of different noisy versions of the same speech signal as input. By encouraging the outputs of the clones to be similar for these different input signals, we train a feature extractor network that is robust to noise. At inference, the salient features form the input to a WaveNet network that generates a natural and clean speech signal with the same attributes as the ground-truth clean signal. As the signal becomes noisier, our system produces natural sounding errors that stay on the speech manifold, in place of traditional artifacts found in other systems. Our experiments confirm that our generative enhancement system provides state-of-the-art enhancement performance within the generative class of enhancers according to a MUSHRA-like test. The clones based system matches or outperforms the other systems at each input signal-to-noise (SNR) range with statistical significance. △ Less

Submitted 10 September, 2019; originally announced September 2019.

Comments: Accepted WASPAA 2019

Salient Speech Representations Based on Cloned Networks

Authors: W. Bastiaan Kleijn, Felicia S. C. Lim, Michael Chinen, Jan Skoglund

Abstract: We define salient features as features that are shared by signals that are defined as being equivalent by a system designer. The definition allows the designer to contribute qualitative information. We aim to find salient features that are useful as conditioning for generative networks. We extract salient features by jointly training a set of clones of an encoder network. Each network clone receiv… ▽ More We define salient features as features that are shared by signals that are defined as being equivalent by a system designer. The definition allows the designer to contribute qualitative information. We aim to find salient features that are useful as conditioning for generative networks. We extract salient features by jointly training a set of clones of an encoder network. Each network clone receives as input a different signal from a set of equivalent signals. The objective function encourages the network clones to map their input into a set of features that is identical across the clones. It additionally encourages feature independence and, optionally, reconstruction of a desired target signal by a decoder. As an application, we train a system that extracts a time-sequence of feature vectors of speech and uses it as a conditioning of a WaveNet generative system, facilitating both coding and enhancement. △ Less

Submitted 19 August, 2019; originally announced August 2019.

Comments: Interspeech 2019

Wavenet based low rate speech coding

Authors: W. Bastiaan Kleijn, Felicia S. C. Lim, Alejandro Luebs, Jan Skoglund, Florian Stimberg, Quan Wang, Thomas C. Walters

Abstract: Traditional parametric coding of speech facilitates low rate but provides poor reconstruction quality because of the inadequacy of the model used. We describe how a WaveNet generative speech model can be used to generate high quality speech from the bit stream of a standard parametric coder operating at 2.4 kb/s. We compare this parametric coder with a waveform coder based on the same generative m… ▽ More Traditional parametric coding of speech facilitates low rate but provides poor reconstruction quality because of the inadequacy of the model used. We describe how a WaveNet generative speech model can be used to generate high quality speech from the bit stream of a standard parametric coder operating at 2.4 kb/s. We compare this parametric coder with a waveform coder based on the same generative model and show that approximating the signal waveform incurs a large rate penalty. Our experiments confirm the high performance of the WaveNet based coder and show that the speech produced by the system is able to additionally perform implicit bandwidth extension and does not significantly impair recognition of the original speaker for the human listener, even when that speaker has not been used during the training of the generative model. △ Less

Submitted 1 December, 2017; originally announced December 2017.

Comments: 5 pages, 2 figures

Persuasive Teachable Agent for Intergenerational Learning

Authors: Su Fang Lim

Abstract: Teachable agents are computer agents based on the pedagogical concept of learning-by-teaching. During the tutoring process, where students take on the role of the tutor to teach a computer agent tutee, learners have been observed to gain deeper understanding of the subject matter. Teachable agents are commonly used in the areas of science and mathematics learning where learners are able to learn c… ▽ More Teachable agents are computer agents based on the pedagogical concept of learning-by-teaching. During the tutoring process, where students take on the role of the tutor to teach a computer agent tutee, learners have been observed to gain deeper understanding of the subject matter. Teachable agents are commonly used in the areas of science and mathematics learning where learners are able to learn complex concepts and deep reasoning by teaching the teachable agent through graphic representation such as concept maps. Literature review on teachable agents as well as observations during field studies conducted by the researcher, have shown that many current teachable agents lack the interaction abilities required to keep learners engage in learning tasks. The result of this is learners deviating from the teaching process, and thus the learners are unable to benefit fully from learning with the teachable agent. The applications of teachable agents are restricted to the learning of academic subjects such as mathematics and science. In this book, we have proposed the Persuasive Teachable Agent (PTA), a teachable agent based on the theoretical framework of persuasion, computational and goal-oriented agent modelling. We argue that the PTA, an autonomous agent, capable of encouraging attitude and behavioural change can offer a more meaningful and engaging learning experiences for learners from different age groups. Based on the findings from our research we argue that persuasive feedback actions generated by the PTA provide significant influence over learner's decision to participate in intergenerational learning. The PTA plays a crucial role in the development of future persuasive technologies in artificially intelligent agents. △ Less

Submitted 27 January, 2016; originally announced January 2016.

Comments: This is a book draft

The Single-Uniprior Index-Coding Problem: The Single-Sender Case and The Multi-Sender Extension

Authors: Lawrence Ong, Chin Keong Ho, Fabian Lim

Abstract: Index coding studies multiterminal source-coding problems where a set of receivers are required to decode multiple (possibly different) messages from a common broadcast, and they each know some messages a priori. In this paper, at the receiver end, we consider a special setting where each receiver knows only one message a priori, and each message is known to only one receiver. At the broadcasting… ▽ More Index coding studies multiterminal source-coding problems where a set of receivers are required to decode multiple (possibly different) messages from a common broadcast, and they each know some messages a priori. In this paper, at the receiver end, we consider a special setting where each receiver knows only one message a priori, and each message is known to only one receiver. At the broadcasting end, we consider a generalized setting where there could be multiple senders, and each sender knows a subset of the messages. The senders collaborate to transmit an index code. This work looks at minimizing the number of total coded bits the senders are required to transmit. When there is only one sender, we propose a pruning algorithm to find a lower bound on the optimal (i.e., the shortest) index codelength, and show that it is achievable by linear index codes. When there are two or more senders, we propose an appending technique to be used in conjunction with the pruning technique to give a lower bound on the optimal index codelength; we also derive an upper bound based on cyclic codes. While the two bounds do not match in general, for the special case where no two distinct senders know any message in common, the bounds match, giving the optimal index codelength. The results are expressed in terms of strongly connected components in directed graphs that represent the index-coding problems. △ Less

Submitted 31 May, 2016; v1 submitted 3 December, 2014; originally announced December 2014.

Comments: Author final manuscript

Journal ref: IEEE Transactions on Information Theory, Vol. 62, No. 6, pp. 3165-3182, June 2016

The Multi-Sender Multicast Index Coding

Authors: Lawrence Ong, Fabian Lim, Chin Keong Ho

Abstract: We focus on the following instance of an index coding problem, where a set of receivers are required to decode multiple messages, whilst each knows one of the messages a priori. In particular, here we consider a generalized setting where they are multiple senders, each sender only knows a subset of messages, and all senders are required to collectively transmit the index code. For a single sender,… ▽ More We focus on the following instance of an index coding problem, where a set of receivers are required to decode multiple messages, whilst each knows one of the messages a priori. In particular, here we consider a generalized setting where they are multiple senders, each sender only knows a subset of messages, and all senders are required to collectively transmit the index code. For a single sender, Ong and Ho (ICC, 2012) have established the optimal index codelength, where the lower bound was obtained using a pruning algorithm. In this paper, the pruning algorithm is simplified, and used in conjunction with an appending technique to give a lower bound to the multi-sender case. An upper bound is derived based on network coding. While the two bounds do not match in general, for the special case where no two senders know any message bit in common, the bounds match, giving the optimal index codelength. The results are derived based on graph theory, and are expressed in terms of strongly connected components. △ Less

Submitted 26 June, 2013; v1 submitted 13 May, 2013; originally announced May 2013.

Comments: This is an extended version of the same-titled paper accepted and to be presented at the IEEE International Symposium on Information Theory (ISIT), Istanbul, in July 2013

Journal ref: Proceedings of the 2013 IEEE International Symposium on Information Theory Proceedings (ISIT), Istanbul. Turkey, 7-12 July 2013, pp. 1147-1151

On U-Statistics and Compressed Sensing II: Non-Asymptotic Worst-Case Analysis

Authors: Fabian Lim, Vladimir Stojanovic

Abstract: In another related work, U-statistics were used for non-asymptotic "average-case" analysis of random compressed sensing matrices. In this companion paper the same analytical tool is adopted differently - here we perform non-asymptotic "worst-case" analysis. Simple union bounds are a natural choice for "worst-case" analyses, however their tightness is an issue (and questioned in previous works).… ▽ More In another related work, U-statistics were used for non-asymptotic "average-case" analysis of random compressed sensing matrices. In this companion paper the same analytical tool is adopted differently - here we perform non-asymptotic "worst-case" analysis. Simple union bounds are a natural choice for "worst-case" analyses, however their tightness is an issue (and questioned in previous works). Here we focus on a theoretical U-statistical result, which potentially allows us to prove that these union bounds are tight. To our knowledge, this kind of (powerful) result is completely new in the context of CS. This general result applies to a wide variety of parameters, and is related to (Stein-Chen) Poisson approximation. In this paper, we consider i) restricted isometries, and ii) mutual coherence. For the bounded case, we show that k-th order restricted isometry constants have tight union bounds, when the measurements m = \mathcal{O}(k (1 + \log(n/k))). Here we require the restricted isometries to grow linearly in k, however we conjecture that this result can be improved to allow them to be fixed. Also, we show that mutual coherence (with the standard estimate \sqrt{(4\log n)/m}) have very tight union bounds. For coherence, the normalization complicates general discussion, and we consider only Gaussian and Bernoulli cases here. △ Less

Submitted 30 October, 2012; originally announced October 2012.

Comments: 12 pages. Submitted to IEEE Transactions on Signal Processing

On U-Statistics and Compressed Sensing I: Non-Asymptotic Average-Case Analysis

Authors: Fabian Lim, Vladimir Marko Stojanovic

Abstract: Hoeffding's U-statistics model combinatorial-type matrix parameters (appearing in CS theory) in a natural way. This paper proposes using these statistics for analyzing random compressed sensing matrices, in the non-asymptotic regime (relevant to practice). The aim is to address certain pessimisms of "worst-case" restricted isometry analyses, as observed by both Blanchard & Dossal, et. al. We sho… ▽ More Hoeffding's U-statistics model combinatorial-type matrix parameters (appearing in CS theory) in a natural way. This paper proposes using these statistics for analyzing random compressed sensing matrices, in the non-asymptotic regime (relevant to practice). The aim is to address certain pessimisms of "worst-case" restricted isometry analyses, as observed by both Blanchard & Dossal, et. al. We show how U-statistics can obtain "average-case" analyses, by relating to statistical restricted isometry property (StRIP) type recovery guarantees. However unlike standard StRIP, random signal models are not required; the analysis here holds in the almost sure (probabilistic) sense. For Gaussian/bounded entry matrices, we show that both l1-minimization and LASSO essentially require on the order of k \cdot [\log((n-k)/u) + \sqrt{2(k/n) \log(n/k)}] measurements to respectively recover at least 1-5u fraction, and 1-4u fraction, of the signals. Noisy conditions are considered. Empirical evidence suggests our analysis to compare well to Donoho & Tanner's recent large deviation bounds for l0/l1-equivalence, in the regime of block lengths 1000-3000 with high undersampling (50-150 measurements); similar system sizes are found in recent CS implementation. In this work, it is assumed throughout that matrix columns are independently sampled. △ Less

Submitted 30 October, 2012; originally announced October 2012.

Comments: 12 pages. 3 pages supplementary material. Submitted to IEEE Trans. Signal Processing

Two Embedding Theorems for Data with Equivalences under Finite Group Action

Authors: Fabian Lim

Abstract: There is recent interest in compressing data sets for non-sequential settings, where lack of obvious orderings on their data space, require notions of data equivalences to be considered. For example, Varshney & Goyal (DCC, 2006) considered multiset equivalences, while Choi & Szpankowski (IEEE Trans. IT, 2012) considered isomorphic equivalences in graphs. Here equivalences are considered under a re… ▽ More There is recent interest in compressing data sets for non-sequential settings, where lack of obvious orderings on their data space, require notions of data equivalences to be considered. For example, Varshney & Goyal (DCC, 2006) considered multiset equivalences, while Choi & Szpankowski (IEEE Trans. IT, 2012) considered isomorphic equivalences in graphs. Here equivalences are considered under a relatively broad framework - finite-dimensional, non-sequential data spaces with equivalences under group action, for which analogues of two well-studied embedding theorems are derived: the Whitney embedding theorem and the Johnson-Lindenstrauss lemma. Only the canonical data points need to be carefully embedded, each such point representing a set of data points equivalent under group action. Two-step embeddings are considered. First, a group invariant is applied to account for equivalences, and then secondly, a linear embedding takes it down to low-dimensions. Our results require hypotheses on discriminability of the applied invariant, such notions related to seperating invariants (Dufresne, 2008), and completeness in pattern recognition (Kakarala, 1992). In the latter theorem, the embedding complexity depends on the size of the canonical part, which may be significantly smaller than the whole data set, up to a factor equal to the size the group. △ Less

Submitted 15 October, 2012; v1 submitted 30 July, 2012; originally announced July 2012.

Comments: 10 page extended abstract plus two sets of supplementary material. 1 figure. Preliminary report

Linear Programming Upper Bounds on Permutation Code Sizes From Coherent Configurations Related to the Kendall Tau Distance Metric

Authors: Fabian Lim, Manabu Hagiwara

Abstract: Recent interest on permutation rank modulation shows the Kendall tau metric as an important distance metric. This note documents our first efforts to obtain upper bounds on optimal code sizes (for said metric) ala Delsarte's approach. For the Hamming metric, Delsarte's seminal work on powerful linear programming (LP) bounds have been extended to permutation codes, via association scheme theory. Fo… ▽ More Recent interest on permutation rank modulation shows the Kendall tau metric as an important distance metric. This note documents our first efforts to obtain upper bounds on optimal code sizes (for said metric) ala Delsarte's approach. For the Hamming metric, Delsarte's seminal work on powerful linear programming (LP) bounds have been extended to permutation codes, via association scheme theory. For the Kendall tau metric, the same extension needs the more general theory of coherent configurations, whereby the optimal code size problem can be formulated as an extremely huge semidefinite programming (SDP) problem. Inspired by recent algebraic techniques for solving SDP's, we consider the dual problem, and propose an LP to search over a subset of dual feasible solutions. We obtain modest improvement over a recent Singleton bound due to Barg and Mazumdar. We regard this work as a starting point, towards fully exploiting the power of Delsarte's method, which are known to give some of the best bounds in the context of binary codes. △ Less

Submitted 5 June, 2012; v1 submitted 1 February, 2012; originally announced February 2012.

Comments: IEEE Intl. Symp. on Inform. Theory 2012. Final Version

Reliability Distributions of Truncated Max-log-map (MLM) Detectors Applied to ISI Channels

Authors: Fabian Lim, Aleksandar Kavcic

Abstract: The max-log-map (MLM) receiver is an approximated version of the well-known, Bahl-Cocke-Jelinek-Raviv (BCJR) algorithm. The MLM algorithm is attractive due to its implementation simplicity. In practice, sliding-window implementations are preferred; these practical implementations consider truncated signaling neighborhoods around each transmission time instant. In this paper, we consider sliding-wi… ▽ More The max-log-map (MLM) receiver is an approximated version of the well-known, Bahl-Cocke-Jelinek-Raviv (BCJR) algorithm. The MLM algorithm is attractive due to its implementation simplicity. In practice, sliding-window implementations are preferred; these practical implementations consider truncated signaling neighborhoods around each transmission time instant. In this paper, we consider sliding-window MLM receivers, where for any integer m, the MLM detector is truncated to a length- m signaling neighborhood. For any number n of chosen times instants, we derive exact expressions for both i) the joint distribution of the MLM symbol reliabilities, and ii) the joint probability of the erroneous MLM symbol detections. We show that the obtained expressions can be efficiently evaluated using Monte-Carlo techniques. Our proposed method is efficient; the most computationally expensive operation (in each Monte-Carlo trial) is an eigenvalue decomposition of a size 2mn by 2mn matrix. Practical truncation lengths can be easily handled. Finally, our proposed method is extremely general, and various scenarios such as correlated noise distributions, modulation coding, etc. may be easily accommodated. △ Less

Submitted 11 April, 2013; v1 submitted 2 July, 2010; originally announced July 2010.

Comments: 17 pages, 11 figures

Journal ref: IEEE Trans. on Inform. Theory, Volume 59, Issue 4, pp. 2411-2425, April 2013