Showing 1–36 of 36 results for author: Hassani, S

ARCH-COMP25 Category Report: Stochastic Models

Authors: Alessandro Abate, Omid Akbarzadeh, Henk A. P. Blom, Sofie Haesaert, Sina Hassani, Abolfazl Lavaei, Frederik Baymler Mathiesen, Rahul Misra, Amy Nejati, Mathis Niehage, Fie Ørum, Anne Remke, Behrad Samari, Ruohan Wang, Rafal Wisniewski, Ben Wooding, Mahdieh Zaker

Abstract: This report is concerned with a friendly competition for formal verification and policy synthesis of stochastic models. The main goal of the report is to introduce new benchmarks and their properties within this category and recommend next steps toward next year's edition of the competition. In particular, this report introduces three recently developed software tools, a new water distribution net… ▽ More This report is concerned with a friendly competition for formal verification and policy synthesis of stochastic models. The main goal of the report is to introduce new benchmarks and their properties within this category and recommend next steps toward next year's edition of the competition. In particular, this report introduces three recently developed software tools, a new water distribution network benchmark, and a collection of simplified benchmarks intended to facilitate further comparisons among tools that were previously not directly comparable. This friendly competition took place as part of the workshop Applied Verification for Continuous and Hybrid Systems (ARCH) in Summer 2025. △ Less

Submitted 21 June, 2025; originally announced June 2025.

Approaches to Responsible Governance of GenAI in Organizations

Authors: Dhari Gandhi, Himanshu Joshi, Lucas Hartman, Shabnam Hassani

Abstract: The rapid evolution of Generative AI (GenAI) has introduced unprecedented opportunities while presenting complex challenges around ethics, accountability, and societal impact. This paper draws on a literature review, established governance frameworks, and industry roundtable discussions to identify core principles for integrating responsible GenAI governance into diverse organizational structures.… ▽ More The rapid evolution of Generative AI (GenAI) has introduced unprecedented opportunities while presenting complex challenges around ethics, accountability, and societal impact. This paper draws on a literature review, established governance frameworks, and industry roundtable discussions to identify core principles for integrating responsible GenAI governance into diverse organizational structures. Our objective is to provide actionable recommendations for a balanced, risk-based governance approach that enables both innovation and oversight. Findings emphasize the need for adaptable risk assessment tools, continuous monitoring practices, and cross-sector collaboration to establish trustworthy GenAI. These insights provide a structured foundation and Responsible GenAI Guide (ResAI) for organizations to align GenAI initiatives with ethical, legal, and operational best practices. △ Less

Submitted 23 April, 2025; originally announced April 2025.

An Empirical Study on LLM-based Classification of Requirements-related Provisions in Food-safety Regulations

Authors: Shabnam Hassani, Mehrdad Sabetzadeh, Daniel Amyot

Abstract: As Industry 4.0 transforms the food industry, the role of software in achieving compliance with food-safety regulations is becoming increasingly critical. Food-safety regulations, like those in many legal domains, have largely been articulated in a technology-independent manner to ensure their longevity and broad applicability. However, this approach leaves a gap between the regulations and the mo… ▽ More As Industry 4.0 transforms the food industry, the role of software in achieving compliance with food-safety regulations is becoming increasingly critical. Food-safety regulations, like those in many legal domains, have largely been articulated in a technology-independent manner to ensure their longevity and broad applicability. However, this approach leaves a gap between the regulations and the modern systems and software increasingly used to implement them. In this article, we pursue two main goals. First, we conduct a Grounded Theory study of food-safety regulations and develop a conceptual characterization of food-safety concepts that closely relate to systems and software requirements. Second, we examine the effectiveness of two families of large language models (LLMs) -- BERT and GPT -- in automatically classifying legal provisions based on requirements-related food-safety concepts. Our results show that: (a) when fine-tuned, the accuracy differences between the best-performing models in the BERT and GPT families are relatively small. Nevertheless, the most powerful model in our experiments, GPT-4o, still achieves the highest accuracy, with an average Precision of 89% and an average Recall of 87%; (b) few-shot learning with GPT-4o increases Recall to 97% but decreases Precision to 65%, suggesting a trade-off between fine-tuning and few-shot learning; (c) despite our training examples being drawn exclusively from Canadian regulations, LLM-based classification performs consistently well on test provisions from the US, indicating a degree of generalizability across regulatory jurisdictions; and (d) for our classification task, LLMs significantly outperform simpler baselines constructed using long short-term memory (LSTM) networks and automatic keyword extraction. △ Less

Submitted 24 January, 2025; originally announced January 2025.

Enhancing Legal Compliance and Regulation Analysis with Large Language Models

Authors: Shabnam Hassani

Abstract: This research explores the application of Large Language Models (LLMs) for automating the extraction of requirement-related legal content in the food safety domain and checking legal compliance of regulatory artifacts. With Industry 4.0 revolutionizing the food industry and with the General Data Protection Regulation (GDPR) reshaping privacy policies and data processing agreements, there is a grow… ▽ More This research explores the application of Large Language Models (LLMs) for automating the extraction of requirement-related legal content in the food safety domain and checking legal compliance of regulatory artifacts. With Industry 4.0 revolutionizing the food industry and with the General Data Protection Regulation (GDPR) reshaping privacy policies and data processing agreements, there is a growing gap between regulatory analysis and recent technological advancements. This study aims to bridge this gap by leveraging LLMs, namely BERT and GPT models, to accurately classify legal provisions and automate compliance checks. Our findings demonstrate promising results, indicating LLMs' significant potential to enhance legal compliance and regulatory analysis efficiency, notably by reducing manual workload and improving accuracy within reasonable time and financial constraints. △ Less

Submitted 26 April, 2024; originally announced April 2024.

Comments: to be published in 32nd IEEE International Requirements Engineering 2024 Conference (RE'24) - Doctoral Symposium. arXiv admin note: text overlap with arXiv:2404.14356

Rethinking Legal Compliance Automation: Opportunities with Large Language Models

Authors: Shabnam Hassani, Mehrdad Sabetzadeh, Daniel Amyot, Jain Liao

Abstract: As software-intensive systems face growing pressure to comply with laws and regulations, providing automated support for compliance analysis has become paramount. Despite advances in the Requirements Engineering (RE) community on legal compliance analysis, important obstacles remain in developing accurate and generalizable compliance automation solutions. This paper highlights some observed limita… ▽ More As software-intensive systems face growing pressure to comply with laws and regulations, providing automated support for compliance analysis has become paramount. Despite advances in the Requirements Engineering (RE) community on legal compliance analysis, important obstacles remain in developing accurate and generalizable compliance automation solutions. This paper highlights some observed limitations of current approaches and examines how adopting new automation strategies that leverage Large Language Models (LLMs) can help address these shortcomings and open up fresh opportunities. Specifically, we argue that the examination of (textual) legal artifacts should, first, employ a broader context than sentences, which have widely been used as the units of analysis in past research. Second, the mode of analysis with legal artifacts needs to shift from classification and information extraction to more end-to-end strategies that are not only accurate but also capable of providing explanation and justification. We present a compliance analysis approach designed to address these limitations. We further outline our evaluation plan for the approach and provide preliminary evaluation results based on data processing agreements (DPAs) that must comply with the General Data Protection Regulation (GDPR). Our initial findings suggest that our approach yields substantial accuracy improvements and, at the same time, provides justification for compliance decisions. △ Less

Submitted 22 April, 2024; originally announced April 2024.

Comments: Accepted for publication at the RE@Next! track of RE 2024

Improving Requirements Completeness: Automated Assistance through Large Language Models

Authors: Dipeeka Luitel, Shabnam Hassani, Mehrdad Sabetzadeh

Abstract: Natural language (NL) is arguably the most prevalent medium for expressing systems and software requirements. Detecting incompleteness in NL requirements is a major challenge. One approach to identify incompleteness is to compare requirements with external sources. Given the rise of large language models (LLMs), an interesting question arises: Are LLMs useful external sources of knowledge for dete… ▽ More Natural language (NL) is arguably the most prevalent medium for expressing systems and software requirements. Detecting incompleteness in NL requirements is a major challenge. One approach to identify incompleteness is to compare requirements with external sources. Given the rise of large language models (LLMs), an interesting question arises: Are LLMs useful external sources of knowledge for detecting potential incompleteness in NL requirements? This article explores this question by utilizing BERT. Specifically, we employ BERT's masked language model (MLM) to generate contextualized predictions for filling masked slots in requirements. To simulate incompleteness, we withhold content from the requirements and assess BERT's ability to predict terminology that is present in the withheld content but absent in the disclosed content. BERT can produce multiple predictions per mask. Our first contribution is determining the optimal number of predictions per mask, striking a balance between effectively identifying omissions in requirements and mitigating noise present in the predictions. Our second contribution involves designing a machine learning-based filter to post-process BERT's predictions and further reduce noise. We conduct an empirical evaluation using 40 requirements specifications from the PURE dataset. Our findings indicate that: (1) BERT's predictions effectively highlight terminology that is missing from requirements, (2) BERT outperforms simpler baselines in identifying relevant yet missing terminology, and (3) our filter significantly reduces noise in the predictions, enhancing BERT's effectiveness as a tool for completeness checking of requirements. △ Less

Submitted 14 February, 2024; v1 submitted 3 August, 2023; originally announced August 2023.

Comments: This article has been accepted at the Requirements Engineering Journal (REJ), REFSQ'23 Special Issue. arXiv admin note: text overlap with arXiv:2302.04792

Using Language Models for Enhancing the Completeness of Natural-language Requirements

Authors: Dipeeka Luitel, Shabnam Hassani, Mehrdad Sabetzadeh

Abstract: [Context and motivation] Incompleteness in natural-language requirements is a challenging problem. [Question/problem] A common technique for detecting incompleteness in requirements is checking the requirements against external sources. With the emergence of language models such as BERT, an interesting question is whether language models are useful external sources for finding potential incomplete… ▽ More [Context and motivation] Incompleteness in natural-language requirements is a challenging problem. [Question/problem] A common technique for detecting incompleteness in requirements is checking the requirements against external sources. With the emergence of language models such as BERT, an interesting question is whether language models are useful external sources for finding potential incompleteness in requirements. [Principal ideas/results] We mask words in requirements and have BERT's masked language model (MLM) generate contextualized predictions for filling the masked slots. We simulate incompleteness by withholding content from requirements and measure BERT's ability to predict terminology that is present in the withheld content but absent in the content disclosed to BERT. [Contribution] BERT can be configured to generate multiple predictions per mask. Our first contribution is to determine how many predictions per mask is an optimal trade-off between effectively discovering omissions in requirements and the level of noise in the predictions. Our second contribution is devising a machine learning-based filter that post-processes predictions made by BERT to further reduce noise. We empirically evaluate our solution over 40 requirements specifications drawn from the PURE dataset [1]. Our results indicate that: (1) predictions made by BERT are highly effective at pinpointing terminology that is missing from requirements, and (2) our filter can substantially reduce noise from the predictions, thus making BERT a more compelling aid for improving completeness in requirements. △ Less

Submitted 9 February, 2023; originally announced February 2023.

Comments: This paper has been accepted at the 29th International Working Conference on Requirement Engineering: Foundation for Software Quality (REFSQ 2023)

Practical Approach of Knowledge Management in Medical Science

Authors: Mahdi Bohlouli, Patrick Uhr, Fabian Merges, Sanaz Mohammad Hassani, Madjid Fathi

Abstract: Knowledge organization, infrastructure, and knowledge-based activities are all subjects that help in the creation of business strategies for the new enterprise. In this paper, the first basics of knowledge-based systems are studied. Practical issues and challenges of Knowledge Management (KM) implementations are then illustrated. Finally, a comparison of different knowledge-based projects is prese… ▽ More Knowledge organization, infrastructure, and knowledge-based activities are all subjects that help in the creation of business strategies for the new enterprise. In this paper, the first basics of knowledge-based systems are studied. Practical issues and challenges of Knowledge Management (KM) implementations are then illustrated. Finally, a comparison of different knowledge-based projects is presented along with abstracted information on their implementation, techniques, and results. Most of these projects are in the field of medical science. Based on our study and evaluation of different KM projects, we conclude that KM is being used in every science, industry, and business. But its importance in medical science and assisted living projects are highlighted nowadays with the most of research institutes. Most medical centers are interested in using knowledge-based services like portals and learning techniques of knowledge for their future innovations and supports. △ Less

Submitted 16 January, 2020; originally announced January 2020.

A New Coding Paradigm for the Primitive Relay Channel

Authors: Marco Mondelli, S. Hamed Hassani, Rüdiger Urbanke

Abstract: We consider the primitive relay channel, where the source sends a message to the relay and to the destination, and the relay helps the communication by transmitting an additional message to the destination via a separate channel. Two well-known coding techniques have been introduced for this setting: decode-and-forward and compress-and-forward. In decode-and-forward, the relay completely decodes t… ▽ More We consider the primitive relay channel, where the source sends a message to the relay and to the destination, and the relay helps the communication by transmitting an additional message to the destination via a separate channel. Two well-known coding techniques have been introduced for this setting: decode-and-forward and compress-and-forward. In decode-and-forward, the relay completely decodes the message and sends some information to the destination; in compress-and-forward, the relay does not decode, and it sends a compressed version of the received signal to the destination using Wyner-Ziv coding. In this paper, we present a novel coding paradigm that provides an improved achievable rate for the primitive relay channel. The idea is to combine compress-and-forward and decode-and-forward via a chaining construction. We transmit over pairs of blocks: in the first block, we use compress-and-forward; and in the second block, we use decode-and-forward. More specifically, in the first block, the relay does not decode, it compresses the received signal via Wyner-Ziv, and it sends only part of the compression to the destination. In the second block, the relay completely decodes the message, it sends some information to the destination, and it also sends the remaining part of the compression coming from the first block. By doing so, we are able to strictly outperform both compress-and-forward and decode-and-forward. Note that the proposed coding scheme can be implemented with polar codes. As such, it has the typical attractive properties of polar coding schemes, namely, quasi-linear encoding and decoding complexity, and error probability that decays at super-polynomial speed. As a running example, we take into account the special case of the erasure relay channel, and we provide a comparison between the rates achievable by our proposed scheme and the existing upper and lower bounds. △ Less

Submitted 23 October, 2019; v1 submitted 9 January, 2018; originally announced January 2018.

Comments: 10 pages, 4 figures, in Proc. of ISIT'18 (short version) and in Algorithms (full version)

Binary Linear Codes with Optimal Scaling: Polar Codes with Large Kernels

Authors: Arman Fazeli, S. Hamed Hassani, Marco Mondelli, Alexander Vardy

Abstract: We prove that, for the binary erasure channel (BEC), the polar-coding paradigm gives rise to codes that not only approach the Shannon limit but do so under the best possible scaling of their block length as a~function of the gap to capacity. This result exhibits the first known family of binary codes that attain both optimal scaling and quasi-linear complexity of encoding and decoding. Our proof i… ▽ More We prove that, for the binary erasure channel (BEC), the polar-coding paradigm gives rise to codes that not only approach the Shannon limit but do so under the best possible scaling of their block length as a~function of the gap to capacity. This result exhibits the first known family of binary codes that attain both optimal scaling and quasi-linear complexity of encoding and decoding. Our proof is based on the construction and analysis of binary polar codes with large kernels. When communicating reliably at rates within $\varepsilon > 0$ of capacity, the code length $n$ often scales as $O(1/\varepsilon^μ)$, where the constant $μ$ is called the scaling exponent. It is known that the optimal scaling exponent is $μ=2$, and it is achieved by random linear codes. The scaling exponent of conventional polar codes (based on the $2\times 2$ kernel) on the BEC is $μ=3.63$. This falls far short of the optimal scaling guaranteed by random codes. Our main contribution is a rigorous proof of the following result: for the BEC, there exist $\ell\times\ell$ binary kernels, such that polar codes constructed from these kernels achieve scaling exponent $μ(\ell)$ that tends to the optimal value of $2$ as $\ell$ grows. We furthermore characterize precisely how large $\ell$ needs to be as a function of the gap between $μ(\ell)$ and $2$. The resulting binary codes maintain the recursive structure of conventional polar codes, and thereby achieve construction complexity $O(n)$ and encoding/decoding complexity $O(n\log n)$. △ Less

Submitted 14 October, 2020; v1 submitted 3 November, 2017; originally announced November 2017.

Comments: 30 pages, 3 figures, presented in part at ITW'18, to appear in IEEE Transactions on Information Theory

Partitioned List Decoding of Polar Codes: Analysis and Improvement of Finite Length Performance

Authors: Seyyed Ali Hashemi, Marco Mondelli, S. Hamed Hassani, Rudiger Urbanke, Warren J. Gross

Abstract: Polar codes represent one of the major recent breakthroughs in coding theory and, because of their attractive features, they have been selected for the incoming 5G standard. As such, a lot of attention has been devoted to the development of decoding algorithms with good error performance and efficient hardware implementation. One of the leading candidates in this regard is represented by successiv… ▽ More Polar codes represent one of the major recent breakthroughs in coding theory and, because of their attractive features, they have been selected for the incoming 5G standard. As such, a lot of attention has been devoted to the development of decoding algorithms with good error performance and efficient hardware implementation. One of the leading candidates in this regard is represented by successive-cancellation list (SCL) decoding. However, its hardware implementation requires a large amount of memory. Recently, a partitioned SCL (PSCL) decoder has been proposed to significantly reduce the memory consumption. In this paper, we examine the paradigm of PSCL decoding from both theoretical and practical standpoints: (i) by changing the construction of the code, we are able to improve the performance at no additional computational, latency or memory cost, (ii) we present an optimal scheme to allocate cyclic redundancy checks (CRCs), and (iii) we provide an upper bound on the list size that allows MAP performance. △ Less

Submitted 29 August, 2017; v1 submitted 15 May, 2017; originally announced May 2017.

Comments: 2017 IEEE Global Communications Conference (GLOBECOM)

Uniform Deviation Bounds for Unbounded Loss Functions like k-Means

Authors: Olivier Bachem, Mario Lucic, S. Hamed Hassani, Andreas Krause

Abstract: Uniform deviation bounds limit the difference between a model's expected loss and its loss on an empirical sample uniformly for all models in a learning problem. As such, they are a critical component to empirical risk minimization. In this paper, we provide a novel framework to obtain uniform deviation bounds for loss functions which are *unbounded*. In our main application, this allows us to obt… ▽ More Uniform deviation bounds limit the difference between a model's expected loss and its loss on an empirical sample uniformly for all models in a learning problem. As such, they are a critical component to empirical risk minimization. In this paper, we provide a novel framework to obtain uniform deviation bounds for loss functions which are *unbounded*. In our main application, this allows us to obtain bounds for $k$-Means clustering under weak assumptions on the underlying distribution. If the fourth moment is bounded, we prove a rate of $\mathcal{O}\left(m^{-\frac12}\right)$ compared to the previously known $\mathcal{O}\left(m^{-\frac14}\right)$ rate. Furthermore, we show that the rate also depends on the kurtosis - the normalized fourth moment which measures the "tailedness" of a distribution. We further provide improved rates under progressively stronger assumptions, namely, bounded higher moments, subgaussianity and bounded support. △ Less

Submitted 27 February, 2017; originally announced February 2017.

Construction of Polar Codes with Sublinear Complexity

Authors: Marco Mondelli, S. Hamed Hassani, Rüdiger Urbanke

Abstract: Consider the problem of constructing a polar code of block length $N$ for the transmission over a given channel $W$. Typically this requires to compute the reliability of all the $N$ synthetic channels and then to include those that are sufficiently reliable. However, we know from [1], [2] that there is a partial order among the synthetic channels. Hence, it is natural to ask whether we can exploi… ▽ More Consider the problem of constructing a polar code of block length $N$ for the transmission over a given channel $W$. Typically this requires to compute the reliability of all the $N$ synthetic channels and then to include those that are sufficiently reliable. However, we know from [1], [2] that there is a partial order among the synthetic channels. Hence, it is natural to ask whether we can exploit it to reduce the computational burden of the construction problem. We show that, if we take advantage of the partial order [1], [2], we can construct a polar code by computing the reliability of roughly a fraction $1/\log^{3/2} N$ of the synthetic channels. In particular, we prove that $N/\log^{3/2} N$ is a lower bound on the number of synthetic channels to be considered and such a bound is tight up to a multiplicative factor $\log\log N$. This set of roughly $N/\log^{3/2} N$ synthetic channels is universal, in the sense that it allows one to construct polar codes for any $W$, and it can be identified by solving a maximum matching problem on a bipartite graph. Our proof technique consists of reducing the construction problem to the problem of computing the maximum cardinality of an antichain for a suitable partially ordered set. As such, this method is general and it can be used to further improve the complexity of the construction problem in case a new partial order on the synthetic channels of polar codes is discovered. △ Less

Submitted 13 July, 2017; v1 submitted 15 December, 2016; originally announced December 2016.

Comments: 9 pages, 3 figures, presented at ISIT'17 and submitted to IEEE Trans. Inform. Theory

Capacity-Achieving Rate-Compatible Polar Codes for General Channels

Authors: Marco Mondelli, S. Hamed Hassani, Ivana Marić, Dennis Hui, Song-Nam Hong

Abstract: We present a rate-compatible polar coding scheme that achieves the capacity of any family of channels. Our solution generalizes the previous results [1], [2] that provide capacity-achieving rate-compatible polar codes for a degraded family of channels. The motivation for our extension comes from the fact that in many practical scenarios, e.g., MIMO systems and non-Gaussian interference, the channe… ▽ More We present a rate-compatible polar coding scheme that achieves the capacity of any family of channels. Our solution generalizes the previous results [1], [2] that provide capacity-achieving rate-compatible polar codes for a degraded family of channels. The motivation for our extension comes from the fact that in many practical scenarios, e.g., MIMO systems and non-Gaussian interference, the channels cannot be ordered by degradation. The main technical contribution of this paper consists in removing the degradation condition. To do so, we exploit the ideas coming from the construction of universal polar codes. Our scheme possesses the usual attractive features of polar codes: low complexity code construction, encoding, and decoding; super-polynomial scaling of the error probability with the block length; and absence of error floors. On the negative side, the scaling of the gap to capacity with the block length is slower than in standard polar codes, and we prove an upper bound on the scaling exponent. △ Less

Submitted 20 January, 2017; v1 submitted 3 November, 2016; originally announced November 2016.

Comments: 7 pages, 2 figures, accepted at WCNC'17 workshop on polar coding

Near-optimal Bayesian Active Learning with Correlated and Noisy Tests

Authors: Yuxin Chen, S. Hamed Hassani, Andreas Krause

Abstract: We consider the Bayesian active learning and experimental design problem, where the goal is to learn the value of some unknown target variable through a sequence of informative, noisy tests. In contrast to prior work, we focus on the challenging, yet practically relevant setting where test outcomes can be conditionally dependent given the hidden target variable. Under such assumptions, common heur… ▽ More We consider the Bayesian active learning and experimental design problem, where the goal is to learn the value of some unknown target variable through a sequence of informative, noisy tests. In contrast to prior work, we focus on the challenging, yet practically relevant setting where test outcomes can be conditionally dependent given the hidden target variable. Under such assumptions, common heuristics, such as greedily performing tests that maximize the reduction in uncertainty of the target, often perform poorly. In this paper, we propose ECED, a novel, computationally efficient active learning algorithm, and prove strong theoretical guarantees that hold with correlated, noisy tests. Rather than directly optimizing the prediction error, at each step, ECED picks the test that maximizes the gain in a surrogate objective, which takes into account the dependencies between tests. Our analysis relies on an information-theoretic auxiliary function to track the progress of ECED, and utilizes adaptive submodularity to attain the near-optimal bound. We demonstrate strong empirical performance of ECED on two problem instances, including a Bayesian experimental design task intended to distinguish among economic theories of how people make risky decisions, and an active preference learning task via pairwise comparisons. △ Less

Submitted 11 July, 2016; v1 submitted 24 May, 2016; originally announced May 2016.

Unified Scaling of Polar Codes: Error Exponent, Scaling Exponent, Moderate Deviations, and Error Floors

Authors: Marco Mondelli, S. Hamed Hassani, Rüdiger Urbanke

Abstract: Consider the transmission of a polar code of block length $N$ and rate $R$ over a binary memoryless symmetric channel $W$ and let $P_e$ be the block error probability under successive cancellation decoding. In this paper, we develop new bounds that characterize the relationship of the parameters $R$, $N$, $P_e$, and the quality of the channel $W$ quantified by its capacity $I(W)$ and its Bhattacha… ▽ More Consider the transmission of a polar code of block length $N$ and rate $R$ over a binary memoryless symmetric channel $W$ and let $P_e$ be the block error probability under successive cancellation decoding. In this paper, we develop new bounds that characterize the relationship of the parameters $R$, $N$, $P_e$, and the quality of the channel $W$ quantified by its capacity $I(W)$ and its Bhattacharyya parameter $Z(W)$. In previous work, two main regimes were studied. In the error exponent regime, the channel $W$ and the rate $R<I(W)$ are fixed, and it was proved that the error probability $P_e$ scales roughly as $2^{-\sqrt{N}}$. In the scaling exponent approach, the channel $W$ and the error probability $P_e$ are fixed and it was proved that the gap to capacity $I(W)-R$ scales as $N^{-1/μ}$. Here, $μ$ is called scaling exponent and this scaling exponent depends on the channel $W$. A heuristic computation for the binary erasure channel (BEC) gives $μ=3.627$ and it was shown that, for any channel $W$, $3.579 \le μ\le 5.702$. Our contributions are as follows. First, we provide the tighter upper bound $μ\le 4.714$ valid for any $W$. With the same technique, we obtain $μ\le 3.639$ for the case of the BEC, which approaches very closely its heuristically derived value. Second, we develop a trade-off between the gap to capacity $I(W)-R$ and the error probability $P_e$ as functions of the block length $N$. In other words, we consider a moderate deviations regime in which we study how fast both quantities, as functions of the block length $N$, simultaneously go to $0$. Third, we prove that polar codes are not affected by error floors. To do so, we fix a polar code of block length $N$ and rate $R$. Then, we vary the channel $W$ and we show that the error probability $P_e$ scales as the Bhattacharyya parameter $Z(W)$ raised to a power that scales roughly like $\sqrt{N}$. △ Less

Submitted 4 August, 2016; v1 submitted 11 January, 2015; originally announced January 2015.

Comments: 23 pages, 4 figures, submitted to IEEE Trans. Inform. Theory and accepted in part at ISIT'15

Alignment of Polarized Sets

Authors: Joseph M. Renes, David Sutter, S. Hamed Hassani

Abstract: Arıkan's polar coding technique is based on the idea of synthesizing $n$ channels from the $n$ instances of the physical channel by a simple linear encoding transformation. Each synthesized channel corresponds to a particular input to the encoder. For large $n$, the synthesized channels become either essentially noiseless or almost perfectly noisy, but in total carry as much information as the ori… ▽ More Arıkan's polar coding technique is based on the idea of synthesizing $n$ channels from the $n$ instances of the physical channel by a simple linear encoding transformation. Each synthesized channel corresponds to a particular input to the encoder. For large $n$, the synthesized channels become either essentially noiseless or almost perfectly noisy, but in total carry as much information as the original $n$ channels. Capacity can therefore be achieved by transmitting messages over the essentially noiseless synthesized channels. Unfortunately, the set of inputs corresponding to reliable synthesized channels is poorly understood, in particular how the set depends on the underlying physical channel. In this work, we present two analytic conditions sufficient to determine if the reliable inputs corresponding to different discrete memoryless channels are aligned or not, i.e. if one set is contained in the other. Understanding the alignment of the polarized sets is important as it is directly related to universality properties of the induced polar codes, which are essential in particular for network coding problems. We demonstrate the performance of our conditions on a few examples for wiretap and broadcast channels. Finally we show that these conditions imply that the simple quantum polar coding scheme of Renes et al. [Phys. Rev. Lett. 109, 050504 (2012)] requires entanglement assistance for general channels, but also show such assistance to be unnecessary in many cases of interest. △ Less

Submitted 28 November, 2014; originally announced November 2014.

Comments: 22 pages, 6 figures

MSC Class: 68P30; 94A15; 81P45

Journal ref: IEEE Journal on Selected Areas in Communications, vol. 34, no. 2, pages 224-238, 2016

How to Achieve the Capacity of Asymmetric Channels

Authors: Marco Mondelli, S. Hamed Hassani, Rüdiger Urbanke

Abstract: We survey coding techniques that enable reliable transmission at rates that approach the capacity of an arbitrary discrete memoryless channel. In particular, we take the point of view of modern coding theory and discuss how recent advances in coding for symmetric channels help provide more efficient solutions for the asymmetric case. We consider, in more detail, three basic coding paradigms. The… ▽ More We survey coding techniques that enable reliable transmission at rates that approach the capacity of an arbitrary discrete memoryless channel. In particular, we take the point of view of modern coding theory and discuss how recent advances in coding for symmetric channels help provide more efficient solutions for the asymmetric case. We consider, in more detail, three basic coding paradigms. The first one is Gallager's scheme that consists of concatenating a linear code with a non-linear mapping so that the input distribution can be appropriately shaped. We explicitly show that both polar codes and spatially coupled codes can be employed in this scenario. Furthermore, we derive a scaling law between the gap to capacity, the cardinality of the input and output alphabets, and the required size of the mapper. The second one is an integrated scheme in which the code is used both for source coding, in order to create codewords distributed according to the capacity-achieving input distribution, and for channel coding, in order to provide error protection. Such a technique has been recently introduced by Honda and Yamamoto in the context of polar codes, and we show how to apply it also to the design of sparse graph codes. The third paradigm is based on an idea of Böcherer and Mathar, and separates the two tasks of source coding and channel coding by a chaining construction that binds together several codewords. We present conditions for the source code and the channel code, and we describe how to combine any source code with any channel code that fulfill those conditions, in order to provide capacity-achieving schemes for asymmetric channels. In particular, we show that polar codes, spatially coupled codes, and homophonic codes are suitable as basic building blocks of the proposed coding strategy. △ Less

Submitted 3 January, 2018; v1 submitted 28 June, 2014; originally announced June 2014.

Comments: 32 pages, 4 figures, presented in part at Allerton'14 and published in IEEE Trans. Inform. Theory

Achieving Marton's Region for Broadcast Channels Using Polar Codes

Authors: Marco Mondelli, S. Hamed Hassani, Igal Sason, Rüdiger Urbanke

Abstract: This paper presents polar coding schemes for the 2-user discrete memoryless broadcast channel (DM-BC) which achieve Marton's region with both common and private messages. This is the best achievable rate region known to date, and it is tight for all classes of 2-user DM-BCs whose capacity regions are known. To accomplish this task, we first construct polar codes for both the superposition as well… ▽ More This paper presents polar coding schemes for the 2-user discrete memoryless broadcast channel (DM-BC) which achieve Marton's region with both common and private messages. This is the best achievable rate region known to date, and it is tight for all classes of 2-user DM-BCs whose capacity regions are known. To accomplish this task, we first construct polar codes for both the superposition as well as the binning strategy. By combining these two schemes, we obtain Marton's region with private messages only. Finally, we show how to handle the case of common information. The proposed coding schemes possess the usual advantages of polar codes, i.e., they have low encoding and decoding complexity and a super-polynomial decay rate of the error probability. We follow the lead of Goela, Abbe, and Gastpar, who recently introduced polar codes emulating the superposition and binning schemes. In order to align the polar indices, for both schemes, their solution involves some degradedness constraints that are assumed to hold between the auxiliary random variables and the channel outputs. To remove these constraints, we consider the transmission of $k$ blocks and employ a chaining construction that guarantees the proper alignment of the polarized indices. The techniques described in this work are quite general, and they can be adopted to many other multi-terminal scenarios whenever there polar indices need to be aligned. △ Less

Submitted 10 October, 2014; v1 submitted 23 January, 2014; originally announced January 2014.

Comments: 26 pages, 11 figures, accepted to IEEE Trans. Inform. Theory and presented in part at ISIT'14

From Polar to Reed-Muller Codes: a Technique to Improve the Finite-Length Performance

Authors: Marco Mondelli, S. Hamed Hassani, Rüdiger Urbanke

Abstract: We explore the relationship between polar and RM codes and we describe a coding scheme which improves upon the performance of the standard polar code at practical block lengths. Our starting point is the experimental observation that RM codes have a smaller error probability than polar codes under MAP decoding. This motivates us to introduce a family of codes that "interpolates" between RM and pol… ▽ More We explore the relationship between polar and RM codes and we describe a coding scheme which improves upon the performance of the standard polar code at practical block lengths. Our starting point is the experimental observation that RM codes have a smaller error probability than polar codes under MAP decoding. This motivates us to introduce a family of codes that "interpolates" between RM and polar codes, call this family ${\mathcal C}_{\rm inter} = \{C_α : α\in [0, 1]\}$, where $C_α \big |_{α= 1}$ is the original polar code, and $C_α \big |_{α= 0}$ is an RM code. Based on numerical observations, we remark that the error probability under MAP decoding is an increasing function of $α$. MAP decoding has in general exponential complexity, but empirically the performance of polar codes at finite block lengths is boosted by moving along the family ${\mathcal C}_{\rm inter}$ even under low-complexity decoding schemes such as, for instance, belief propagation or successive cancellation list decoder. We demonstrate the performance gain via numerical simulations for transmission over the erasure channel as well as the Gaussian channel. △ Less

Submitted 3 September, 2014; v1 submitted 14 January, 2014; originally announced January 2014.

Comments: 8 pages, 7 figures, in IEEE Transactions on Communications, 2014 and in ISIT'14

Universal Polar Codes

Authors: S. Hamed Hassani, Rudiger Urbanke

Abstract: Polar codes, invented by Arikan in 2009, are known to achieve the capacity of any binary-input memoryless output-symmetric channel. One of the few drawbacks of the original polar code construction is that it is not universal. This means that the code has to be tailored to the channel if we want to transmit close to capacity. We present two "polar-like" schemes which are capable of achieving the… ▽ More Polar codes, invented by Arikan in 2009, are known to achieve the capacity of any binary-input memoryless output-symmetric channel. One of the few drawbacks of the original polar code construction is that it is not universal. This means that the code has to be tailored to the channel if we want to transmit close to capacity. We present two "polar-like" schemes which are capable of achieving the compound capacity of the whole class of binary-input memoryless output-symmetric channels with low complexity. Roughly speaking, for the first scheme we stack up $N$ polar blocks of length $N$ on top of each other but shift them with respect to each other so that they form a "staircase." Coding then across the columns of this staircase with a standard Reed-Solomon code, we can achieve the compound capacity using a standard successive decoder to process the rows (the polar codes) and in addition a standard Reed-Solomon erasure decoder to process the columns. Compared to standard polar codes this scheme has essentially the same complexity per bit but a block length which is larger by a factor $O(N \log_2(N)/ε)$, where $ε$ is the gap to capacity. For the second scheme we first show how to construct a true polar code which achieves the compound capacity for a finite number of channels. We achieve this by introducing special "polarization" steps which "align" the good indices for the various channels. We then show how to exploit the compactness of the space of binary-input memoryless output-symmetric channels to reduce the compound capacity problem for this class to a compound capacity problem for a finite set of channels. This scheme is similar in spirit to standard polar codes, but the price for universality is a considerably larger blocklength. We close with what we consider to be some interesting open problems. △ Less

Submitted 13 December, 2013; v1 submitted 27 July, 2013; originally announced July 2013.

Comments: Submitted to IEEE transactions on Information Theory

Scaling Exponent of List Decoders with Applications to Polar Codes

Authors: Marco Mondelli, S. Hamed Hassani, Rüdiger Urbanke

Abstract: Motivated by the significant performance gains which polar codes experience under successive cancellation list decoding, their scaling exponent is studied as a function of the list size. In particular, the error probability is fixed and the trade-off between block length and back-off from capacity is analyzed. A lower bound is provided on the error probability under $\rm MAP$ decoding with list si… ▽ More Motivated by the significant performance gains which polar codes experience under successive cancellation list decoding, their scaling exponent is studied as a function of the list size. In particular, the error probability is fixed and the trade-off between block length and back-off from capacity is analyzed. A lower bound is provided on the error probability under $\rm MAP$ decoding with list size $L$ for any binary-input memoryless output-symmetric channel and for any class of linear codes such that their minimum distance is unbounded as the block length grows large. Then, it is shown that under $\rm MAP$ decoding, although the introduction of a list can significantly improve the involved constants, the scaling exponent itself, i.e., the speed at which capacity is approached, stays unaffected for any finite list size. In particular, this result applies to polar codes, since their minimum distance tends to infinity as the block length increases. A similar result is proved for genie-aided successive cancellation decoding when transmission takes place over the binary erasure channel, namely, the scaling exponent remains constant for any fixed number of helps from the genie. Note that since genie-aided successive cancellation decoding might be strictly worse than successive cancellation list decoding, the problem of establishing the scaling exponent of the latter remains open. △ Less

Submitted 22 September, 2014; v1 submitted 18 April, 2013; originally announced April 2013.

Comments: 14 pages, submitted to IEEE Trans. Inform. Theory and presented in part to ITW'13

The Least Degraded and the Least Upgraded Channel with respect to a Channel Family

Authors: Wei Liu, S. Hamed Hassani, Ruediger Urbanke

Abstract: Given a family of binary-input memoryless output-symmetric (BMS) channels having a fixed capacity, we derive the BMS channel having the highest (resp. lowest) capacity among all channels that are degraded (resp. upgraded) with respect to the whole family. We give an explicit characterization of this channel as well as an explicit formula for the capacity of this channel. Given a family of binary-input memoryless output-symmetric (BMS) channels having a fixed capacity, we derive the BMS channel having the highest (resp. lowest) capacity among all channels that are degraded (resp. upgraded) with respect to the whole family. We give an explicit characterization of this channel as well as an explicit formula for the capacity of this channel. △ Less

Submitted 18 April, 2013; originally announced April 2013.

Comments: Submitted to 2013 IEEE Information Theory Workshop

Finite-Length Scaling of Polar Codes

Authors: S. Hamed Hassani, Kasra Alishahi, Rudiger Urbanke

Abstract: Consider a binary-input memoryless output-symmetric channel $W$. Such a channel has a capacity, call it $I(W)$, and for any $R<I(W)$ and strictly positive constant $P_{\rm e}$ we know that we can construct a coding scheme that allows transmission at rate $R$ with an error probability not exceeding $P_{\rm e}$. Assume now that we let the rate $R$ tend to $I(W)$ and we ask how we have to "scale" the… ▽ More Consider a binary-input memoryless output-symmetric channel $W$. Such a channel has a capacity, call it $I(W)$, and for any $R<I(W)$ and strictly positive constant $P_{\rm e}$ we know that we can construct a coding scheme that allows transmission at rate $R$ with an error probability not exceeding $P_{\rm e}$. Assume now that we let the rate $R$ tend to $I(W)$ and we ask how we have to "scale" the blocklength $N$ in order to keep the error probability fixed to $P_{\rm e}$. We refer to this as the "finite-length scaling" behavior. This question was addressed by Strassen as well as Polyanskiy, Poor and Verdu, and the result is that $N$ must grow at least as the square of the reciprocal of $I(W)-R$. Polar codes are optimal in the sense that they achieve capacity. In this paper, we are asking to what degree they are also optimal in terms of their finite-length behavior. Our approach is based on analyzing the dynamics of the un-polarized channels. The main results of this paper can be summarized as follows. Consider the sum of Bhattacharyya parameters of sub-channels chosen (by the polar coding scheme) to transmit information. If we require this sum to be smaller than a given value $P_{\rm e}>0$, then the required block-length $N$ scales in terms of the rate $R < I(W)$ as $N \geq \fracα{(I(W)-R)^{\underlineμ}}$, where $α$ is a positive constant that depends on $P_{\rm e}$ and $I(W)$, and $\underlineμ = 3.579$. Also, we show that with the same requirement on the sum of Bhattacharyya parameters, the block-length scales in terms of the rate like $N \leq \fracβ{(I(W)-R)^{\overlineμ}}$, where $β$ is a constant that depends on $P_{\rm e}$ and $I(W)$, and $\overlineμ=6$. △ Less

Submitted 22 July, 2014; v1 submitted 17 April, 2013; originally announced April 2013.

Comments: In IEEE Transactions on Information Theory, 2014

The Space of Solutions of Coupled XORSAT Formulae

Authors: S. Hamed Hassani, Nicolas Macris, Rudiger Urbanke

Abstract: The XOR-satisfiability (XORSAT) problem deals with a system of $n$ Boolean variables and $m$ clauses. Each clause is a linear Boolean equation (XOR) of a subset of the variables. A $K$-clause is a clause involving $K$ distinct variables. In the random $K$-XORSAT problem a formula is created by choosing $m$ $K$-clauses uniformly at random from the set of all possible clauses on $n$ variables. The s… ▽ More The XOR-satisfiability (XORSAT) problem deals with a system of $n$ Boolean variables and $m$ clauses. Each clause is a linear Boolean equation (XOR) of a subset of the variables. A $K$-clause is a clause involving $K$ distinct variables. In the random $K$-XORSAT problem a formula is created by choosing $m$ $K$-clauses uniformly at random from the set of all possible clauses on $n$ variables. The set of solutions of a random formula exhibits various geometrical transitions as the ratio $\frac{m}{n}$ varies. We consider a {\em coupled} $K$-XORSAT ensemble, consisting of a chain of random XORSAT models that are spatially coupled across a finite window along the chain direction. We observe that the threshold saturation phenomenon takes place for this ensemble and we characterize various properties of the space of solutions of such coupled formulae. △ Less

Submitted 3 March, 2013; originally announced March 2013.

Comments: Submitted to ISIT 2013

Polar Codes: Robustness of the Successive Cancellation Decoder with Respect to Quantization

Authors: S. Hamed Hassani, Rudiger Urbanke

Abstract: Polar codes provably achieve the capacity of a wide array of channels under successive decoding. This assumes infinite precision arithmetic. Given the successive nature of the decoding algorithm, one might worry about the sensitivity of the performance to the precision of the computation. We show that even very coarsely quantized decoding algorithms lead to excellent performance. More concretely… ▽ More Polar codes provably achieve the capacity of a wide array of channels under successive decoding. This assumes infinite precision arithmetic. Given the successive nature of the decoding algorithm, one might worry about the sensitivity of the performance to the precision of the computation. We show that even very coarsely quantized decoding algorithms lead to excellent performance. More concretely, we show that under successive decoding with an alphabet of cardinality only three, the decoder still has a threshold and this threshold is a sizable fraction of capacity. More generally, we show that if we are willing to transmit at a rate $δ$ below capacity, then we need only $c \log(1/δ)$ bits of precision, where $c$ is a universal constant. △ Less

Submitted 20 September, 2012; originally announced September 2012.

Comments: In ISIT 2012

On the Construction of Polar Codes

Authors: Ramtin Pedarsani, S. Hamed Hassani, Ido Tal, Emre Telatar

Abstract: We consider the problem of efficiently constructing polar codes over binary memoryless symmetric (BMS) channels. The complexity of designing polar codes via an exact evaluation of the polarized channels to find which ones are "good" appears to be exponential in the block length. In \cite{TV11}, Tal and Vardy show that if instead the evaluation if performed approximately, the construction has only… ▽ More We consider the problem of efficiently constructing polar codes over binary memoryless symmetric (BMS) channels. The complexity of designing polar codes via an exact evaluation of the polarized channels to find which ones are "good" appears to be exponential in the block length. In \cite{TV11}, Tal and Vardy show that if instead the evaluation if performed approximately, the construction has only linear complexity. In this paper, we follow this approach and present a framework where the algorithms of \cite{TV11} and new related algorithms can be analyzed for complexity and accuracy. We provide numerical and analytical results on the efficiency of such algorithms, in particular we show that one can find all the "good" channels (except a vanishing fraction) with almost linear complexity in block-length (except a polylogarithmic factor). △ Less

Submitted 20 September, 2012; originally announced September 2012.

Comments: In ISIT 2011

Universal Bounds on the Scaling Behavior of Polar Codes

Authors: Ali Goli, S. Hamed Hassani, Rudiger Urbanke

Abstract: We consider the problem of determining the trade-off between the rate and the block-length of polar codes for a given block error probability when we use the successive cancellation decoder. We take the sum of the Bhattacharyya parameters as a proxy for the block error probability, and show that there exists a universal parameter $μ$ such that for any binary memoryless symmetric channel $W$ with c… ▽ More We consider the problem of determining the trade-off between the rate and the block-length of polar codes for a given block error probability when we use the successive cancellation decoder. We take the sum of the Bhattacharyya parameters as a proxy for the block error probability, and show that there exists a universal parameter $μ$ such that for any binary memoryless symmetric channel $W$ with capacity $I(W)$, reliable communication requires rates that satisfy $R< I(W)-αN^{-\frac{1}μ}$, where $α$ is a positive constant and $N$ is the block-length. We provide lower bounds on $μ$, namely $μ\geq 3.553$, and we conjecture that indeed $μ=3.627$, the parameter for the binary erasure channel. △ Less

Submitted 13 May, 2012; originally announced May 2012.

Comments: To be presented at ISIT 2012

Threshold Saturation in Spatially Coupled Constraint Satisfaction Problems

Authors: S. Hamed Hassani, Nicolas Macris, Rudiger Urbanke

Abstract: We consider chains of random constraint satisfaction models that are spatially coupled across a finite window along the chain direction. We investigate their phase diagram at zero temperature using the survey propagation formalism and the interpolation method. We prove that the SAT-UNSAT phase transition threshold of an infinite chain is identical to the one of the individual standard model, and i… ▽ More We consider chains of random constraint satisfaction models that are spatially coupled across a finite window along the chain direction. We investigate their phase diagram at zero temperature using the survey propagation formalism and the interpolation method. We prove that the SAT-UNSAT phase transition threshold of an infinite chain is identical to the one of the individual standard model, and is therefore not affected by spatial coupling. We compute the survey propagation complexity using population dynamics as well as large degree approximations, and determine the survey propagation threshold. We find that a clustering phase survives coupling. However, as one increases the range of the coupling window, the survey propagation threshold increases and saturates towards the phase transition threshold. We also briefly discuss other aspects of the problem. Namely, the condensation threshold is not affected by coupling, but the dynamic threshold displays saturation towards the condensation one. All these features may provide a new avenue for obtaining better provable algorithmic lower bounds on phase transition thresholds of the individual standard model. △ Less

Submitted 12 June, 2012; v1 submitted 23 December, 2011; originally announced December 2011.

Rate-Dependent Analysis of the Asymptotic Behavior of Channel Polarization

Authors: S. Hamed Hassani, Ryuhei Mori, Toshiyuki Tanaka, Rudiger Urbanke

Abstract: For a binary-input memoryless symmetric channel $W$, we consider the asymptotic behavior of the polarization process in the large block-length regime when transmission takes place over $W$. In particular, we study the asymptotics of the cumulative distribution $\mathbb{P}(Z_n \leq z)$, where $\{Z_n\}$ is the Bhattacharyya process defined from $W$, and its dependence on the rate of transmission. On… ▽ More For a binary-input memoryless symmetric channel $W$, we consider the asymptotic behavior of the polarization process in the large block-length regime when transmission takes place over $W$. In particular, we study the asymptotics of the cumulative distribution $\mathbb{P}(Z_n \leq z)$, where $\{Z_n\}$ is the Bhattacharyya process defined from $W$, and its dependence on the rate of transmission. On the basis of this result, we characterize the asymptotic behavior, as well as its dependence on the rate, of the block error probability of polar codes using the successive cancellation decoder. This refines the original bounds by Arıkan and Telatar. Our results apply to general polar codes based on $\ell \times \ell$ kernel matrices. We also provide lower bounds on the block error probability of polar codes using the MAP decoder. The MAP lower bound and the successive cancellation upper bound coincide when $\ell=2$, but there is a gap for $\ell>2$. △ Less

Submitted 4 October, 2011; v1 submitted 2 October, 2011; originally announced October 2011.

Comments: Submitted to IEEE Transactions on Information Theory

Chains of Mean Field Models

Authors: S. Hamed Hassani, Nicolas Macris, Ruediger Urbanke

Abstract: We consider a collection of Curie-Weiss (CW) spin systems, possibly with a random field, each of which is placed along the positions of a one-dimensional chain. The CW systems are coupled together by a Kac-type interaction in the longitudinal direction of the chain and by an infinite range interaction in the direction transverse to the chain. Our motivations for studying this model come from recen… ▽ More We consider a collection of Curie-Weiss (CW) spin systems, possibly with a random field, each of which is placed along the positions of a one-dimensional chain. The CW systems are coupled together by a Kac-type interaction in the longitudinal direction of the chain and by an infinite range interaction in the direction transverse to the chain. Our motivations for studying this model come from recent findings in the theory of error correcting codes based on spatially coupled graphs. We find that, although much simpler than the codes, the model studied here already displays similar behaviors. We are interested in the van der Waals curve in a regime where the size of each Curie-Weiss model tends to infinity, and the length of the chain and range of the Kac interaction are large but finite. Below the critical temperature, and with appropriate boundary conditions, there appears a series of equilibrium states representing kink-like interfaces between the two equilibrium states of the individual system. The van der Waals curve oscillates periodically around the Maxwell plateau. These oscillations have a period inversely proportional to the chain length and an amplitude exponentially small in the range of the interaction; in other words the spinodal points of the chain model lie exponentially close to the phase transition threshold. The amplitude of the oscillations is closely related to a Peierls-Nabarro free energy barrier for the motion of the kink along the chain. Analogies to similar phenomena and their possible algorithmic significance for graphical models of interest in coding theory and theoretical computer science are pointed out. △ Less

Submitted 20 December, 2011; v1 submitted 4 May, 2011; originally announced May 2011.

Coupled Graphical Models and Their Thresholds

Authors: S. Hamed Hassani, Nicolas Macris, Ruediger Urbanke

Abstract: The excellent performance of convolutional low-density parity-check codes is the result of the spatial coupling of individual underlying codes across a window of growing size, but much smaller than the length of the individual codes. Remarkably, the belief-propagation threshold of the coupled ensemble is boosted to the maximum-a-posteriori one of the individual system. We investigate the generalit… ▽ More The excellent performance of convolutional low-density parity-check codes is the result of the spatial coupling of individual underlying codes across a window of growing size, but much smaller than the length of the individual codes. Remarkably, the belief-propagation threshold of the coupled ensemble is boosted to the maximum-a-posteriori one of the individual system. We investigate the generality of this phenomenon beyond coding theory: we couple general graphical models into a one-dimensional chain of large individual systems. For the later we take the Curie-Weiss, random field Curie-Weiss, $K$-satisfiability, and $Q$-coloring models. We always find, based on analytical as well as numerical calculations, that the message passing thresholds of the coupled systems come very close to the static ones of the individual models. The remarkable properties of convolutional low-density parity-check codes are a manifestation of this very general phenomenon. △ Less

Submitted 4 May, 2011; originally announced May 2011.

Comments: In proceedings of ITW 2010

Near concavity of the growth rate for coupled LDPC chains

Authors: S. Hamed Hassani, Nicolas Macris, Ryuhei Mori

Abstract: Convolutional Low-Density-Parity-Check (LDPC) ensembles have excellent performance. Their iterative threshold increases with their average degree, or with the size of the coupling window in randomized constructions. In the later case, as the window size grows, the Belief Propagation (BP) threshold attains the maximum-a-posteriori (MAP) threshold of the underlying ensemble. In this contribution we… ▽ More Convolutional Low-Density-Parity-Check (LDPC) ensembles have excellent performance. Their iterative threshold increases with their average degree, or with the size of the coupling window in randomized constructions. In the later case, as the window size grows, the Belief Propagation (BP) threshold attains the maximum-a-posteriori (MAP) threshold of the underlying ensemble. In this contribution we show that a similar phenomenon happens for the growth rate of coupled ensembles. Loosely speaking, we observe that as the coupling strength grows, the growth rate of the coupled ensemble comes close to the concave hull of the underlying ensemble's growth rate. For ensembles randomly coupled across a window the growth rate actually tends to the concave hull of the underlying one as the window size increases. Our observations are supported by the calculations of the combinatorial growth rate, and that of the growth rate derived from the replica method. The observed concavity is a general feature of coupled mean field graphical models and is already present at the level of coupled Curie-Weiss models. There, the canonical free energy of the coupled system tends to the concave hull of the underlying one. As we explain, the behavior of the growth rate of coupled ensembles is exactly analogous. △ Less

Submitted 4 April, 2011; originally announced April 2011.

Comments: 5 pages, submitted to ISIT 2011

On the scaling of Polar Codes: II. The behavior of un-polarized channels

Authors: S. Hamed Hassani, Kasra Alishahi, Rudiger Urbanke

Abstract: We provide upper and lower bounds on the escape rate of the Bhattacharyya process corresponding to polar codes and transmission over the the binary erasure channel. More precisely, we bound the exponent of the number of sub-channels whose Bhattacharyya constant falls in a fixed interval $[a,b]$. Mathematically this can be stated as bounding the limit… ▽ More We provide upper and lower bounds on the escape rate of the Bhattacharyya process corresponding to polar codes and transmission over the the binary erasure channel. More precisely, we bound the exponent of the number of sub-channels whose Bhattacharyya constant falls in a fixed interval $[a,b]$. Mathematically this can be stated as bounding the limit $\lim_{n \to \infty} \frac{1}{n} \ln \mathbb{P}(Z_n \in [a,b])$, where $Z_n$ is the Bhattacharyya process. The quantity $\mathbb{P}(Z_n \in [a,b])$ represents the fraction of sub-channels that are still un-polarized at time $n$. △ Less

Submitted 18 February, 2010; v1 submitted 16 February, 2010; originally announced February 2010.

Comments: Submitted to ISIT 2010

On the scaling of Polar codes: I. The behavior of polarized channels

Authors: S. Hamed Hassani, Rudiger Urbanke

Abstract: We consider the asymptotic behavior of the polarization process for polar codes when the blocklength tends to infinity. In particular, we study the problem of asymptotic analysis of the cumulative distribution $\mathbb{P}(Z_n \leq z)$, where $Z_n=Z(W_n)$ is the Bhattacharyya process, and its dependence to the rate of transmission R. We show that for a BMS channel $W$, for $R < I(W)$ we have… ▽ More We consider the asymptotic behavior of the polarization process for polar codes when the blocklength tends to infinity. In particular, we study the problem of asymptotic analysis of the cumulative distribution $\mathbb{P}(Z_n \leq z)$, where $Z_n=Z(W_n)$ is the Bhattacharyya process, and its dependence to the rate of transmission R. We show that for a BMS channel $W$, for $R < I(W)$ we have $\lim_{n \to \infty} \mathbb{P} (Z_n \leq 2^{-2^{\frac{n}{2}+\sqrt{n} \frac{Q^{-1}(\frac{R}{I(W)})}{2} +o(\sqrt{n})}}) = R$ and for $R<1- I(W)$ we have $\lim_{n \to \infty} \mathbb{P} (Z_n \geq 1-2^{-2^{\frac{n}{2}+ \sqrt{n} \frac{Q^{-1}(\frac{R}{1-I(W)})}{2} +o(\sqrt{n})}}) = R$, where $Q(x)$ is the probability that a standard normal random variable will obtain a value larger than $x$. As a result, if we denote by $\mathbb{P}_e ^{\text{SC}}(n,R)$ the probability of error using polar codes of block-length $N=2^n$ and rate $R<I(W)$ under successive cancellation decoding, then $\log(-\log(\mathbb{P}_e ^{\text{SC}}(n,R)))$ scales as $\frac{n}{2}+\sqrt{n}\frac{Q^{-1}(\frac{R}{I(W)})}{2}+ o(\sqrt{n})$. We also prove that the same result holds for the block error probability using the MAP decoder, i.e., for $\log(-\log(\mathbb{P}_e ^{\text{MAP}}(n,R)))$. △ Less

Submitted 28 January, 2010; v1 submitted 15 January, 2010; originally announced January 2010.

Comments: Submitted to ISIT 2010

The Compound Capacity of Polar Codes

Authors: S. Hamed Hassani, Satish Babu Korada, Ruediger Urbanke

Abstract: We consider the compound capacity of polar codes under successive cancellation decoding for a collection of binary-input memoryless output-symmetric channels. By deriving a sequence of upper and lower bounds, we show that in general the compound capacity under successive decoding is strictly smaller than the unrestricted compound capacity. We consider the compound capacity of polar codes under successive cancellation decoding for a collection of binary-input memoryless output-symmetric channels. By deriving a sequence of upper and lower bounds, we show that in general the compound capacity under successive decoding is strictly smaller than the unrestricted compound capacity. △ Less

Submitted 19 July, 2009; originally announced July 2009.

Comments: 5 pages