Showing 1–16 of 16 results for author: Jian, G

Pruning Large Language Models with Semi-Structural Adaptive Sparse Training

Authors: Weiyu Huang, Yuezhou Hu, Guohao Jian, Jun Zhu, Jianfei Chen

Abstract: The remarkable success of Large Language Models (LLMs) relies heavily on their substantial scale, which poses significant challenges during model deployment in terms of latency and memory consumption. Recently, numerous studies have attempted to compress LLMs using one-shot pruning methods. However, these methods often suffer from considerable performance degradation on complex language understand… ▽ More The remarkable success of Large Language Models (LLMs) relies heavily on their substantial scale, which poses significant challenges during model deployment in terms of latency and memory consumption. Recently, numerous studies have attempted to compress LLMs using one-shot pruning methods. However, these methods often suffer from considerable performance degradation on complex language understanding tasks, raising concerns about the feasibility of pruning in LLMs. To address this issue, we propose Adaptive Sparse Trainer (AST), a novel and efficient retraining framework tailored for semi-structured sparse models. AST enables models to learn optimal masks during the weight update process without incurring additional computational overhead. Furthermore, we demonstrate that incorporating knowledge distillation significantly improves retraining efficiency and enhances model performance under fixed computational constraints. Additionally, a supplementary set of well-initialized parameters is integrated to further augment the model's efficacy. AST achieves state-of-the-art performance with minimal training cost. When applied to the LLaMA2-7B model, AST reduces the perplexity and zero-shot accuracy gap between dense and 2:4 semi-structured sparse models to 0.6 and 1.16%, respectively, utilizing less than 0.4% of the pretraining tokens and GPU hours. Our work demonstrates the feasibility of deploying semi-structured sparse LLMs and offers a promising alternative for achieving highly compressed models when combined with existing quantization techniques. △ Less

Submitted 18 December, 2024; v1 submitted 30 July, 2024; originally announced July 2024.

Comments: Accepted at AAAI25

On generalized quasi-cyclic codes over $\mathbb{Z}_4$

Authors: Jian Gao, Xiangrui Meng, Fang-Wei Fu

Abstract: Based on good algebraic structures and practicabilities, generalized quasi-cyclic (GQC) codes play important role in coding theory. In this paper, we study some results on GQC codes over $\mathbb{Z}_4$ including the normalized generating set, the minimum generating set and the normalized generating set of their dual codes. As an application, new $\mathbb{Z}_4$-linear codes and good nonlinear binar… ▽ More Based on good algebraic structures and practicabilities, generalized quasi-cyclic (GQC) codes play important role in coding theory. In this paper, we study some results on GQC codes over $\mathbb{Z}_4$ including the normalized generating set, the minimum generating set and the normalized generating set of their dual codes. As an application, new $\mathbb{Z}_4$-linear codes and good nonlinear binary codes are constructed from GQC codes over $\mathbb{Z}_4$. △ Less

Submitted 22 March, 2022; originally announced March 2022.

Two classes of linear codes and their generalized Hamming weights

Authors: Gaopeng Jian

Abstract: The generalized Hamming weights (GHWs) are fundamental parameters of linear codes. In this paper, we investigate the generalized Hamming weights of two classes of linear codes constructed from defining sets and determine them completely employing a number-theoretic approach. The generalized Hamming weights (GHWs) are fundamental parameters of linear codes. In this paper, we investigate the generalized Hamming weights of two classes of linear codes constructed from defining sets and determine them completely employing a number-theoretic approach. △ Less

Submitted 7 May, 2019; originally announced May 2019.

A class of linear codes with few weights

Authors: Gaopeng Jian

Abstract: Linear codes have been an interesting topic in both theory and practice for many years. In this paper, a class of $q$-ary linear codes with few weights are presented and their weight distributions are determined using Gauss periods. Some of the linear codes obtained are optimal or almost optimal with respect to the Griesmer bound. As s applications, these linear codes can be used to construct secr… ▽ More Linear codes have been an interesting topic in both theory and practice for many years. In this paper, a class of $q$-ary linear codes with few weights are presented and their weight distributions are determined using Gauss periods. Some of the linear codes obtained are optimal or almost optimal with respect to the Griesmer bound. As s applications, these linear codes can be used to construct secret sharing schemes with nice access structures. △ Less

Submitted 23 November, 2018; originally announced November 2018.

Generalized Hamming weights of three classes of linear codes

Authors: Gaopeng Jian

Abstract: The generalized Hamming weights of a linear code have been extensively studied since Wei first use them to characterize the cryptography performance of a linear code over the wire-tap channel of type II. In this paper, we investigate the generalized Hamming weights of three classes of linear codes constructed through defining sets and determine them partly for some cases. Particularly, in the semi… ▽ More The generalized Hamming weights of a linear code have been extensively studied since Wei first use them to characterize the cryptography performance of a linear code over the wire-tap channel of type II. In this paper, we investigate the generalized Hamming weights of three classes of linear codes constructed through defining sets and determine them partly for some cases. Particularly, in the semiprimitive case we solve an problem left in Yang et al. (IEEE Trans. Inf. Theory 61(9): 4905--4913, 2015). △ Less

Submitted 12 February, 2017; v1 submitted 30 December, 2016; originally announced December 2016.

Two-weight and three-weight linear codes based on Weil sums

Authors: Gaopeng Jian

Abstract: Linear codes with few weights have applications in secret sharing, authentication codes, association schemes and strongly regular graphs. In this paper, several classes of two-weight and three-weight linear codes are presented and their weight distributions are determined using Weil sums. Some of the linear codes obtained are optimal or almost optimal with respect to the Griesmer bound. Linear codes with few weights have applications in secret sharing, authentication codes, association schemes and strongly regular graphs. In this paper, several classes of two-weight and three-weight linear codes are presented and their weight distributions are determined using Weil sums. Some of the linear codes obtained are optimal or almost optimal with respect to the Griesmer bound. △ Less

Submitted 23 November, 2018; v1 submitted 21 December, 2016; originally announced December 2016.

Cryptanalysis and Improvement of an Improved Two Factor Authentication Scheme for Telecare Medicine Information Systems

Authors: Gaopeng Jian, Rongquan Feng

Abstract: Telecare medical information systems (TMIS) aim to provide healthcare services remotely. Efficient and secure mechanism for authentication and key agreement is required in order to guarantee the security and privacy of patients in TMIS. Telecare medical information systems (TMIS) aim to provide healthcare services remotely. Efficient and secure mechanism for authentication and key agreement is required in order to guarantee the security and privacy of patients in TMIS. △ Less

Submitted 30 December, 2016; v1 submitted 5 July, 2016; originally announced July 2016.

Comments: After checking the manuscript again a crucial error was founded in the section " Security weaknesses of Amin et al.'s scheme". In fact Amin et al.'s scheme resists offline password guessing attacks but the manuscript claims that it doesn't

On a class of left metacyclic codes

Authors: Cao Yonglin, Cao Yuan, Fu Fang-Wei, Gao Jian

Abstract: Let $G_{(m,3,r)}=\langle x,y\mid x^m=1, y^3=1,yx=x^ry\rangle$ be a metacyclic group of order $3m$, where ${\rm gcd}(m,r)=1$, $1<r<m$ and $r^3\equiv 1$ (mod $m$). Then left ideals of the group algebra $\mathbb{F}_q[G_{(m,3,r)}]$ are called left metacyclic codes over $\mathbb{F}_q$ of length $3m$, and abbreviated as left $G_{(m,3,r)}$-codes. A system theory for left $G_{(m,3,r)}$-codes is developed… ▽ More Let $G_{(m,3,r)}=\langle x,y\mid x^m=1, y^3=1,yx=x^ry\rangle$ be a metacyclic group of order $3m$, where ${\rm gcd}(m,r)=1$, $1<r<m$ and $r^3\equiv 1$ (mod $m$). Then left ideals of the group algebra $\mathbb{F}_q[G_{(m,3,r)}]$ are called left metacyclic codes over $\mathbb{F}_q$ of length $3m$, and abbreviated as left $G_{(m,3,r)}$-codes. A system theory for left $G_{(m,3,r)}$-codes is developed for the case of ${\rm gcd}(m,q)=1$ and $r\equiv q^ε$ for some positive integer $ε$, only using finite field theory and basic theory of cyclic codes and skew cyclic codes. The fact that any left $G_{(m,3,r)}$-code is a direct sum of concatenated codes with inner codes ${\cal A}_i$ and outer codes $C_i$ is proved, where ${\cal A}_i$ is a minimal cyclic code over $\mathbb{F}_q$ of length $m$ and $C_i$ is a skew cyclic code of length $3$ over an extension field of $\mathbb{F}_q$. Then an explicit expression for each outer code in any concatenated code is provided. Moreover, the dual code of each left $G_{(m,3,r)}$-code is given and self-orthogonal left $G_{(m,3,r)}$-codes are determined. △ Less

Submitted 15 June, 2016; originally announced June 2016.

On cyclic codes over $\mathbb{Z}_q+u\mathbb{Z}_q$

Authors: Jian Gao, Fang-Wei Fu, Ling Xiao, Rama Krishna Bandi

Abstract: Let $R=\mathbb{Z}_q+u\mathbb{Z}_q$, where $q=p^s$ and $u^2=0$. In this paper, some structural properties of cyclic codes over the ring $R$ are considered. A necessary and sufficient condition for cyclic codes over the ring $R$ to be free is obtained and a BCH-type bound on the minimum Hamming distance for them is also given. Let $R=\mathbb{Z}_q+u\mathbb{Z}_q$, where $q=p^s$ and $u^2=0$. In this paper, some structural properties of cyclic codes over the ring $R$ are considered. A necessary and sufficient condition for cyclic codes over the ring $R$ to be free is obtained and a BCH-type bound on the minimum Hamming distance for them is also given. △ Less

Submitted 26 April, 2015; v1 submitted 16 January, 2015; originally announced January 2015.

Comments: 11. arXiv admin note: text overlap with arXiv:1309.1623

MSC Class: 94B05; 11T71

On double cyclic codes over Z_4

Authors: Jian Gao, Minjia Shi, Tingting Wu, Fang-Wei Fu

Abstract: Let $R=\mathbb{Z}_4$ be the integer ring mod $4$. A double cyclic code of length $(r,s)$ over $R$ is a set that can be partitioned into two parts that any cyclic shift of the coordinates of both parts leaves invariant the code. These codes can be viewed as $R[x]$-submodules of $R[x]/(x^r-1)\times R[x]/(x^s-1)$. In this paper, we determine the generator polynomials of this family of codes as… ▽ More Let $R=\mathbb{Z}_4$ be the integer ring mod $4$. A double cyclic code of length $(r,s)$ over $R$ is a set that can be partitioned into two parts that any cyclic shift of the coordinates of both parts leaves invariant the code. These codes can be viewed as $R[x]$-submodules of $R[x]/(x^r-1)\times R[x]/(x^s-1)$. In this paper, we determine the generator polynomials of this family of codes as $R[x]$-submodules of $R[x]/(x^r-1)\times R[x]/(x^s-1)$. Further, we also give the minimal generating sets of this family of codes as $R$-submodules of $R[x]/(x^r-1)\times R[x]/(x^s-1)$. Some optimal or suboptimal nonlinear binary codes are obtained from this family of codes. Finally, we determine the relationship of generators between the double cyclic code and its dual. △ Less

Submitted 6 January, 2015; originally announced January 2015.

Comments: 16

MSC Class: 11T71; 94B05

Self-dual codes and quadratic residue codes over the ring $\mathbb{Z}_9+u\mathbb{Z}_9$

Authors: Jian Gao, XianFang Wang, Fang-Wei Fu

Abstract: In this paper, we introduce a new definitions of the Gray weight and the Gray map for linear codes over $\mathbb{Z}_9+u\mathbb{Z}_9$ with $u^2=u$. Some results on self-dual codes over this ring are investigated. Further, the structural properties of quadratic residue codes are also considered. Two self-dual codes with parameters $[22,11,5]$ and $[24,12,9]$ over $\mathbb{Z}_9$ are obtained. In this paper, we introduce a new definitions of the Gray weight and the Gray map for linear codes over $\mathbb{Z}_9+u\mathbb{Z}_9$ with $u^2=u$. Some results on self-dual codes over this ring are investigated. Further, the structural properties of quadratic residue codes are also considered. Two self-dual codes with parameters $[22,11,5]$ and $[24,12,9]$ over $\mathbb{Z}_9$ are obtained. △ Less

Submitted 31 December, 2014; v1 submitted 13 May, 2014; originally announced May 2014.

Comments: 12 pages. arXiv admin note: substantial text overlap with arXiv:1402.6771

MSC Class: 94B05; 11T71

On Linear Codes over $\mathbb{Z}_4+v\mathbb{Z}_4$

Authors: Jian Gao, Yun Gao, Fang-Wei Fu

Abstract: Linear codes are considered over the ring $\mathbb{Z}_4+v\mathbb{Z}_4$, where $v^2=v$. Gray weight, Gray maps for linear codes are defined and MacWilliams identity for the Gray weight enumerator is given. Self-dual codes, construction of Euclidean isodual codes, unimodular complex lattices, MDS codes and MGDS codes over $\mathbb{Z}_4+v\mathbb{Z}_4$ are studied. Cyclic codes and quadratic residue c… ▽ More Linear codes are considered over the ring $\mathbb{Z}_4+v\mathbb{Z}_4$, where $v^2=v$. Gray weight, Gray maps for linear codes are defined and MacWilliams identity for the Gray weight enumerator is given. Self-dual codes, construction of Euclidean isodual codes, unimodular complex lattices, MDS codes and MGDS codes over $\mathbb{Z}_4+v\mathbb{Z}_4$ are studied. Cyclic codes and quadratic residue codes are also considered. Finally, some examples for illustrating the main work are given. △ Less

Submitted 31 December, 2014; v1 submitted 26 February, 2014; originally announced February 2014.

Comments: 25 pages

MSC Class: 94B05; 94B15

Quasi-Cyclic Codes Over Finite Chain Rings

Authors: Jian Gao, Linzhi Shen, Fang-Wei Fu

Abstract: In this paper, we mainly consider quasi-cyclic (QC) codes over finite chain rings. We study module structures and trace representations of QC codes, which lead to some lower bounds on the minimum Hamming distance of QC codes. Moreover, we investigate the structural properties of 1-generator QC codes. Under some conditions, we discuss the enumeration of 1-generator QC codes and describe how to obta… ▽ More In this paper, we mainly consider quasi-cyclic (QC) codes over finite chain rings. We study module structures and trace representations of QC codes, which lead to some lower bounds on the minimum Hamming distance of QC codes. Moreover, we investigate the structural properties of 1-generator QC codes. Under some conditions, we discuss the enumeration of 1-generator QC codes and describe how to obtain the one and only one generator for each 1-generator QC code. △ Less

Submitted 6 September, 2013; originally announced September 2013.

Comments: 26

MSC Class: 11T71; 94B05; 94B15

Skew Generalized Quasi-Cyclic Codes over Finite Fields

Authors: Jian Gao, Linzhi Shen, Fang-Wei Fu

Abstract: In this work, we study a class of generalized quasi-cyclic (GQC) codes called skew GQC codes. By the factorization theory of ideals, we give the Chinese Remainder Theorem over the skew polynomial ring, which leads to a canonical decomposition of skew GQC codes. We also focus on some characteristics of skew GQC codes in details. For a 1-generator skew GQC code, we define the parity-check polynomial… ▽ More In this work, we study a class of generalized quasi-cyclic (GQC) codes called skew GQC codes. By the factorization theory of ideals, we give the Chinese Remainder Theorem over the skew polynomial ring, which leads to a canonical decomposition of skew GQC codes. We also focus on some characteristics of skew GQC codes in details. For a 1-generator skew GQC code, we define the parity-check polynomial, determine the dimension and give a lower bound on the minimum Hamming distance. The skew quasi-cyclic (QC) codes are also discussed briefly. △ Less

Submitted 6 September, 2013; originally announced September 2013.

Comments: 19

MSC Class: 94B05; 94B15

Generalized Quasi-Cyclic Codes Over $\mathbb{F}_q+u\mathbb{F}_q$

Authors: Jian Gao, Fang-Wei Fu, Linzhi Shen

Abstract: Generalized quasi-cyclic (GQC) codes with arbitrary lengths over the ring $\mathbb{F}_{q}+u\mathbb{F}_{q}$, where $u^2=0$, $q=p^n$, $n$ a positive integer and $p$ a prime number, are investigated. By the Chinese Remainder Theorem, structural properties and the decomposition of GQC codes are given. For 1-generator GQC codes, minimal generating sets and lower bounds on the minimum distance are given… ▽ More Generalized quasi-cyclic (GQC) codes with arbitrary lengths over the ring $\mathbb{F}_{q}+u\mathbb{F}_{q}$, where $u^2=0$, $q=p^n$, $n$ a positive integer and $p$ a prime number, are investigated. By the Chinese Remainder Theorem, structural properties and the decomposition of GQC codes are given. For 1-generator GQC codes, minimal generating sets and lower bounds on the minimum distance are given. As a special class of GQC codes, quasi-cyclic (QC) codes over $\mathbb{F}_q+u\mathbb{F}_q$ are also discussed briefly in this paper. △ Less

Submitted 6 July, 2013; originally announced July 2013.

Comments: 17 pages

MSC Class: 11T71; 94B05

One generator $(1+u)$-quasi twisted codes over $F_2+uF_2$

Authors: Jian Gao, Qiong Kong

Abstract: This paper gives the minimum generating sets of three types of one generator $(1+u)$-quasi twisted (QT) codes over $F_2+uF_2$, $u^2=0$. Moreover, it discusses the generating sets and the lower bounds on the minimum Lee distance of a special class of $A_2$ type one generator $(1+u)$-QT codes. Some good (optimal or suboptimal) linear codes over $F_2$ are obtained by these types of one generator… ▽ More This paper gives the minimum generating sets of three types of one generator $(1+u)$-quasi twisted (QT) codes over $F_2+uF_2$, $u^2=0$. Moreover, it discusses the generating sets and the lower bounds on the minimum Lee distance of a special class of $A_2$ type one generator $(1+u)$-QT codes. Some good (optimal or suboptimal) linear codes over $F_2$ are obtained by these types of one generator $(1+u)$-QT codes. △ Less

Submitted 30 June, 2013; originally announced July 2013.

Comments: 7 pages. In Mathematical Computation, 2013(1)

MSC Class: 94B05