Showing 1–50 of 65 results for author: Coolen, A C C

Dynamical Properties of Dense Associative Memory

Authors: Kazushi Mimura, Jun'ichi Takeuchi, Yuto Sumikawa, Yoshiyuki Kabashima, Anthony C. C. Coolen

Abstract: The dense associative memory is one of the basic modern Hopfield networks and can store large numbers of memory patterns. While the stationary state storage capacity has been investigated so far, its dynamical properties have not been discussed. In this paper, we analyze the dynamics by means of an exact approach based on generating functional analysis. It allows us to investigate convergence prop… ▽ More The dense associative memory is one of the basic modern Hopfield networks and can store large numbers of memory patterns. While the stationary state storage capacity has been investigated so far, its dynamical properties have not been discussed. In this paper, we analyze the dynamics by means of an exact approach based on generating functional analysis. It allows us to investigate convergence properties as well as the size of the attraction basins. We also analyze the stationary state of the updating rule. △ Less

Submitted 1 June, 2025; originally announced June 2025.

Comments: 10 pages, 1 figure

Roles of repertoire diversity in robustness of humoral immune response

Authors: Alexander Mozeika, Franca Fraternali, Deborah Dunn-Walters, Anthony C. C. Coolen

Abstract: The adaptive immune system relies on diversity of its repertoire of receptors to protect the organism from a great variety of pathogens. Since the initial repertoire is the result of random gene rearrangement, binding of receptors is not limited to pathogen-associated antigens but also includes self antigens. There is a fine balance between having a diverse repertoire, protecting from many differe… ▽ More The adaptive immune system relies on diversity of its repertoire of receptors to protect the organism from a great variety of pathogens. Since the initial repertoire is the result of random gene rearrangement, binding of receptors is not limited to pathogen-associated antigens but also includes self antigens. There is a fine balance between having a diverse repertoire, protecting from many different pathogens, and yet reducing its self-reactivity as far as possible to avoid damage to self. In the ageing immune system this balance is altered, manifesting in reduced specificity of response to pathogens or vaccination on a background of higher self-reactivity. To answer the question whether age-related changes of repertoire in the diversity and self/non-self affinity balance of antibodies could explain the reduced efficacy of the humoral response in older people, we construct a minimal mathematical model of the humoral immune response. The principle of least damage allows us, for a given repertoire of antibodies, to resolve a tension between the necessity to neutralise target antigens as quickly as possible and the requirement to limit the damage to self antigens leading to an optimal dynamics of immune response. The model predicts slowing down of immune response for repertoires with reduced diversity and increased self-reactivity. △ Less

Submitted 29 October, 2019; originally announced October 2019.

Analysis of overfitting in the regularized Cox model

Authors: M Sheikh, A. C. C. Coolen

Abstract: The Cox proportional hazards model is ubiquitous in the analysis of time-to-event data. However, when the data dimension p is comparable to the sample size $N$, maximum likelihood estimates for its regression parameters are known to be biased or break down entirely due to overfitting. This prompted the introduction of the so-called regularized Cox model. In this paper we use the replica method fro… ▽ More The Cox proportional hazards model is ubiquitous in the analysis of time-to-event data. However, when the data dimension p is comparable to the sample size $N$, maximum likelihood estimates for its regression parameters are known to be biased or break down entirely due to overfitting. This prompted the introduction of the so-called regularized Cox model. In this paper we use the replica method from statistical physics to investigate the relationship between the true and inferred regression parameters in regularized multivariate Cox regression with L2 regularization, in the regime where both p and N are large but with p/N ~ O(1). We thereby generalize a recent study from maximum likelihood to maximum a posteriori inference. We also establish a relationship between the optimal regularization parameter and p/N, allowing for straightforward overfitting corrections in time-to-event analysis. △ Less

Submitted 25 July, 2019; v1 submitted 14 April, 2019; originally announced April 2019.

Replica methods for loopy sparse random graphs

Authors: A C C Coolen

Abstract: I report on the development of a novel statistical mechanical formalism for the analysis of random graphs with many short loops, and processes on such graphs. The graphs are defined via maximum entropy ensembles, in which both the degrees (via hard constraints) and the adjacency matrix spectrum (via a soft constraint) are prescribed. The sum over graphs can be done analytically, using a replica fo… ▽ More I report on the development of a novel statistical mechanical formalism for the analysis of random graphs with many short loops, and processes on such graphs. The graphs are defined via maximum entropy ensembles, in which both the degrees (via hard constraints) and the adjacency matrix spectrum (via a soft constraint) are prescribed. The sum over graphs can be done analytically, using a replica formalism with complex replica dimensions. All known results for tree-like graphs are recovered in a suitable limit. For loopy graphs, the emerging theory has an appealing and intuitive structure, suggests how message passing algorithms should be adapted, and what is the structure of theories describing spin systems on loopy architectures. However, the formalism is still largely untested, and may require further adjustment and refinement. △ Less

Submitted 29 January, 2016; originally announced January 2016.

Comments: 11 pages, no figures. To be published in Proceedings of The International Meeting on High-Dimensional Data-Driven Science (HD3-2015), Kyoto, Japan, on 14-17 December, 2015

Protein interaction networks and biology: towards the connection

Authors: Alessia Annibale, Anthony C. C. Coolen, Nuria Planell-Morell

Abstract: Protein interaction networks (PIN) are popular means to visualize the proteome. However, PIN datasets are known to be noisy, incomplete and biased by the experimental protocols used to detect protein interactions. This paper aims at understanding the connection between true protein interactions and the protein interaction datasets that have been obtained using the most popular experimental techniq… ▽ More Protein interaction networks (PIN) are popular means to visualize the proteome. However, PIN datasets are known to be noisy, incomplete and biased by the experimental protocols used to detect protein interactions. This paper aims at understanding the connection between true protein interactions and the protein interaction datasets that have been obtained using the most popular experimental techniques, i.e. mass spectronomy (MS) and yeast two-hybrid (Y2H). We show that the most natural adjacency matrix of protein interaction networks has a separable form, and this induces precise relations between moments of the degree distribution and the number of short loops. These relations provide powerful tools to test the reliability of datasets and hint at the underlying biological mechanism with which proteins and complexes recruit each other. △ Less

Submitted 4 January, 2015; originally announced January 2015.

Comments: 38 pages, 10 figures

MSC Class: 92Bxx (Primary); 82Bxx (Secondary)

Entropy of random graph ensembles constrained with generalised degrees

Authors: Ekaterina S. Roberts, Anthonius C. C. Coolen

Abstract: Generalised degrees provide a natural bridge between local and global topological properties of networks. We define the generalised degree to be the number of neighbours of a node within one and two steps respectively. Tailored random graph ensembles are used to quantify and compare topological properties of networks in a systematic and precise manner, using concepts from information theory. We ca… ▽ More Generalised degrees provide a natural bridge between local and global topological properties of networks. We define the generalised degree to be the number of neighbours of a node within one and two steps respectively. Tailored random graph ensembles are used to quantify and compare topological properties of networks in a systematic and precise manner, using concepts from information theory. We calculate the Shannon entropy of random graph ensembles constrained with a specified generalised degree distribution. We find that the outcome has a natural connection with the degree-degree correlation which is implied by specifying a generalised degree distribution. We demonstrate how generalised degrees can be used to qualitatively and quantitatively describe a network. △ Less

Submitted 14 September, 2013; originally announced September 2013.

Comments: 24 pages, 9 figures

Immune networks: multi-tasking capabilities near saturation

Authors: Elena Agliari, Alessia Annibale, Adriano Barra, A. C. C. Coolen, Daniele Tantari

Abstract: Pattern-diluted associative networks were introduced recently as models for the immune system, with nodes representing T-lymphocytes and stored patterns representing signalling protocols between T- and B-lymphocytes. It was shown earlier that in the regime of extreme pattern dilution, a system with $N_T$ T-lymphocytes can manage a number $N_B!=!\order(N_T^δ)$ of B-lymphocytes simultaneously, with… ▽ More Pattern-diluted associative networks were introduced recently as models for the immune system, with nodes representing T-lymphocytes and stored patterns representing signalling protocols between T- and B-lymphocytes. It was shown earlier that in the regime of extreme pattern dilution, a system with $N_T$ T-lymphocytes can manage a number $N_B!=!\order(N_T^δ)$ of B-lymphocytes simultaneously, with $δ!<!1$. Here we study this model in the extensive load regime $N_B!=!αN_T$, with also a high degree of pattern dilution, in agreement with immunological findings. We use graph theory and statistical mechanical analysis based on replica methods to show that in the finite-connectivity regime, where each T-lymphocyte interacts with a finite number of B-lymphocytes as $N_T\to\infty$, the T-lymphocytes can coordinate effective immune responses to an extensive number of distinct antigen invasions in parallel. As $α$ increases, the system eventually undergoes a second order transition to a phase with clonal cross-talk interference, where the system's performance degrades gracefully. Mathematically, the model is equivalent to a spin system on a finitely connected graph with many short loops, so one would expect the available analytical methods, which all assume locally tree-like graphs, to fail. Yet it turns out to be solvable. Our results are supported by numerical simulations. △ Less

Submitted 25 May, 2013; originally announced May 2013.

Journal ref: Journal of Physics A 46, 415003 (2013). (IOP Select: Highlights-2013)

Retrieving Infinite Numbers of Patterns in a Spin-Glass Model of Immune Networks

Authors: Elena Agliari, Alessia Annibale, Adriano Barra, A. C. C. Coolen, Daniele Tantari

Abstract: The similarity between neural and immune networks has been known for decades, but so far we did not understand the mechanism that allows the immune system, unlike associative neural networks, to recall and execute a large number of memorized defense strategies {\em in parallel}. The explanation turns out to lie in the network topology. Neurons interact typically with a large number of other neuron… ▽ More The similarity between neural and immune networks has been known for decades, but so far we did not understand the mechanism that allows the immune system, unlike associative neural networks, to recall and execute a large number of memorized defense strategies {\em in parallel}. The explanation turns out to lie in the network topology. Neurons interact typically with a large number of other neurons, whereas interactions among lymphocytes in immune networks are very specific, and described by graphs with finite connectivity. In this paper we use replica techniques to solve a statistical mechanical immune network model with `coordinator branches' (T-cells) and `effector branches' (B-cells), and show how the finite connectivity enables the system to manage an extensive number of immune clones simultaneously, even above the percolation threshold. The system exhibits only weak ergodicity breaking, so that both multiple antigen defense and homeostasis can be accomplished. △ Less

Submitted 17 March, 2017; v1 submitted 9 May, 2013; originally announced May 2013.

Comments: Editor's Choice 2017

Journal ref: EPL 117, 28003 (2017)

Immune networks: multi-tasking capabilities at medium load

Authors: Elena Agliari, Alessia Annibale, Adriano Barra, A. C. C. Coolen, Daniele Tantari

Abstract: Associative network models featuring multi-tasking properties have been introduced recently and studied in the low load regime, where the number $P$ of simultaneously retrievable patterns scales with the number $N$ of nodes as $P\sim \log N$. In addition to their relevance in artificial intelligence, these models are increasingly important in immunology, where stored patterns represent strategies… ▽ More Associative network models featuring multi-tasking properties have been introduced recently and studied in the low load regime, where the number $P$ of simultaneously retrievable patterns scales with the number $N$ of nodes as $P\sim \log N$. In addition to their relevance in artificial intelligence, these models are increasingly important in immunology, where stored patterns represent strategies to fight pathogens and nodes represent lymphocyte clones. They allow us to understand the crucial ability of the immune system to respond simultaneously to multiple distinct antigen invasions. Here we develop further the statistical mechanical analysis of such systems, by studying the medium load regime, $P \sim N^δ$ with $δ\in (0,1]$. We derive three main results. First, we reveal the nontrivial architecture of these networks: they exhibit a high degree of modularity and clustering, which is linked to their retrieval abilities. Second, by solving the model we demonstrate for $δ<1$ the existence of large regions in the phase diagram where the network can retrieve all stored patterns simultaneously. Finally, in the high load regime $δ=1$ we find that the system behaves as a spin glass, suggesting that finite-connectivity frameworks are required to achieve effective retrieval. △ Less

Submitted 28 February, 2013; originally announced February 2013.

Comments: 26 pages, 10 figures

Controlled Markovian dynamics of graphs: unbiased generation of random graphs with prescribed topological properties

Authors: E. S. Roberts, A. Annibale, A. C. C. Coolen

Abstract: We analyze the properties of degree-preserving Markov chains based on elementary edge switchings in undirected and directed graphs. We give exact yet simple formulas for the mobility of a graph (the number of possible moves) in terms of its adjacency matrix. This formula allows us to define acceptance probabilities for edge switchings, such that the Markov chains become controlled Glauber-type det… ▽ More We analyze the properties of degree-preserving Markov chains based on elementary edge switchings in undirected and directed graphs. We give exact yet simple formulas for the mobility of a graph (the number of possible moves) in terms of its adjacency matrix. This formula allows us to define acceptance probabilities for edge switchings, such that the Markov chains become controlled Glauber-type detailed balance processes, designed to evolve to any required invariant measure (representing the asymptotic frequencies with which the allowed graphs are visited during the process). △ Less

Submitted 9 March, 2012; v1 submitted 5 March, 2012; originally announced March 2012.

Comments: 4 pages, 1 figure, contribution to the conference NOMA11 (Evora), to be published in Springer Proceedings in Mathematics and Statistics

MSC Class: 60J20; 90C35

What you see is not what you get: how sampling affects macroscopic features of biological networks

Authors: A. Annibale, A. C. C. Coolen

Abstract: We use mathematical methods from the theory of tailored random graphs to study systematically the effects of sampling on topological features of large biological signalling networks. Our aim in doing so is to increase our quantitative understanding of the relation between true biological networks and the imperfect and often biased samples of these networks that are reported in public data reposito… ▽ More We use mathematical methods from the theory of tailored random graphs to study systematically the effects of sampling on topological features of large biological signalling networks. Our aim in doing so is to increase our quantitative understanding of the relation between true biological networks and the imperfect and often biased samples of these networks that are reported in public data repositories and used by biomedical scientists. We derive exact explicit formulae for degree distributions and degree correlation kernels of sampled networks, in terms of the degree distributions and degree correlation kernels of the underlying true network, for a broad family of sampling protocols that include (un-)biased node and/or link undersampling as well as (un-)biased link oversampling. Our predictions are in excellent agreement with numerical simulations. △ Less

Submitted 1 June, 2011; originally announced June 2011.

Comments: 26 pages, 8 figures

MSC Class: 82B44; 05Cxx

Tailored graph ensembles as proxies or null models for real networks II: results on directed graphs

Authors: E. S. Roberts, A. C. C. Coolen, T. Schlitt

Abstract: We generate new mathematical tools with which to quantify the macroscopic topological structure of large directed networks. This is achieved via a statistical mechanical analysis of constrained maximum entropy ensembles of directed random graphs with prescribed joint distributions for in- and outdegrees and prescribed degree-degree correlation functions. We calculate exact and explicit formulae fo… ▽ More We generate new mathematical tools with which to quantify the macroscopic topological structure of large directed networks. This is achieved via a statistical mechanical analysis of constrained maximum entropy ensembles of directed random graphs with prescribed joint distributions for in- and outdegrees and prescribed degree-degree correlation functions. We calculate exact and explicit formulae for the leading orders in the system size of the Shannon entropies and complexities of these ensembles, and for information-theoretic distances. The results are applied to data on gene regulation networks. △ Less

Submitted 31 January, 2011; originally announced January 2011.

Comments: 21 pages, 1 figure, submitted to J. Phys. A

Network resilience against intelligent attacks constrained by degree dependent node removal cost

Authors: A Annibale, A C C Coolen, G Bianconi

Abstract: We study the resilience of complex networks against attacks in which nodes are targeted intelligently, but where disabling a node has a cost to the attacker which depends on its degree. Attackers have to meet these costs with limited resources, which constrains their actions. A network's integrity is quantified in terms of the efficacy of the process that it supports. We calculate how the optimal… ▽ More We study the resilience of complex networks against attacks in which nodes are targeted intelligently, but where disabling a node has a cost to the attacker which depends on its degree. Attackers have to meet these costs with limited resources, which constrains their actions. A network's integrity is quantified in terms of the efficacy of the process that it supports. We calculate how the optimal attack strategy and the most attack-resistant network degree statistics depend on the node removal cost function and the attack resources. The resilience of networks against intelligent attacks is found to depend strongly on the node removal cost function faced by the attacker. In particular, if node removal costs increase sufficiently fast with the node degree, power law networks are found to be more resilient than Poissonian ones, even against optimized intelligent attacks. △ Less

Submitted 24 May, 2010; originally announced May 2010.

Comments: 28 pages, 5 figures

Tailored graph ensembles as proxies or null models for real networks I: tools for quantifying structure

Authors: A. Annibale, A. C. C. Coolen, L. P. Fernandes, F. Fraternali, J. Kleinjung

Abstract: We study the tailoring of structured random graph ensembles to real networks, with the objective of generating precise and practical mathematical tools for quantifying and comparing network topologies macroscopically, beyond the level of degree statistics. Our family of ensembles can produce graphs with any prescribed degree distribution and any degree-degree correlation function, its control pa… ▽ More We study the tailoring of structured random graph ensembles to real networks, with the objective of generating precise and practical mathematical tools for quantifying and comparing network topologies macroscopically, beyond the level of degree statistics. Our family of ensembles can produce graphs with any prescribed degree distribution and any degree-degree correlation function, its control parameters can be calculated fully analytically, and as a result we can calculate (asymptotically) formulae for entropies and complexities, and for information-theoretic distances between networks, expressed directly and explicitly in terms of their measured degree distribution and degree correlations. △ Less

Submitted 13 August, 2009; v1 submitted 12 August, 2009; originally announced August 2009.

Comments: 25 pages, 3 figures

Parallel dynamics of disordered Ising spin systems on finitely connected directed random graphs with arbitrary degree distributions

Authors: Kazushi Mimura, A. C. C. Coolen

Abstract: We study the stochastic parallel dynamics of Ising spin systems defined on finitely connected directed random graphs with arbitrary degree distributions, using generating functional analysis. For fully asymmetric graphs the dynamics of the system can be completely solved, due to the asymptotic absence of loops. For arbitrary graph symmetry, we solve the dynamics exactly for the first few time st… ▽ More We study the stochastic parallel dynamics of Ising spin systems defined on finitely connected directed random graphs with arbitrary degree distributions, using generating functional analysis. For fully asymmetric graphs the dynamics of the system can be completely solved, due to the asymptotic absence of loops. For arbitrary graph symmetry, we solve the dynamics exactly for the first few time steps, and we construct approximate stationary solutions. △ Less

Submitted 23 August, 2009; v1 submitted 9 July, 2009; originally announced July 2009.

Comments: 20 pages, 4 figures

Journal ref: J. Phys. A: Math. Theor., 42, 415001 (2009)

Constrained Markovian dynamics of random graphs

Authors: A. C. C. Coolen, A. De Martino, A. Annibale

Abstract: We introduce a statistical mechanics formalism for the study of constrained graph evolution as a Markovian stochastic process, in analogy with that available for spin systems, deriving its basic properties and highlighting the role of the `mobility' (the number of allowed moves for any given graph). As an application of the general theory we analyze the properties of degree-preserving Markov cha… ▽ More We introduce a statistical mechanics formalism for the study of constrained graph evolution as a Markovian stochastic process, in analogy with that available for spin systems, deriving its basic properties and highlighting the role of the `mobility' (the number of allowed moves for any given graph). As an application of the general theory we analyze the properties of degree-preserving Markov chains based on elementary edge switchings. We give an exact yet simple formula for the mobility in terms of the graph's adjacency matrix and its spectrum. This formula allows us to define acceptance probabilities for edge switchings, such that the Markov chains become controlled Glauber-type detailed balance processes, designed to evolve to any required invariant measure (representing the asymptotic frequencies with which the allowed graphs are visited during the process). As a corollary we also derive a condition in terms of simple degree statistics, sufficient to guarantee that, in the limit where the number of nodes diverges, even for state-independent acceptance probabilities of proposed moves the invariant measure of the process will be uniform. We test our theory on synthetic graphs and on realistic larger graphs as studied in cellular biology. △ Less

Submitted 26 May, 2009; originally announced May 2009.

Comments: 28 pages, 6 figures

Dynamical replica analysis of processes on finitely connected random graphs II: Dynamics in the Griffiths phase of the diluted Ising ferromagnet

Authors: A. Mozeika, A. C. C. Coolen

Abstract: We study the Glauber dynamics of Ising spin models with random bonds, on finitely connected random graphs. We generalize a recent dynamical replica theory with which to predict the evolution of the joint spin-field distribution, to include random graphs with arbitrary degree distributions. The theory is applied to Ising ferromagnets on randomly diluted Bethe lattices, where we study the evolutio… ▽ More We study the Glauber dynamics of Ising spin models with random bonds, on finitely connected random graphs. We generalize a recent dynamical replica theory with which to predict the evolution of the joint spin-field distribution, to include random graphs with arbitrary degree distributions. The theory is applied to Ising ferromagnets on randomly diluted Bethe lattices, where we study the evolution of the magnetization and the internal energy. It predicts a prominent slowing down of the flow in the Griffiths phase, it suggests a further dynamical transition at lower temperatures within the Griffiths phase, and it is verified quantitatively by the results of Monte Carlo simulations. △ Less

Submitted 7 January, 2009; originally announced January 2009.

Comments: 30 pages, 4 figures, submitted to J.Phys.A

Entropies of complex networks with hierarchically constrained topologies

Authors: Ginestra Bianconi, Anthony C. C. Coolen, Conrad J. Perez Vicente

Abstract: The entropy of a hierarchical network topology in an ensemble of sparse random networks with "hidden variables" associated to its nodes, is the log-likelihood that a given network topology is present in the chosen ensemble.We obtain a general formula for this entropy,which has a clear simple interpretation in some simple limiting cases. The results provide new keys with which to solve the genera… ▽ More The entropy of a hierarchical network topology in an ensemble of sparse random networks with "hidden variables" associated to its nodes, is the log-likelihood that a given network topology is present in the chosen ensemble.We obtain a general formula for this entropy,which has a clear simple interpretation in some simple limiting cases. The results provide new keys with which to solve the general problem of "fitting" a given network with an appropriate ensemble of random networks. △ Less

Submitted 16 October, 2009; v1 submitted 8 March, 2008; originally announced March 2008.

Comments: (18 pages)

Journal ref: Phys. Rev. 78, 0161141 (2008)

Spin models on random graphs with controlled topologies beyond degree constraints

Authors: C. J. Perez-Vicente, A. C. C. Coolen

Abstract: We study Ising spin models on finitely connected random interaction graphs which are drawn from an ensemble in which not only the degree distribution $p(k)$ can be chosen arbitrarily, but which allows for further fine-tuning of the topology via preferential attachment of edges on the basis of an arbitrary function Q(k,k') of the degrees of the vertices involved. We solve these models using finit… ▽ More We study Ising spin models on finitely connected random interaction graphs which are drawn from an ensemble in which not only the degree distribution $p(k)$ can be chosen arbitrarily, but which allows for further fine-tuning of the topology via preferential attachment of edges on the basis of an arbitrary function Q(k,k') of the degrees of the vertices involved. We solve these models using finite connectivity equilibrium replica theory, within the replica symmetric ansatz. In our ensemble of graphs, phase diagrams of the spin system are found to depend no longer only on the chosen degree distribution, but also on the choice made for Q(k,k'). The increased ability to control interaction topology in solvable models beyond prescribing only the degree distribution of the interaction graph enables a more accurate modeling of real-world interacting particle systems by spin systems on suitably defined random graphs. △ Less

Submitted 4 March, 2008; originally announced March 2008.

Comments: 21 pages, 4 figures, submitted to J Phys A

Dynamical replica analysis of processes on finitely connected random graphs I: vertex covering

Authors: A. Mozeika, A. C. C. Coolen

Abstract: We study the stochastic dynamics of Ising spin models with random bonds, interacting on finitely connected Poissonnian random graphs. We use the dynamical replica method to derive closed dynamical equations for the joint spin-field probability distribution, and solve these within the replica symmetry ansatz. Although the theory is developed in a general setting, with a view to future application… ▽ More We study the stochastic dynamics of Ising spin models with random bonds, interacting on finitely connected Poissonnian random graphs. We use the dynamical replica method to derive closed dynamical equations for the joint spin-field probability distribution, and solve these within the replica symmetry ansatz. Although the theory is developed in a general setting, with a view to future applications in various other fields, in this paper we apply it mainly to the dynamics of the Glauber algorithm (extended with cooling schedules) when running on the so-called vertex cover optimization problem. Our theoretical predictions are tested against both Monte Carlo simulations and known results from equilibrium studies. In contrast to previous dynamical analyses based on deriving closed equations for only a small numbers of scalar order parameters, the agreement between theory and experiment in the present study is nearly perfect. △ Less

Submitted 7 December, 2007; originally announced December 2007.

Comments: 26 pages LaTeX, 5 figures

Generating functional analysis of batch minority games with arbitrary strategy numbers

Authors: N. Shayeghi, A. C. C. Coolen

Abstract: Both the phenomenology and the theory of minority games (MG) with more than two strategies per agent are different from those of the conventional and extensively studied case S=2. MGs with $S>2$ exhibit nontrivial statistics of the frequencies with which the agents select from their available decision making strategies, with far-reaching implications. In the few theoretical MG studies with… ▽ More Both the phenomenology and the theory of minority games (MG) with more than two strategies per agent are different from those of the conventional and extensively studied case S=2. MGs with $S>2$ exhibit nontrivial statistics of the frequencies with which the agents select from their available decision making strategies, with far-reaching implications. In the few theoretical MG studies with $S>2$ published so far, these statistics could not be calculated analytically. This prevented solution even in ergodic stationary states; equations for order parameters could only be closed approximately, using simulation data. Here we carry out a generating functional analysis of fake history batch MGs with arbitary values of $S$, and give an analytical solution of the strategy frequency problem. This leads to closed equations for order parameters in the ergodic regime, exact expressions for strategy selection statistics, and phase diagrams. Our results find perfect confirmation in numerical simulations. △ Less

Submitted 16 June, 2006; originally announced June 2006.

Comments: 16 pages LaTex, 17 eps figures (reduced in size; for high resolution versions please see http://www.mth.kcl.ac.uk/~nima/MG_MixS)

Dynamical replica analysis of disordered Ising spin systems on finitely connected random graphs

Authors: J. P. L. Hatchett, I. Pérez Castillo, A. C. C. Coolen, N. S. Skantzos

Abstract: We study the dynamics of macroscopic observables such as the magnetization and the energy per degree of freedom in Ising spin models on random graphs of finite connectivity, with random bonds and/or heterogeneous degree distributions. To do so we generalize existing implementations of dynamical replica theory and cavity field techniques to systems with strongly disordered and locally tree-like i… ▽ More We study the dynamics of macroscopic observables such as the magnetization and the energy per degree of freedom in Ising spin models on random graphs of finite connectivity, with random bonds and/or heterogeneous degree distributions. To do so we generalize existing implementations of dynamical replica theory and cavity field techniques to systems with strongly disordered and locally tree-like interactions. We illustrate our results via application to the dynamics of e.g. $\pm J$ spin-glasses on random graphs and of the overlap in finite connectivity Sourlas codes. All results are tested against Monte Carlo simulations. △ Less

Submitted 12 April, 2005; originally announced April 2005.

Comments: 4 pages, 14 .eps files

Report number: FOG 2005 1

Journal ref: Phys. Rev. Lett. (2005) 95, 117204

Reply to cond-mat/0503325: Comment on `Generating functional analysis of Minority Games with real merket histories' by KH Ho, WC Man, FK Chow and HF Chau

Authors: A. C. C. Coolen

Abstract: The Comment cond-mat/0503325 is built around two core statements, both of which are plainly incorrect. The Comment cond-mat/0503325 is built around two core statements, both of which are plainly incorrect. △ Less

Submitted 21 March, 2005; originally announced March 2005.

Comments: 1 page of LaTeX

Generating functional analysis of Minority Games with real market histories

Authors: A. C. C. Coolen

Abstract: It is shown how the generating functional method of De Dominicis can be used to solve the dynamics of the original version of the minority game (MG), in which agents observe real as opposed to fake market histories. Here one again finds exact closed equations for correlation and response functions, but now these are defined in terms of two connected effective non-Markovian stochastic processes:… ▽ More It is shown how the generating functional method of De Dominicis can be used to solve the dynamics of the original version of the minority game (MG), in which agents observe real as opposed to fake market histories. Here one again finds exact closed equations for correlation and response functions, but now these are defined in terms of two connected effective non-Markovian stochastic processes: a single effective agent equation similar to that of the `fake' history models, and a second effective equation for the overall market bid itself (the latter is absent in `fake' history models). The result is an exact theory, from which one can calculate from first principles both the persistent observables in the MG and the distribution of history frequencies. △ Less

Submitted 13 October, 2004; originally announced October 2004.

Comments: 39 pages, 5 postscript figures, iop style

Diagonalization of replicated transfer matrices for disordered Ising spin systems

Authors: T. Nikoletopoulos, A. C. C. Coolen

Abstract: We present an alternative procedure for solving the eigenvalue problem of replicated transfer matrices describing disordered spin systems with (random) 1D nearest neighbor bonds and/or random fields, possibly in combination with (random) long range bonds. Our method is based on transforming the original eigenvalue problem for a $2^n\times 2^n$ matrix (where $n\to 0$) into an eigenvalue problem f… ▽ More We present an alternative procedure for solving the eigenvalue problem of replicated transfer matrices describing disordered spin systems with (random) 1D nearest neighbor bonds and/or random fields, possibly in combination with (random) long range bonds. Our method is based on transforming the original eigenvalue problem for a $2^n\times 2^n$ matrix (where $n\to 0$) into an eigenvalue problem for integral operators. We first develop our formalism for the Ising chain with random bonds and fields, where we recover known results. We then apply our methods to models of spins which interact simultaneously via a one-dimensional ring and via more complex long-range connectivity structures, e.g. $1+\infty$ dimensional neural networks and `small world' magnets. Numerical simulations confirm our predictions satisfactorily. △ Less

Submitted 12 May, 2004; v1 submitted 12 May, 2004; originally announced May 2004.

Comments: 24 pages, LaTex, IOP macros

Asymmetrically extremely dilute neural networks with non-trivial dynamics

Authors: J. P. L. Hatchett, A. C. C. Coolen

Abstract: We study graded response attractor neural networks with asymmetrically extremely dilute interactions and Langevin dynamics. We solve our model in the thermodynamic limit using generating functional analysis, and find (in contrast to the binary neurons case) that even in statics one cannot eliminate the non-persistent order parameters. The macroscopic dynamics is driven by the (non-trivial) joint… ▽ More We study graded response attractor neural networks with asymmetrically extremely dilute interactions and Langevin dynamics. We solve our model in the thermodynamic limit using generating functional analysis, and find (in contrast to the binary neurons case) that even in statics one cannot eliminate the non-persistent order parameters. The macroscopic dynamics is driven by the (non-trivial) joint distribution of neurons and fields, rather than just the (Gaussian) field distribution. We calculate phase transition lines and present simulation results in support of our theory. △ Less

Submitted 30 April, 2004; originally announced April 2004.

Comments: 15 pages, postscript figures

Analytic solution of attractor neural networks on scale-free graphs

Authors: I. Pérez Castillo, B. Wemmenhove, J. P. L. Hatchett, A. C. C. Coolen, N. S. Skantzos, T. Nikoletopoulos

Abstract: We study the influence of network topology on retrieval properties of recurrent neural networks, using replica techniques for diluted systems. The theory is presented for a network with an arbitrary degree distribution $p(k)$ and applied to power law distributions $p(k) \sim k^{-γ}$, i.e. to neural networks on scale-free graphs. A bifurcation analysis identifies phase boundaries between the para… ▽ More We study the influence of network topology on retrieval properties of recurrent neural networks, using replica techniques for diluted systems. The theory is presented for a network with an arbitrary degree distribution $p(k)$ and applied to power law distributions $p(k) \sim k^{-γ}$, i.e. to neural networks on scale-free graphs. A bifurcation analysis identifies phase boundaries between the paramagnetic phase and either a retrieval phase or a spin glass phase. Using a population dynamics algorithm, the retrieval overlap and spin glass order parameters may be calculated throughout the phase diagram. It is shown that there is an enhancement of the retrieval properties compared with a Poissonian random graph. We compare our findings with simulations. △ Less

Submitted 1 April, 2004; originally announced April 2004.

Comments: 11 pages, 5 eps figures, LaTeX

Report number: FoG/2004/3

Slowly evolving geometry in recurrent neural networks I: extreme dilution regime

Authors: B. Wemmenhove, N. S. Skantzos, A. C. C. Coolen

Abstract: We study extremely diluted spin models of neural networks in which the connectivity evolves in time, although adiabatically slowly compared to the neurons, according to stochastic equations which on average aim to reduce frustration. The (fast) neurons and (slow) connectivity variables equilibrate separately, but at different temperatures. Our model is exactly solvable in equilibrium. We obtain… ▽ More We study extremely diluted spin models of neural networks in which the connectivity evolves in time, although adiabatically slowly compared to the neurons, according to stochastic equations which on average aim to reduce frustration. The (fast) neurons and (slow) connectivity variables equilibrate separately, but at different temperatures. Our model is exactly solvable in equilibrium. We obtain phase diagrams upon making the condensed ansatz (i.e. recall of one pattern). These show that, as the connectivity temperature is lowered, the volume of the retrieval phase diverges and the fraction of mis-aligned spins is reduced. Still one always retains a region in the retrieval phase where recall states other than the one corresponding to the `condensed' pattern are locally stable, so the associative memory character of our model is preserved. △ Less

Submitted 24 March, 2004; originally announced March 2004.

Comments: 18 pages, 6 figures

Journal ref: J. Phys. A: Math. Gen. 37 (2004) 7653

Parallel dynamics of disordered Ising spin systems on finitely connected random graphs

Authors: J. P. L. Hatchett, B. Wemmenhove, I. Perez Castillo, T. Nikoletopoulos, N. S. Skantzos, A. C. C. Coolen

Abstract: We study the dynamics of bond-disordered Ising spin systems on random graphs with finite connectivity, using generating functional analysis. Rather than disorder-averaged correlation and response functions (as for fully connected systems), the dynamic order parameter is here a measure which represents the disorder averaged single-spin path probabilities, given external perturbation field paths.… ▽ More We study the dynamics of bond-disordered Ising spin systems on random graphs with finite connectivity, using generating functional analysis. Rather than disorder-averaged correlation and response functions (as for fully connected systems), the dynamic order parameter is here a measure which represents the disorder averaged single-spin path probabilities, given external perturbation field paths. In the limit of completely asymmetric graphs our macroscopic laws close already in terms of the single-spin path probabilities at zero external field. For the general case of arbitrary graph symmetry we calculate the first few time steps of the dynamics exactly, and we work out (numerical and analytical) procedures for constructing approximate stationary solutions of our equations. Simulation results support our theoretical predictions. △ Less

Submitted 5 March, 2004; originally announced March 2004.

Comments: LaTeX, 10 eps figures, submitted to J. Phys. A

Report number: FoG/2004/2

Replicated Transfer Matrix Analysis of Ising Spin Models on `Small World' Lattices

Authors: T. Nikoletopoulos, A. C. C. Coolen, I. Perez-Castillo, N. S. Skantzos, J. P. L. Hatchett, B. Wemmenhove

Abstract: We calculate equilibrium solutions for Ising spin models on `small world' lattices, which are constructed by super-imposing random and sparse Poissonian graphs with finite average connectivity c onto a one-dimensional ring. The nearest neighbour bonds along the ring are ferromagnetic, whereas those corresponding to the Poisonnian graph are allowed to be random. Our models thus generally contain… ▽ More We calculate equilibrium solutions for Ising spin models on `small world' lattices, which are constructed by super-imposing random and sparse Poissonian graphs with finite average connectivity c onto a one-dimensional ring. The nearest neighbour bonds along the ring are ferromagnetic, whereas those corresponding to the Poisonnian graph are allowed to be random. Our models thus generally contain quenched connectivity and bond disorder. Within the replica formalism, calculating the disorder-averaged free energy requires the diagonalization of replicated transfer matrices. In addition to developing the general replica symmetric theory, we derive phase diagrams and calculate effective field distributions for two specific cases: that of uniform sparse long-range bonds (i.e. `small world' magnets), and that of (+J/-J) random sparse long-range bonds (i.e. `small world' spin-glasses). △ Less

Submitted 19 February, 2004; originally announced February 2004.

Comments: 22 pages, LaTeX, IOP macros, eps figures

Report number: FoG/2004/1

Finite Connectivity Attractor Neural Networks

Authors: B. Wemmenhove, A. C. C. Coolen

Abstract: We study a family of diluted attractor neural networks with a finite average number of (symmetric) connections per neuron. As in finite connectivity spin glasses, their equilibrium properties are described by order parameter functions, for which we derive an integral equation in replica symmetric (RS) approximation. A bifurcation analysis of this equation reveals the locations of the paramagneti… ▽ More We study a family of diluted attractor neural networks with a finite average number of (symmetric) connections per neuron. As in finite connectivity spin glasses, their equilibrium properties are described by order parameter functions, for which we derive an integral equation in replica symmetric (RS) approximation. A bifurcation analysis of this equation reveals the locations of the paramagnetic to recall and paramagnetic to spin-glass transition lines in the phase diagram. The line separating the retrieval phase from the spin-glass phase is calculated at zero temperature. All phase transitions are found to be continuous. △ Less

Submitted 11 April, 2003; originally announced April 2003.

Comments: 17 pages, 4 figures

Journal ref: J. Phys. A: Math. Gen. 36 (2003) 9617

Dynamics of a spherical minority game

Authors: T. Galla, A. C. C. Coolen, D. Sherrington

Abstract: We present an exact dynamical solution of a spherical version of the batch minority game (MG) with random external information. The control parameters in this model are the ratio of the number of possible values for the public information over the number of agents, and the radius of the spherical constraint on the microscopic degrees of freedom. We find a phase diagram with three phases: two wit… ▽ More We present an exact dynamical solution of a spherical version of the batch minority game (MG) with random external information. The control parameters in this model are the ratio of the number of possible values for the public information over the number of agents, and the radius of the spherical constraint on the microscopic degrees of freedom. We find a phase diagram with three phases: two without anomalous response (an oscillating versus a frozen state), and a further frozen phase with divergent integrated response. In contrast to standard MG versions, we can also calculate the volatility exactly. Our study reveals similarities between the spherical and the conventional MG, but also intriguing differences. Numerical simulations confirm our analytical results. △ Less

Submitted 28 March, 2003; originally announced March 2003.

Comments: 16 pages, 3 figures; submitted to J. Phys. A

Journal ref: J. Phys. A: Math. Gen. 36 (2003) 11159-11172

Coupled dynamics of sequence selection and compactification in mean-field hetero-polymers

Authors: H. Chakravorty, A. C. C. Coolen, D. Sherrington

Abstract: We study a simple solvable model describing the genesis of monomer sequences for hetero-polymers (such as proteins), as the result of the equilibration of a slow stochastic genetic selection process which is assumed to be driven by the competing demands of functionality and reproducibility of the polymer's folded structure. Since reproducibility is defined in terms of properties of the folding p… ▽ More We study a simple solvable model describing the genesis of monomer sequences for hetero-polymers (such as proteins), as the result of the equilibration of a slow stochastic genetic selection process which is assumed to be driven by the competing demands of functionality and reproducibility of the polymer's folded structure. Since reproducibility is defined in terms of properties of the folding process, one is led to the analysis of the coupled dynamics of (fast) polymer folding and (slow) genetic sequence selection. For the present mean-field model this analysis can be carried out using the finite-dimensional replica method, leading to exact results for (first- and second-order) transitions and to rich phase diagrams. △ Less

Submitted 14 June, 2002; originally announced June 2002.

Comments: 21 pages, 7 figures

Report number: KCL-MTH-02-15

Non-equilibrium statistical mechanics of Minority Games

Authors: A. C. C. Coolen

Abstract: In this paper I give a brief introduction to a family of simple but non-trivial models designed to increase our understanding of collective processes in markets, the so-called Minority Games, and their non-equilibrium statistical mathematical analysis. Since the most commonly studied members of this family define disordered stochastic processes without detailed balance, the canonical technique f… ▽ More In this paper I give a brief introduction to a family of simple but non-trivial models designed to increase our understanding of collective processes in markets, the so-called Minority Games, and their non-equilibrium statistical mathematical analysis. Since the most commonly studied members of this family define disordered stochastic processes without detailed balance, the canonical technique for finding exact solutions is found to be generating functional analysis a la De Dominicis, as originally developed in the spin-glass community. △ Less

Submitted 13 May, 2002; originally announced May 2002.

Comments: 14 pages, short review for Cergy 2002 conference proceedings

Report number: KCL-MTH-02-12

Hierarchical Self-Programming in Recurrent Neural Networks

Authors: T Uezu, A C C Coolen

Abstract: We study self-programming in recurrent neural networks where both neurons (the `processors') and synaptic interactions (`the programme') evolve in time simultaneously, according to specific coupled stochastic equations. The interactions are divided into a hierarchy of $L$ groups with adiabatically separated and monotonically increasing time-scales, representing sub-routines of the system program… ▽ More We study self-programming in recurrent neural networks where both neurons (the `processors') and synaptic interactions (`the programme') evolve in time simultaneously, according to specific coupled stochastic equations. The interactions are divided into a hierarchy of $L$ groups with adiabatically separated and monotonically increasing time-scales, representing sub-routines of the system programme of decreasing volatility. We solve this model in equilibrium, assuming ergodicity at every level, and find as our replica-symmetric solution a formalism with a structure similar but not identical to Parisi's $L$-step replica symmetry breaking scheme. Apart from differences in details of the equations (due to the fact that here interactions, rather than spins, are grouped into clusters with different time-scales), in the present model the block sizes $m_i$ of the emerging ultrametric solution are not restricted to the interval $[0,1]$, but are independent control parameters, defined in terms of the noise strengths of the various levels in the hierarchy, which can take any value in $[0,\infty\ket$. This is shown to lead to extremely rich phase diagrams, with an abundance of first-order transitions especially when the level of stochasticity in the interaction dynamics is chosen to be low. △ Less

Submitted 6 September, 2001; originally announced September 2001.

Comments: 53 pages, 19 figures. Submitted to J. Phys. A

Dynamical Solution of the On-Line Minority Game

Authors: A C C Coolen, J A F Heimel

Abstract: We solve the dynamics of the on-line minority game, with general types of decision noise, using generating functional techniques a la De Dominicis and the temporal regularization procedure of Bedeaux et al. The result is a macroscopic dynamical theory in the form of closed equations for correlation- and response functions defined via an effective continuous-time single-trader process, which are… ▽ More We solve the dynamics of the on-line minority game, with general types of decision noise, using generating functional techniques a la De Dominicis and the temporal regularization procedure of Bedeaux et al. The result is a macroscopic dynamical theory in the form of closed equations for correlation- and response functions defined via an effective continuous-time single-trader process, which are exact in both the ergodic and in the non-ergodic regime of the minority game. Our solution also explains why, although one cannot formally truncate the Kramers-Moyal expansion of the process after the Fokker-Planck term, upon doing so one still finds the correct solution, that the previously proposed diffusion matrices for the Fokker-Planck term are incomplete, and how previously proposed approximations of the market volatility can be traced back to ergodicity assumptions. △ Less

Submitted 30 July, 2001; originally announced July 2001.

Comments: 25 pages LaTeX, no figures

Dynamics of the Batch Minority Game with Inhomogeneous Decision Noise

Authors: A. C. C. Coolen, J. A. F. Heimel, D. Sherrington

Abstract: We study the dynamics of a version of the batch minority game, with random external information and with different types of inhomogeneous decision noise (additive and multiplicative), using generating functional techniques à la De Dominicis. The control parameters in this model are the ratio $α=p/N$ of the number $p$ of possible values for the external information over the number $N$ of trading… ▽ More We study the dynamics of a version of the batch minority game, with random external information and with different types of inhomogeneous decision noise (additive and multiplicative), using generating functional techniques à la De Dominicis. The control parameters in this model are the ratio $α=p/N$ of the number $p$ of possible values for the external information over the number $N$ of trading agents, and the statistical properties of the agents' decision noise parameters. The presence of decision noise is found to have the general effect of damping macroscopic oscillations, which explains why in certain parameter regions it can effectively reduce the market volatility, as observed in earlier studies. In the limit $N\to\infty$ we (i) solve the first few time steps of the dynamics (for any $α$), (ii) calculate the location $α_c$ of the phase transition (signaling the onset of anomalous response), and (iii) solve the statics for $α>α_c$. We find that $α_c$ is not sensitive to additive decision noise, but we arrive at non-trivial phase diagrams in the case of multiplicative noise. Our theoretical results find excellent confirmation in numerical simulations. △ Less

Submitted 4 July, 2001; v1 submitted 29 June, 2001; originally announced June 2001.

Comments: 14 pages LaTeX, 8 postscript figures, in PRE style (minor textual changes in revision)

Report number: KCL-MTH-01-24

Dynamics of on-line Hebbian learning with structurally unrealizable restricted training sets

Authors: Jun-ichi Inoue, A. C. C. Coolen

Abstract: We present an exact solution for the dynamics of on-line Hebbian learning in neural networks, with restricted and unrealizable training sets. In contrast to other studies on learning with restricted training sets, unrealizability is here caused by structural mismatch, rather than data noise: the teacher machine is a perceptron with a reversed wedge-type transfer function, while the student machi… ▽ More We present an exact solution for the dynamics of on-line Hebbian learning in neural networks, with restricted and unrealizable training sets. In contrast to other studies on learning with restricted training sets, unrealizability is here caused by structural mismatch, rather than data noise: the teacher machine is a perceptron with a reversed wedge-type transfer function, while the student machine is a perceptron with a sigmoidal transfer function. We calculate the glassy dynamics of the macroscopic performance measures, training error and generalization error, and the (non-Gaussian) student field distribution. Our results, which find excellent confirmation in numerical simulations, provide a new benchmark test for general formalisms with which to study unrealizable learning processes with restricted training sets. △ Less

Submitted 30 April, 2001; originally announced May 2001.

Comments: 7 pages including 3 figures, using IOP latex2e preprint class file

Solvable Lattice Gas Models of Random Heteropolymers at Finite Density: II. Dynamics and Transitions to Compact States

Authors: H. Chakravorty, J. van Mourik, A. C. C. Coolen

Abstract: In this paper we analyse both the dynamics and the high density physics of the infinite dimensional lattice gas model for random heteropolymers recently introduced in \cite{jort}. Restricting ourselves to site-disordered heteropolymers, we derive exact closed deterministic evolution equations for a suitable set of dynamic order parameters (in the thermodynamic limit), and use these to study the… ▽ More In this paper we analyse both the dynamics and the high density physics of the infinite dimensional lattice gas model for random heteropolymers recently introduced in \cite{jort}. Restricting ourselves to site-disordered heteropolymers, we derive exact closed deterministic evolution equations for a suitable set of dynamic order parameters (in the thermodynamic limit), and use these to study the dynamics of the system for different choices of the monomer polarity parameters. We also study the equilibrium properties of the system in the high density limit, which leads to a phase diagram exhibiting transitions between swollen states, compact states, and regions with partial compactification. Our results find excellent verification in numerical simulations, and have a natural and appealing interpretation in terms of real heteropolymers. △ Less

Submitted 4 July, 2001; v1 submitted 24 February, 2001; originally announced February 2001.

Comments: 12 pages, 8 eps figures, revised version (to be published in EPJ)

Supervised Learning with Restricted Training Sets: a Generating Functional Analysis

Authors: J. A. F. Heimel, A. C. C. Coolen

Abstract: We study the dynamics of supervised on-line learning of realizable tasks in feed-forward neural networks. We focus on the regime where the number of examples used for training is proportional to the number of input channels N. Using generating function techniques from spin glass theory, we are able to average over the composition of the training set and transform the problem for N to infinity to… ▽ More We study the dynamics of supervised on-line learning of realizable tasks in feed-forward neural networks. We focus on the regime where the number of examples used for training is proportional to the number of input channels N. Using generating function techniques from spin glass theory, we are able to average over the composition of the training set and transform the problem for N to infinity to an effective single pattern system, described completely by the student autocovariance, the student-teacher overlap and the student response function, with exact closed equations. Our method applies to arbitrary learning rules, i.e. not necessarily of a gradient-descent type. The resulting exact macroscopic dynamical equations can be integrated without finite-size effects up to any degree of accuracy, but their main value is in providing an exact and simple starting point for analytical approximation schemes. Finally, we show how, in the region of absent anomalous response and using the hypothesis that (as in detailed balance systems) the short-time part of the various operators can be transformed away, one can describe the stationary state of the network succesfully by a set of coupled equations involving only four scalar order parameters. △ Less

Submitted 15 February, 2001; originally announced February 2001.

Comments: 16 pages, 5 figures, uses iopart class

A Solvable Model of Secondary Structure Formation in Random Hetero-Polymers

Authors: N. S. Skantzos, J. van Mourik, A. C. C. Coolen

Abstract: We propose and solve a simple model describing secondary structure formation in random hetero-polymers. It describes monomers with a combination of one-dimensional short-range interactions (representing steric forces and hydrogen bonds) and infinite range interactions (representing polarity forces). We solve our model using a combination of mean field and random field techniques, leading to phas… ▽ More We propose and solve a simple model describing secondary structure formation in random hetero-polymers. It describes monomers with a combination of one-dimensional short-range interactions (representing steric forces and hydrogen bonds) and infinite range interactions (representing polarity forces). We solve our model using a combination of mean field and random field techniques, leading to phase diagrams exhibiting second-order transitions between folded, partially folded and unfolded states, including regions where folding depends on initial conditions. Our theoretical results, which are in excellent agreement with numerical simulations, lead to an appealing physical picture of the folding process: the polarity forces drive the transition to a collapsed state, the steric forces introduce monomer specificity, and the hydrogen bonds stabilise the conformation by damping the frustration-induced multiplicity of states. △ Less

Submitted 31 January, 2001; originally announced January 2001.

Comments: 24 pages, 14 figures

Journal ref: J. Phys. A, 34 (2001) 4437

Generating Functional Analysis of the Dynamics of the Batch Minority Game with Random External Information

Authors: J. A. F. Heimel, A. C. C. Coolen

Abstract: We study the dynamics of the batch minority game, with random external information, using generating functional techniques a la De Dominicis. The relevant control parameter in this model is the ratio $α=p/N$ of the number $p$ of possible values for the external information over the number $N$ of trading agents. In the limit $N\to\infty$ we calculate the location $α_c$ of the phase transition (si… ▽ More We study the dynamics of the batch minority game, with random external information, using generating functional techniques a la De Dominicis. The relevant control parameter in this model is the ratio $α=p/N$ of the number $p$ of possible values for the external information over the number $N$ of trading agents. In the limit $N\to\infty$ we calculate the location $α_c$ of the phase transition (signaling the onset of anomalous response), and solve the statics for $α>α_c$ exactly. The temporal correlations in global market fluctuations turn out not to decay to zero for infinitely widely separated times. For $α<α_c$ the stationary state is shown to be non-unique. For $α\to 0$ we analyse our equations in leading order in $α$, and find asymptotic solutions with diverging volatility $σ=\order(α^{-{1/2}})$ (as regularly observed in simulations), but also asymptotic solutions with vanishing volatility $σ=\order(α^{1/2})$. The former, however, are shown to emerge only if the agents' initial strategy valuations are below a specific critical value. △ Less

Submitted 11 December, 2000; v1 submitted 4 December, 2000; originally announced December 2000.

Comments: 15 pages, 6 figures, uses Revtex. Replaced an old version of volatility graph that. Rephrased and updated some references

Journal ref: Phys. Rev. E 63, 056121 (2001)

Coupled dynamics of fast spins and slow exchange interactions in the XY spin-glass

Authors: G Jongen, J Anemueller, D Bolle, A C C Coolen, C Perez-Vicente

Abstract: We investigate an XY spin-glass model in which both spins and interactions (or couplings) evolve in time, but with widely separated time-scales. For large times this model can be solved using replica theory, requiring two levels of replicas, one level for the spins and one for the couplings. We define the relevant order parameters, and derive a phase diagram in the replica-symmetric approximatio… ▽ More We investigate an XY spin-glass model in which both spins and interactions (or couplings) evolve in time, but with widely separated time-scales. For large times this model can be solved using replica theory, requiring two levels of replicas, one level for the spins and one for the couplings. We define the relevant order parameters, and derive a phase diagram in the replica-symmetric approximation, which exhibits two distinct spin-glass phases. The first phase is characterized by freezing of the spins only, whereas in the second phase both spins and couplings are frozen. A detailed stability analysis leads also to two distinct corresponding de Almeida-Thouless lines, each marking continuous replica-symmetry breaking. Numerical simulations support our theoretical study. △ Less

Submitted 3 November, 2000; originally announced November 2000.

Comments: 30 pages, 15 figures, LaTeX (J. Phys. A style)

Report number: KCL-MTH-00-60

Cluster derivation of Parisi's RSB solution for disordered systems

Authors: J. van Mourik, A. C. C. Coolen

Abstract: We propose a general scheme in which disordered systems are allowed to sacrifice energy equi-partitioning and separate into a hierarchy of ergodic sub-systems (clusters) with different characteristic time-scales and temperatures. The details of the break-up follow from the requirement of stationarity of the entropy of the slower cluster, at every level in the hierarchy. We apply our ideas to the… ▽ More We propose a general scheme in which disordered systems are allowed to sacrifice energy equi-partitioning and separate into a hierarchy of ergodic sub-systems (clusters) with different characteristic time-scales and temperatures. The details of the break-up follow from the requirement of stationarity of the entropy of the slower cluster, at every level in the hierarchy. We apply our ideas to the Sherrington-Kirkpatrick model, and show how the Parisi solution can be {\it derived} quantitatively from plausible physical principles. Our approach gives new insight into the physics behind Parisi's solution and its relations with other theories, numerical experiments, and short range models. △ Less

Submitted 11 September, 2000; originally announced September 2000.

Comments: 7 pages 5 figures

Attractor Modulation and Proliferation in 1+$\infty$ Dimensional Neural Networks

Authors: N. S. Skantzos, A. C. C. Coolen

Abstract: We extend a recently introduced class of exactly solvable models for recurrent neural networks with competition between 1D nearest neighbour and infinite range information processing. We increase the potential for further frustration and competition in these models, as well as their biological relevance, by adding next-nearest neighbour couplings, and we allow for modulation of the attractors so… ▽ More We extend a recently introduced class of exactly solvable models for recurrent neural networks with competition between 1D nearest neighbour and infinite range information processing. We increase the potential for further frustration and competition in these models, as well as their biological relevance, by adding next-nearest neighbour couplings, and we allow for modulation of the attractors so that we can interpolate continuously between situations with different numbers of stored patterns. Our models are solved by combining mean field and random field techniques. They exhibit increasingly complex phase diagrams with novel phases, separated by multiple first- and second order transitions (dynamical and thermodynamic ones), and, upon modulating the attractor strengths, non-trivial scenarios of phase diagram deformation. Our predictions are in excellent agreement with numerical simulations. △ Less

Submitted 2 August, 2000; originally announced August 2000.

Comments: 16 pages, 15 postscript figures, Latex

Report number: Department of Mathematics, King's College London, KCL-MTH-00-49

Journal ref: J. Phys. A, 34 (2001) 929

Cluster Derivation of the Parisi Scheme for Disordered Systems

Authors: A. C. C. Coolen, J. van Mourik

Abstract: We propose a general quantitative scheme in which systems are given the freedom to sacrifice energy equi-partitioning on the relevant time-scales of observation, and have phase transitions by separating autonomously into ergodic sub-systems (clusters) with different characteristic time-scales and temperatures. The details of the break-up follow uniquely from the requirement of zero entropy for t… ▽ More We propose a general quantitative scheme in which systems are given the freedom to sacrifice energy equi-partitioning on the relevant time-scales of observation, and have phase transitions by separating autonomously into ergodic sub-systems (clusters) with different characteristic time-scales and temperatures. The details of the break-up follow uniquely from the requirement of zero entropy for the slower cluster. Complex systems, such as the Sherrington-Kirkpatrick model, are found to minimise their free energy by spontaneously decomposing into a hierarchy of ergodically equilibrating degrees of freedom at different (effective) temperatures. This leads exactly and uniquely to Parisi's replica symmetry breaking scheme. Our approach, which is somewhat akin to an earlier one by Sompolinsky, gives new insight into the physical interpretation of the Parisi scheme and its relations with other approaches, numerical experiments, and short range models. Furthermore, our approach shows that the Parisi scheme can be derived quantitatively and uniquely from plausible physical principles. △ Less

Submitted 19 July, 2000; originally announced July 2000.

Comments: 6 pages, 3 figures, proceedings of international conference on "Disordered And Complex Systems", 10-14 July 2000 King's College London

Non-Deterministic Learning Dynamics in Large Neural Networks due to Structural Data Bias

Authors: H. C. Rae, J. A. F. Heimel, A. C. C. Coolen

Abstract: We study the dynamics of on-line learning in large perceptrons, for the case of training sets with a structural bias of the input vectors, by deriving exact and closed macroscopic dynamical laws using non-equilibrium statistical mechanical tools. In sharp contrast to the more conventional theories developed for homogeneously distributed or only weakly biased data, these laws are found to describ… ▽ More We study the dynamics of on-line learning in large perceptrons, for the case of training sets with a structural bias of the input vectors, by deriving exact and closed macroscopic dynamical laws using non-equilibrium statistical mechanical tools. In sharp contrast to the more conventional theories developed for homogeneously distributed or only weakly biased data, these laws are found to describe a non-trivial and persistently non-deterministic macroscopic evolution, and a generalisation error which retains both stochastic and sample-to-sample fluctuations, even for infinitely large networks. Furthermore, for the standard error-correcting microscopic algorithms (such as the perceptron learning rule) one obtains learning curves with distinct bias-induced phases. Our theoretical predictions find excellent confirmation in numerical simulations. △ Less

Submitted 13 July, 2000; originally announced July 2000.

Comments: 20 pages

Report number: KCL-MTH-00-39

Journal ref: J.Phys.A. 33 (2000) 8703-8722

Statistical Mechanics of Recurrent Neural Networks II. Dynamics

Authors: A. C. C. Coolen

Abstract: A lecture notes style review of the non-equilibrium statistical mechanics of recurrent neural networks with discrete and continuous neurons (e.g. Ising, graded-response, coupled-oscillators). To be published in the Handbook of Biological Physics (North-Holland). Accompanied by a similar review (part I) dealing with the statics. A lecture notes style review of the non-equilibrium statistical mechanics of recurrent neural networks with discrete and continuous neurons (e.g. Ising, graded-response, coupled-oscillators). To be published in the Handbook of Biological Physics (North-Holland). Accompanied by a similar review (part I) dealing with the statics. △ Less

Submitted 1 June, 2000; originally announced June 2000.

Comments: 49 pages, LaTeX

Report number: KCL-MTH-00-11

Statistical Mechanics of Recurrent Neural Networks I. Statics

Authors: A. C. C. Coolen

Abstract: A lecture notes style review of the equilibrium statistical mechanics of recurrent neural networks with discrete and continuous neurons (e.g. Ising, coupled-oscillators). To be published in the Handbook of Biological Physics (North-Holland). Accompanied by a similar review (part II) dealing with the dynamics. A lecture notes style review of the equilibrium statistical mechanics of recurrent neural networks with discrete and continuous neurons (e.g. Ising, coupled-oscillators). To be published in the Handbook of Biological Physics (North-Holland). Accompanied by a similar review (part II) dealing with the dynamics. △ Less

Submitted 1 June, 2000; originally announced June 2000.

Comments: 49 pages, LaTeX

Report number: KCL-MTH-00-11

Exact Solution of the Infinite-Range Quantum Mattis Model

Authors: P. Sollich, Hidetoshi Nishimori, A. C. C. Coolen, A. J. van der Sijs

Abstract: We have solved the quantum version of the Mattis model with infinite-range interactions. A variational approach gives the exact solution for the infinite-range system, in spite of the non-commutative nature of the quantum spin components; this implies that quantum effects are not predominant in determining the macroscopic properties of the system. Nevertheless, the model has a surprisingly rich… ▽ More We have solved the quantum version of the Mattis model with infinite-range interactions. A variational approach gives the exact solution for the infinite-range system, in spite of the non-commutative nature of the quantum spin components; this implies that quantum effects are not predominant in determining the macroscopic properties of the system. Nevertheless, the model has a surprisingly rich phase behaviour, exhibiting phase diagrams with tricritical, three-phase and critical end points. △ Less

Submitted 7 May, 2000; originally announced May 2000.

Comments: 14 pages, 11 figures

Journal ref: J. Phys. Soc. Jpn. 69 (2000) 3200