Showing 1–9 of 9 results for author: Vicente, R

Restricted Boltzmann Machine Flows and The Critical Temperature of Ising models

Authors: Rodrigo Veiga, Renato Vicente

Abstract: We explore alternative experimental setups for the iterative sampling (flow) from Restricted Boltzmann Machines (RBM) mapped on the temperature space of square lattice Ising models by a neural network thermometer. This framework has been introduced to explore connections between RBM-based deep neural networks and the Renormalization Group (RG). It has been found that, under certain conditions, the… ▽ More We explore alternative experimental setups for the iterative sampling (flow) from Restricted Boltzmann Machines (RBM) mapped on the temperature space of square lattice Ising models by a neural network thermometer. This framework has been introduced to explore connections between RBM-based deep neural networks and the Renormalization Group (RG). It has been found that, under certain conditions, the flow of an RBM trained with Ising spin configurations approaches in the temperature space a value around the critical one: $ k_B T_c / J \approx 2.269$. In this paper we consider datasets with no information about model topology to argue that a neural network thermometer is not an accurate way to detect whether the RBM has learned scale invariance or not. △ Less

Submitted 29 March, 2022; v1 submitted 17 June, 2020; originally announced June 2020.

Comments: 13 figures. Code available at https://github.com/rodsveiga/rbm_flows_ising

Statistical mechanics of reputation systems in autonomous networks

Authors: Andre Manoel, Renato Vicente

Abstract: Reputation systems seek to infer which members of a community can be trusted based on ratings they issue about each other. We construct a Bayesian inference model and simulate approximate estimates using belief propagation (BP). The model is then mapped onto computing equilibrium properties of a spin glass in a random field and analyzed by employing the replica symmetric cavity approach. Having th… ▽ More Reputation systems seek to infer which members of a community can be trusted based on ratings they issue about each other. We construct a Bayesian inference model and simulate approximate estimates using belief propagation (BP). The model is then mapped onto computing equilibrium properties of a spin glass in a random field and analyzed by employing the replica symmetric cavity approach. Having the fraction of trustful nodes and environment noise level as control parameters, we evaluate the theoretical performance in terms of estimation error and the robustness of the BP approximation in different scenarios. Regions of degraded performance are then explained by the convergence properties of the BP algorithm and by the emergence of a glassy phase. △ Less

Submitted 11 April, 2013; v1 submitted 27 November, 2012; originally announced November 2012.

Comments: 20 pages, 14 figures

Journal ref: Journal of Statistical Mechanics (2013) P08002

Common Underlying Dynamics in an Emerging Market: From Minutes to Months

Authors: Renato Vicente, Charles M. de Toledo, Vitor B. P. Leite, Nestor Caticha

Abstract: We analyse a period spanning 35 years of activity in the Sao Paulo Stock Exchange Index (IBOVESPA) and show that the Heston model with stochastic volatility is capable of explaining price fluctuations for time scales ranging from 5 minutes to 100 days with a single set of parameters. We also show that the Heston model is inconsistent with the observed behavior of the volatility autocorrelation f… ▽ More We analyse a period spanning 35 years of activity in the Sao Paulo Stock Exchange Index (IBOVESPA) and show that the Heston model with stochastic volatility is capable of explaining price fluctuations for time scales ranging from 5 minutes to 100 days with a single set of parameters. We also show that the Heston model is inconsistent with the observed behavior of the volatility autocorrelation function. We deal with the latter inconsistency by introducing a slow time scale to the model. The fact that the price dynamics in a period of 35 years of macroeconomical unrest may be modeled by the same stochastic process is evidence for a general underlying microscopic market dynamics. △ Less

Submitted 5 February, 2004; originally announced February 2004.

Comments: 11 pages, 8 figures, subimitted

Error-correcting code on a cactus: a solvable model

Authors: Renato Vicente, David Saad, Yoshiyuki Kabashima

Abstract: An exact solution to a family of parity check error-correcting codes is provided by mapping the problem onto a Husimi cactus. The solution obtained in the thermodynamic limit recovers the replica symmetric theory results and provides a very good approximation to finite systems of moderate size. The probability propagation decoding algorithm emerges naturally from the analysis. A phase transition… ▽ More An exact solution to a family of parity check error-correcting codes is provided by mapping the problem onto a Husimi cactus. The solution obtained in the thermodynamic limit recovers the replica symmetric theory results and provides a very good approximation to finite systems of moderate size. The probability propagation decoding algorithm emerges naturally from the analysis. A phase transition between decoding success and failure phases is found to coincide with an information-theoretic upper bound. The method is employed to compare Gallager and MN codes. △ Less

Submitted 30 May, 2000; v1 submitted 5 May, 2000; originally announced May 2000.

Comments: 7 pages, 3 figures, with minor corrections

Journal ref: Europhys. Lett. 51 (2000), 698-704

The Statistical Physics of Regular Low-Density Parity-Check Error-Correcting Codes

Authors: Tatsuto Murayama, Yoshiyuki Kabashima, David Saad, Renato Vicente

Abstract: A variation of Gallager error-correcting codes is investigated using statistical mechanics. In codes of this type, a given message is encoded into a codeword which comprises Boolean sums of message bits selected by two randomly constructed sparse matrices. The similarity of these codes to Ising spin systems with random interaction makes it possible to assess their typical performance by analytic… ▽ More A variation of Gallager error-correcting codes is investigated using statistical mechanics. In codes of this type, a given message is encoded into a codeword which comprises Boolean sums of message bits selected by two randomly constructed sparse matrices. The similarity of these codes to Ising spin systems with random interaction makes it possible to assess their typical performance by analytical methods developed in the study of disordered systems. The typical case solutions obtained via the replica method are consistent with those obtained in simulations using belief propagation (BP) decoding. We discuss the practical implications of the results obtained and suggest a computationally efficient construction for one of the more practical configurations. △ Less

Submitted 8 March, 2000; originally announced March 2000.

Comments: 35 pages, 4 figures

Journal ref: Phys. Rev. E 62(2) 1577-1591 (2000)

Statistical Physics of Irregular Low-Density Parity-Check Codes

Authors: Renato Vicente, David Saad, Yoshiyuki Kabashima

Abstract: Low-density parity-check codes with irregular constructions have been recently shown to outperform the most advanced error-correcting codes to date. In this paper we apply methods of statistical physics to study the typical properties of simple irregular codes. We use the replica method to find a phase transition which coincides with Shannon's coding bound when appropriate parameters are chosen… ▽ More Low-density parity-check codes with irregular constructions have been recently shown to outperform the most advanced error-correcting codes to date. In this paper we apply methods of statistical physics to study the typical properties of simple irregular codes. We use the replica method to find a phase transition which coincides with Shannon's coding bound when appropriate parameters are chosen. The decoding by belief propagation is also studied using statistical physics arguments; the theoretical solutions obtained are in good agreement with simulations. We compare the performance of irregular with that of regular codes and discuss the factors that contribute to the improvement in performance. △ Less

Submitted 3 April, 2000; v1 submitted 25 August, 1999; originally announced August 1999.

Comments: 20 pages, 9 figures, revised version submitted to JPA

Journal ref: J. Phys. A 33 (2000) 6527-6542

Finite-connectivity systems as error-correcting codes

Authors: Renato Vicente, David Saad, Yoshiyuki Kabashima

Abstract: We investigate the performance of parity check codes using the mapping onto Ising spin systems proposed by Sourlas. We study codes where each parity check comprises products of K bits selected from the original digital message with exactly C checks per message bit. We show, using the replica method, that these codes saturate Shannon's coding bound for $K\to\infty$ when the code rate K/C is finit… ▽ More We investigate the performance of parity check codes using the mapping onto Ising spin systems proposed by Sourlas. We study codes where each parity check comprises products of K bits selected from the original digital message with exactly C checks per message bit. We show, using the replica method, that these codes saturate Shannon's coding bound for $K\to\infty$ when the code rate K/C is finite. We then examine the finite temperature case to asses the use of simulated annealing methods for decoding, study the performance of the finite K case and extend the analysis to accommodate different types of noisy channels. The connection between statistical physics and belief propagation decoders is discussed and the dynamics of the decoding itself is analyzed. Further insight into new approaches for improving the code performance is given. △ Less

Submitted 20 August, 1999; v1 submitted 23 April, 1999; originally announced April 1999.

Comments: 32 pages, 12 figures, to appear in PRE

Journal ref: Phys. Rev. E 60 5352-5366 (1999)

Statistical Mechanics of Online Learning of Drifting Concepts : A Variational Approach

Authors: Renato Vicente, Osame Kinouchi, Nestor Caticha

Abstract: We review the application of Statistical Mechanics methods to the study of online learning of a drifting concept in the limit of large systems. The model where a feed-forward network learns from examples generated by a time dependent teacher of the same architecture is analyzed. The best possible generalization ability is determined exactly, through the use of a variational method. The construct… ▽ More We review the application of Statistical Mechanics methods to the study of online learning of a drifting concept in the limit of large systems. The model where a feed-forward network learns from examples generated by a time dependent teacher of the same architecture is analyzed. The best possible generalization ability is determined exactly, through the use of a variational method. The constructive variational method also suggests a learning algorithm. It depends, however, on some unavailable quantities, such as the present performance of the student. The construction of estimators for these quantities permits the implementation of a very effective, highly adaptive algorithm. Several other algorithms are also studied for comparison with the optimal bound and the adaptive algorithm, for different types of time evolution of the rule. △ Less

Submitted 28 January, 1998; originally announced January 1998.

Comments: 24 pages, 8 figures, to appear in Machine Learning Journal

Journal ref: Machine Learning 32 179-201 (1998)

Functional Optimisation of Online Algorithms in Multilayer Neural Networks

Authors: Renato Vicente, Nestor Caticha

Abstract: We study the online dynamics of learning in fully connected soft committee machines in the student-teacher scenario. The locally optimal modulation function, which determines the learning algorithm, is obtained from a variational argument in such a manner as to maximise the average generalisation error decay per example. Simulations results for the resulting algorithm are presented for a few cas… ▽ More We study the online dynamics of learning in fully connected soft committee machines in the student-teacher scenario. The locally optimal modulation function, which determines the learning algorithm, is obtained from a variational argument in such a manner as to maximise the average generalisation error decay per example. Simulations results for the resulting algorithm are presented for a few cases. The symmetric phase plateaux are found to be vastly reduced in comparison to those found when online backpropagation algorithms are used. A discussion of the implementation of these ideas as practical algorithms is given. △ Less

Submitted 2 June, 1997; originally announced June 1997.

Journal ref: J. Phys A: Math. Gen. 30 (1997) L599-L605