Subscribe to the gold package and get unlimited access to Shamra Academy
Register a new userAn optimal Q-state neural network using mutual information
61
0
0.0
(
0
)
Ask ChatGPT about the research
No Arabic abstract
Starting from the mutual information we present a method in order to find a hamiltonian for a fully connected neural network model with an arbitrary, finite number of neuron states, Q. For small initial correlations between the neurons and the patterns it leads to optimal retrieval performance. For binary neurons, Q=2, and biased patterns we recover the Hopfield model. For three-state neurons, Q=3, we find back the recently introduced Blume-Emery-Griffiths network hamiltonian. We derive its phase diagram and compare it with those of related three-state models. We find that the retrieval region is the largest.
rate research
Read More
It is known that a trained Restricted Boltzmann Machine (RBM) on the binary Monte Carlo Ising spin configurations, generates a series of iterative reconstructed spin configurations which spontaneously flow and stabilize to the critical point of physical system. Here we construct a variety of Neural Network (NN) flows using the RBM and (variational) autoencoders, to study the q-state Potts and clock models on the square lattice for q = 2, 3, 4. The NN are trained on Monte Carlo spin configurations at various temperatures. We find that the trained NN flow does develop a stable point that coincides with critical point of the q-state spin models. The behavior of the NN flow is nontrivial and generative, since the training is unsupervised and without any prior knowledge about the critical point and the Hamiltonian of the underlying spin model. Moreover, we find that the convergence of the flow is independent of the types of NNs and spin models, hinting a universal behavior. Our results strengthen the potential applicability of the notion of the NN flow in studying various states of matter and offer additional evidence on the connection with the Renormalization Group flow.
The dynamics and the stationary states of an exactly solvable three-state layered feed-forward neural network model with asymmetric synaptic connections, finite dilution and low pattern activity are studied in extension of a recent work on a recurrent network. Detailed phase diagrams are obtained for the stationary states and for the time evolution of the retrieval overlap with a single pattern. It is shown that the network develops instabilities for low thresholds and that there is a gradual improvement in network performance with increasing threshold up to an optimal stage. The robustness to synaptic noise is checked and the effects of dilution and of variable threshold on the information content of the network are also established.
The optimal capacity of a diluted Blume-Emery-Griffiths neural network is studied as a function of the pattern activity and the embedding stability using the Gardner entropy approach. Annealed dilution is considered, cutting some of the couplings referring to the ternary patterns themselves and some of the couplings related to the active patterns, both simultaneously (synchronous dilution) or independently (asynchronous dilution). Through the de Almeida-Thouless criterion it is found that the replica-symmetric solution is locally unstable as soon as there is dilution. The distribution of the couplings shows the typical gap with a width depending on the amount of dilution, but this gap persists even in cases where a particular type of coupling plays no role in the learning process.
The three-state Ising neural network with synchronous updating and variable dilution is discussed starting from the appropriate Hamiltonians. The thermodynamic and retrieval properties are examined using replica mean-field theory. Capacity-temperature phase diagrams are derived for several values of the pattern activity and different gradations of dilution, and the information content is calculated. The results are compared with those for sequential updating. The effect of self-coupling is established. Also the dynamics is studied using the generating function technique for both synchronous and sequential updating. Typical flow diagrams for the overlap order parameter are presented. The differences with the signal-to-noise approach are outlined.
An Ashkin-Teller neural network, allowing for two types of neurons is considered in the case of low loading as a function of the strength of the respective couplings between these neurons. The storage and retrieval of embedded patterns built from the two types of neurons, with different degrees of (in)dependence is studied. In particular, thermodynamic properties including the existence and stability of Mattis states are discussed. Furthermore, the dynamic behaviour is examined by deriving flow equations for the macroscopic overlap. It is found that for linked patterns the model shows better retrieval properties than a corresponding Hopfield model.
Log in to be able to interact and post comments
comments
Fetching comments
Sign in to be able to follow your search criteria
