Hopfield Network - notes on the program

The Hopfield Network is an auto-associative network that can be used to store (and reconstruct) bipolar patterns. It consists of only one layer, which acts as both input and output layer. The activity of a node can take the values -1 and 1 and each node can read the activities of the other nodes, where each incoming signal is associated with a weight.

To save a pattern, the activities are set accordingly and then the weights are adjusted according to Hebb's learning rule, i.e. they are strengthened if the activities of two connected nodes have the same value and weakened if this is not the case.

To repair a noisy pattern, the activities are first set accordingly and then adjusted according to a simple rule until a stable final state is reached, which is guaranteed. The simple rule calculates the sign of the scalar product from the activities of the other nodes and the assigned weights.

In our example, we use a two-dimensional layer to store two patterns.


Unit Class HopfieldUnit - Using JLayer Annotations

The HopfieldUnit class provides fields and methods for saving and reconstructing patterns. At the top is the nested static class Signal for modelling bipolar values. This is followed by three layer fields:

  • LayerField activation of type Signal stores the current activation of the node and is accompanied by the field prev for storing the previous activation.
  • LayerField input of type Signal[] is connected to the activation fields of the other nodes and is accompanied by the vector field weight, which contains a weight component for each incoming activation.
  • LayerField ix of type int[] is used to store the index of the current node.

Since the parameter isIndex has the value true, LayerField ix is calculated automatically when an array of type HopfieldUnit is wrapped. 


@LayerUnit
public class HopfieldUnit {

  static class Signal {
    int val;				// bipolar values
  } 

  @LayerField 
  Signal activation = new Signal();	// activation signal
  Signal prev = new Signal();		// previous activation

  @LayerField 
  Signal[] input;			// signals from other units
  int[] weight;				// weight vector

  @LayerField(isIndex = true)
  int[] ix;				// index of this unit

  @LayerMethod
  void initWeights() {
    weight = new int[input.length];
    for (int i = 0; i < input.length; i++) {
      weight[i] = 0;
    }
  }
  
  @LayerMethod
  void setActivation(int[][] pattern) {
    activation.val = pattern[ix[0]][ix[1]];
  }
	
  @LayerMethod
  void storePattern() {
    for (int i = 0; i < input.length; i++) {
      weight[i] = weight[i] + activation.val * input[i].val;
    }
  }
	
  @LayerMethod
  void nextActivation() {
    int dotproduct = 0;
    for (int i = 0; i <input.length; i++) {
      dotproduct += weight[i] * input[i].val;
    }
    prev.val = activation.val;
    activation.val = (int)Math.signum(dotproduct);
    if (activation.val == 0) activation.val = prev.val;
  }

  @LayerMethod
  boolean isStable() {
    return activation.val == prev.val;
  }

  @LayerMethod
  Signal getActivation() {
    return activation;
  }
    
}

The rest of the class definition consists of layer methods.

  • LayerMethod initWeights() is used to initialise the weight vector.
  • LayerMethod setActivation() is used to input a pattern to be stored or reconstructed.
  • LayerMethod storePattern() is used to store a pattern. The weights are adjusted according to the Hebbian learning rule.
  • LayerMethod nextActivation() is used to reconstruct a pattern (one step). The sign of the scalar product is calculated from the incoming activities and the assigned weights.
  • LayerMethod isStable() can be used to check whether a stable state has been reached during the reconstruction of a pattern.
  • LayerMethod getActivation() is used to output a (reconstructed) pattern.

Establishing Global Structure and Dynamics - Using the Generated Wrapper Class

From unit class HopfieldUnit the JLayer annotation processor automatically generates the wrapper class Layer_HopfieldUnit_ which is used by class HopfieldNet to model the global structure and dynamics of a two dimensional Hopfield network.

At the top, the definition of relation others can be found, which specifically applies for all different pairs of two-dimensional indices. This is followed by the fields lines and cols, used to store the size of the network, and the fields hopArray and hopLayer, used to refer to the actual array of HopfieldUnits or to the wrapped array, respectively. 


public class HopfieldNet {

  Relation others = new Relation() {
    @Override
    public boolean contains(int[] x, int[] y) {
      return (x[0] != y[0]) | (x[1] != y[1]);
    }
  };
	
  int lines, cols;

  HopfieldUnit[][]    hopArray;
  Layer_HopfieldUnit_ hopLayer;

  public void createNet(int lines, int cols) {
    this.lines = lines;
    this.cols = cols;
    hopArray = new HopfieldUnit[lines][cols];
    for(int i = 0; i < lines; i++) {
      for(int j = 0; j < cols; j++) {
        hopArray[i][j] = new HopfieldUnit();
      }
    }
    hopLayer = new Layer_HopfieldUnit_(hopArray);
    hopLayer.input.connect(hopLayer.activation, others);
    hopLayer.initWeights();
  }

  public void setActivations(int[][] pattern) {
    hopLayer.setActivation(pattern);
  }

  public void storePattern(int[][] pattern) {
    hopLayer.storePattern();
  }

  public boolean nextActivations() {
    hopLayer.nextActivation();
    BasedLayer isStable = hopLayer.isStable();
    for(int i = 0; i < lines; i++) {
      for(int j = 0; j < cols; j++) {
        if (!isStable.get(i,j)) return false;
      }
    }
    return true;
  }

  public BasedLayer getActivations() {
    return hopLayer.getActivation();
  }

}

Further modelling is carried out using five methods based on the fields mentioned above.

  • The createNet() method creates and initialises the array hopArray, wraps it and binds the result to the hopLayer variable. Relation others is then used to link the vector layer hopLayer.input with the field layer hopLayer.activation, and finally the weights of the network are initialised.
  • Method setActivation() can be used to input a pattern.
  • Method storePattern() can be used to store a pattern.
  • Method nextActivation() performs one step in the pattern recognition process.
  • Method getActivation() is used to output a (reconstructed) pattern.

More Information