Brain-State-in-Box (BSB) Model

The "brain-state-in-box" sounds like we have a brain in a box without body.  But the model is defined as follows:

Let W be a symmetric weight matrix whose largest eigenvalues have positive real components.  In addition, W is required to be positive semi-definite, i.e.,  x^TWx >= 0 for all x.  Let x(0) denotes the initial state vector.  The BSB algorithm is defined by the pair of equations:

        y(n) = x(n) + h Wx(n),

        x(n+1) = f(y(n)).

Or more concisely, the updating rule of the "brain state" x (a vector) is

        x  <-  f(x + h W x)

where h is a small positive constant called the feedback factor.  The f is a piecewise-linear function of the form

        f(x) = +1     if x > 1;

        f(x) = x     if  -1 < x < 1;

        f(x) = -1     if x < -1.

When the W is choosing with the required property (positivity of largest eigenvalues), the effect of the algorithm is to drive the system for components of x to binary values +1 or 1 for each of the neuron.   We can view it as a mapping from continuous inputs x(0) to discrete binary outputs.  The final states are of the form (-1,+1,-1,-1,+1,+1, ..., +1).  This represents a corner of cube in an N-dimensional space of linear size 2, centered at origin.  This is the box of the brain-state-in "a box".   The dynamics is such that the state moves to the wall of the box and then drives to the corner of the box.

There is a similar energy function for the BSB model given by

        E = - (h/2) S_ij w_ij x_i x_j = - (h/2) x^T W x.

Exercise: show that the BSB dynamics reduces energy or at least does not increase E.

It turns out much more general conditions exist to decide if an energy function exists.  This is given in the form a Cohen-Grossberg theorem discussed in the textbook.

What is a good use of BSB model?  A natural application for the BSB model is clustering.  Such as the classification of radar signals from the source of emitters.  The matrix W has to be (unsupervised) learned using some of the methods discussed in early chapters. 

Implement Hopfield network model to store the two pictures past to you earlier and some of your own digital pictures (back and white only).  Study its effectiveness by runs the Hopfield dynamics starting with a corrupted picture or picture with noise added.