Linear Associative Network

Learning and memory

    Learning --- Aquisition

   Memory --- storage and retrieval of information

 

   Associative learning

        is based on the relationship of one stimulus to another

   Examples:

        Classical conditioning (Pavlov)

            bell ring --- food --- salivation

        Operant conditioning (Thorndike)

            trial-and-error learning

            lever pressing (out of curiosity) ---> food.

 

   Non-associative learning

        Habituation --- decrease in behavior reflex response to repeated harmless stimulus

         Imitation learning

 

   Stages of memory

        Iconic --- e.g. visual afterimages (1 sec)

        Short-term working memory

            small capacity (7 items plus/minus 1)

            What is the possible mechanism?

 

        Long-term memory

                large capacity (some estimates 109 bits, relatively permanent).

 

Hebbian rule for synaptic modification

Physical basis of memory is in synaptic modification which alters the mapping of inputs to outputs.
The strength of the modification is determined by how often input and output activity occurs together, and by the strengths of the input and output activities.

Hebb ("Organization of behavior", 1949) ---  "When an axon of cell A is near enough to excite a cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells, such that A's efficiency as one of the cells firing B, is increased"     

A simple model

        DWij = a figj

Linear model of associative memory

Storage: Let {fn,gn} be a set of input/output activity pairs.

            WnWn-1+gnfnT

Recall:   g = Wf
Perfect Recall, if {fn} are orthonormal, the system shows perfect recall.

    g = Wnf = gm

An example (training of association B->I, I->T, T->P, Mathematica code here)

Interference effect. As the number of instances increases, the quality of recall decreases.
The recall is rather robust against noise.
Input needs not be complete to get correct output