AssociativeMemoryResearch

Associative Memory Research

@@Home -> ArtificialIntelligenceResearch -> AssociativeMemoryResearch

---

Associative Memory (AM) research covers technologies enabling implementation of associative memory which enables thought process and links previous experience to novel situations.

Technologies

• Kohonen networks

Types of Associations

Feel the difference between:

• Clear concept can be restored from noisy data
• Most related concept can be restored by its small part
• Several concepts can be derived from feature/another concept

Thoughts

Pribram's model

• alternative to the transcortical model of neocortical organization
  • extrinsic sectors (primary projection areas) - neocortical areas whose fibers enter or leave the cerebral hemispheres
  • intrinsic sectors (association areas) - their fibers remain within the cerebrum
• principal interaction of extrinsic and intrinsic systems occurs at the thalamic level
  • contribution of intrinsic neocortex to the final output of the extrinsic system is mediated by the convergence of influences from both intrinsic and extrinsic systems by subcortical mechanisms
  • intrinsic system may influence also the input of the extrinsic systems by regulation of peripheral sensory mechanisms

Interesting Pictures

• Human Memory Systems - see link

• Cognitive Cycle - see Link

• Generic Auto-Associative Memory - see link

• Context Binding - see link

Articles Review

---

Multi-Associative Memory in fLIF Cell Assemblies (CA)

see link.

Based on:

• Hebbs Cell Assembly Theory (CA is neural basis for concepts)
• network of biologically plausible fLIF (fatiguing, Leaky, Integrate and Fire) neurons

Introduction, Background:

• hypo: Concepts are stored as CAs, associations are connections between CAs
• concepts connected as 1-1,1-N,N-M
• associations can be context-sensitive - retrieval of an associated concept can be based on a combination of the base concept and the context
• AM features: priming, differential associations, timing, gradual learning and change, encoding instances (and others)

CAs and auto-associative memory:

• CA theory: objects, ideas, stimuli and even abstract concepts are represented in the brain by simultaneous activation of large groups of neurons with high mutual synaptic strengths
• long-term memory: neurons are learned by Hebbian rule from mutual activation, gradually assembling into CAs after repeated and persistent activation
• short-term memory: CA is activated when its certain number of neurons is activated, then CA reverberates due to high mutual synaptic strengths
• CA is a form of auto-associative memory
• Hopfield Model: binary neurons, well-connected network, bidirectional weighted connections, Hebbian learning

CAs and multi-associative memory:

• Psychologically, memories are not stored as individual concepts, but large collections of associated concepts that have many to many connections
• repeated co-activation of multiple CAs result in the formation of multiple and sequential associations, and sometimes new CAs

Multi-associative memory models:

• Non-Holographic Associative Memory (1969): well-connected network that can learn to map input bit patterns to output bit patterns; input CAs are connected to output CAs via learned one way associations
• The Linear Associator (Kohonen, 1977): feed-forward, well connected network;
• Multi Modular Associative Memory (1999): well connected modules, resilient to corrupted input
• Valiant model (2005): random graphs, biologically implausible learning, theoretical model of memorisation and association based on four quantitative parameters associated with the cortex:
  • the number of neurons per concept
  • number of synapses per neuron
  • synaptic strengths
  • number of neurons in total
• Interactive activation model (1981): each concept is represented by a node, and connections are made between nodes to show how closely related these are; not well connected
• Finally:
  • simulated neural systems can encode multi-associative memories
  • well connected systems are not a good model of the brain
  • use partitioning the system into modules, and sparsely connected random graphs
  • there models do not account for some human characteristics, e.g. context effects

Computation model for simulation:

• fLIF neural network:

- fLIF neurons collect activation from pre-synaptic neurons and fire on surpassing a threshold T
- on firing, a neuron loses its activation level, otherwise the activation leaks gradually:
Ait = Ait-1/d + Sum( Wij * Sj ).
d - decay factor.
- firing is a binary event, and activation of Wij is sent to all neurons j to which the firing neuron i has a connection.
- fatiguing causes the threshold to be dynamic:
t+1 = Tt + Ft.
- Ft is positive (F+) if the neuron fires at t and negative (F-) otherwise

• Network architecture:

- network is a whole or split into several subnetworks (for some simulations)
- intra-subnet synapses are biologically inspired distance biased connections (most likely excitatory connections to neighbouring neurons)
- subnet is a rectangular array of neurons with distance organized toroidally
- inhibitory connections within a subnet and all inter-subnet connections are set up randomly
- connectivity rule for excitatory neurons; connection i->j exists if Cij=1:
Cij = 1; if r < 1/(d*v)
r - random between 0 and 1
d - the neuronal distance (value=5 works well for all simulations)
v - the connection probability
- long distance intra-network connections are inspired by biological long distance axons with many synapses
- networks are divided into multiple CAs in response to stimuli using unsupervised learning algorithms
- the CAs are orthogonal and represent different concepts, and this is based on training