Neil
E. Cotter
Associate Professor
(Lecturer)
Electrical and Computer
Engineering Dept
University of Utah
Neural Networks
Biological Neurons
Membrane potentials
†Resting potential
†Nernst-Planck equation overview
†Diffusion currents
†Drift currents
†Nernst-Planck equation
†Goldman-Hodgkin-Katz equation
Cable theory
†Membrane I-V curve
†Linear membrane model
†Transmission line equation
Impedance ladders
†Parallel resistors
†Laplace transformed impedances
†Thevenin equivalents
†Superposition
†Infinite resistor ladders
†Termination
†Synapse
Action potentials
†Hodgkin-Huxley model
†Propagation
†Models summary
FitzHugh-Nagumo Neuron
†Equations
†Nullclines
†Limit cycles
†Summary
McCulloch-Pitts model
Perceptrons
†Analogy to biological neuron
†Limitations of perceptron: Minsky Papert
†Universal approximation
Sigmoid Networks
Logistic neuron
†Neuron
†Logistic function
†Squashing function derivative
Backward Error Propagation
Example 1: y=x1+x2 (pdf)
Example 3: y=x1^2 + x2^2 (pdf)
Polynomial/Sigma-Pi Networks
†Definitions
†Ill-conditioned polynomial fits
CMAC
†Definition
†Optimal weights
†Variable step size learning
†CMAC and Kolmogorov networks
Self-organizing maps: Kohonen
†Definition
†Learning
†Physical layout of neurons
*†Phase series:
Transforming to phases
Fixed weight network
Pulse-Width Modulated Net
†PWM spectra
†Convergence
Boltzmann Machine
†Architecture
†Energy minimization
†Energy and synaptic weights
†High order statistics
†Learning
†Travelling salesman problem
Adaptive Resonance: Grossberg
Hopfield network
†Applications
†Neuron
†Network
†Binary
†Continuous
†Linearization
Lyapunov functions
†Binary network
†Continuous network
†Outer product learning
†Unlearning
†Optimization
†Memory capacity
†Deconvolution example
Adaptive Control Networks
Linear Feedback Error: Kawato
Dynamic backprop: Narendra
Recurrent backprop: Williams Zipser
Backprop thru time: Werbos
Adaptive critic: Barto et al
Temporal differences: Sutton
Learning as dynamic system
Supervised Learning
†Backprop thru time
†Temporal differences
†Learning as dynamic system
VCON
Overview
†Integrate and fire model
†Voltage controlled oscillator neuron (VCON)
†Voltage controlled oscillators
†Phase locked loops
VCON
Overview
†Phase locked loops
†Synapse
†Phase locking
†Neuron
†Free running behavior
External stimulation
†Pulse generation
†Phase resetting
averaging
†Bogoliuboff’s theorem
†RLC example
VCON phase locking
†Equations
†Change of variables
†Periodicity
†Subtleties
†Relationship to Hopfield network
*†Pulsed Network:
Neuron
Decision regions
Learning
Universal Approximation
