by Neil E. Cotter
Research Assistant Professor
(† = missing, * = article)
Optimization
Lagrange Multipliers
Karush-Kuhn-Tucker Theorem
Descent Algorithms
Newton's method
Line search
Gradient (or steepest) descent
Conjugate gradient methods
Polak-Ribiere method
Modified Newton's method
Linear Optimization
Linear programming
Example Problem 1 (pdf)
Simplex algorithm
Interior-point algorithms
Quadratic Programming
Global Optimization
Downhill simplex algorithm
Genetic algorithms
†Diffusion algorithms
Random fields
Markov random fields (MRF)
Gibbs random fields (GRF)
Equivalence of MRF and GRF
Convergence
Notation
Kolmogorov-Chapman equation
Forgetting initial conditions
†Approaches invariant distribution
†Invariant distribution is Gibbs
†Freezing to global minima
Metropolis algorithm
Simulated annealing
*Prejudicial search
Discrete Optimization
Dynamic programming
NP-completene problems
Downhill simplex algorithm
Continuation
†Dynamic programming: Werbos
†Random optimization
Stochastic approx
CMAC learning
Gradient descent
†K-means squared