Lecture 8 CSCI 784 - Least Mean Square Algorithm

• Pragmatics:
  • One misterm Feb 26 vs Two midterms Feb 12 and Mar 31
  • From Last time: Correlation Matrix (handout forth coming)
  • Gaussian Classifiers
• Background:
  • simultaneous equations
  • Gaussian elimination
  • correlation coefficient: pho = E[(X-muX)(Y-muY)]/(sigmaX * sigmaY)
• Spatial Filters
  • collection of p sensors (fig 5.1)
  • calculate linear combination of signals
  • compare with known output to choose wieghts so as to minimize mean-square-error
• mean square error
• error surface
• Expansion of mean-square-error
• Notations
  • mean-square-value
  • cross-correlation
  • auto-correlation of inputs
• Find minimum J
  • take gradient of J with respect to w_k
  • set gradients = 0 for k = 1, 2, ... p
  • solve system of linear equations
  • A^-1, Gauss-Jordon
• Method of Steepest Descent
  • Use -1 * gradient as direction
  • figure 5.2
  • delta-w = ...
• Least-Mean-Square Algorithm
  uses instantaneous estimates
  • r_dx(k,n) = x_k(n) d(n)
  • r(j, k, n) = x_j(n) x_k(n)
• Simplification of delta-w(n)
• Summary of LMS (Table 5.1)
• Skip 5.5, 5.6
• Learning Rate Annealing Schedule
  • Choices for eta(n)
• Adaline ADAptive LINear Element fig 5.6

[Handouts:]

[Readings:]

[Assignment:]