Fast Iterative Kernel PCA

N. N. Schraudolph, S. Günter, and S. Vishwanathan. Fast Iterative Kernel PCA. In Advances in Neural Information Processing Systems (NIPS), pp. 1225–1232, MIT Press, Cambridge, MA, 2007.

Download

pdf djvu ps.gz
560.4kB   133.0kB   1.5MB  

Abstract

We introduce two methods to improve convergence of the Kernel Hebbian Algorithm (KHA) for iterative kernel PCA. KHA has a scalar gain parameter which is either held constant or decreased as 1/t, leading to slow convergence. Our KHA/et algorithm accelerates KHA by incorporating the reciprocal of the current estimated eigenvalues as a gain vector. We then derive and apply Stochastic Meta-Descent (SMD) to KHA/et; this further speeds convergence by performing gain adaptation in RKHS. Experimental results for kernel PCA and spectral clustering of USPS digits as well as motion capture and image de-noising problems confirm that our methods converge substantially faster than conventional KHA.

BibTeX Entry

@inproceedings{SchGueVis07,
     author = {Nicol N. Schraudolph and Simon G\"unter and
               S.~V.~N. Vishwanathan},
      title = {\href{http://nic.schraudolph.org/pubs/SchGueVis07.pdf}{
               Fast Iterative Kernel {PCA}}},
      pages = {1225--1232},
     editor = {Bernhard Sch\"olkopf and John C. Platt and Thomas Hofmann},
  booktitle =  nips,
     volume =  19,
  publisher = {MIT Press},
    address = {Cambridge, MA},
       year =  2007,
   b2h_type = {Top Conferences},
  b2h_topic = {>Stochastic Meta-Descent, Kernel Methods, Unsupervised Learning},
   abstract = {
    We introduce two methods to improve convergence of the Kernel Hebbian
    Algorithm (KHA) for iterative kernel PCA. KHA has a scalar gain parameter
    which is either held constant or decreased as 1/t, leading to slow
    convergence.  Our KHA/et algorithm accelerates KHA by incorporating the
    reciprocal of the current estimated eigenvalues as a gain vector. We then
    derive and apply Stochastic Meta-Descent (SMD) to KHA/et; this further
    speeds convergence by performing gain adaptation in RKHS. Experimental
    results for kernel PCA and spectral clustering of USPS digits as well as
    motion capture and image de-noising problems confirm that our methods
    converge substantially faster than conventional KHA. 
}}

Generated by bib2html.pl (written by Patrick Riley) on Thu Sep 25, 2014 12:00:33