Fast Iterative Kernel PCA
N. N. Schraudolph, S. Günter, and S. Vishwanathan. Fast Iterative Kernel PCA. In Advances in Neural Information Processing Systems (NIPS), pp. 1225–1232, MIT Press, Cambridge, MA, 2007.
Download
560.4kB | 133.0kB | 1.5MB |
Abstract
We introduce two methods to improve convergence of the Kernel Hebbian Algorithm (KHA) for iterative kernel PCA. KHA has a scalar gain parameter which is either held constant or decreased as 1/t, leading to slow convergence. Our KHA/et algorithm accelerates KHA by incorporating the reciprocal of the current estimated eigenvalues as a gain vector. We then derive and apply Stochastic Meta-Descent (SMD) to KHA/et; this further speeds convergence by performing gain adaptation in RKHS. Experimental results for kernel PCA and spectral clustering of USPS digits as well as motion capture and image de-noising problems confirm that our methods converge substantially faster than conventional KHA.
BibTeX Entry
@inproceedings{SchGueVis07, author = {Nicol N. Schraudolph and Simon G\"unter and S.~V.~N. Vishwanathan}, title = {\href{http://nic.schraudolph.org/pubs/SchGueVis07.pdf}{ Fast Iterative Kernel {PCA}}}, pages = {1225--1232}, editor = {Bernhard Sch\"olkopf and John C. Platt and Thomas Hofmann}, booktitle = nips, volume = 19, publisher = {MIT Press}, address = {Cambridge, MA}, year = 2007, b2h_type = {Top Conferences}, b2h_topic = {>Stochastic Meta-Descent, Kernel Methods, Unsupervised Learning}, abstract = { We introduce two methods to improve convergence of the Kernel Hebbian Algorithm (KHA) for iterative kernel PCA. KHA has a scalar gain parameter which is either held constant or decreased as 1/t, leading to slow convergence. Our KHA/et algorithm accelerates KHA by incorporating the reciprocal of the current estimated eigenvalues as a gain vector. We then derive and apply Stochastic Meta-Descent (SMD) to KHA/et; this further speeds convergence by performing gain adaptation in RKHS. Experimental results for kernel PCA and spectral clustering of USPS digits as well as motion capture and image de-noising problems confirm that our methods converge substantially faster than conventional KHA. }}