Unsupervised Discrimination of Clustered Data via Optimization of Binary Information Gain
N. N. Schraudolph and T.
J. Sejnowski. Unsupervised Discrimination of Clustered Data
via Optimization of Binary Information Gain. In
Advances in Neural Information Processing Systems (NIPS), pp. 499–506,
Morgan Kaufmann, San Mateo, CA, 1993.
In Ph.D.
thesis
Download
122.4kB | 101.2kB | 85.5kB |
Abstract
We present the information-theoretic derivation of a learning algorithm that clusters unlabelled data with linear discriminants. In contrast to methods that try to preserve information about the input patterns, we maximize the information gained from observing the output of robust binary discriminators implemented with sigmoid nodes. We derive a local weight adaptation rule via gradient ascent in this objective, demonstrate its dynamics on some simple data sets, relate our approach to previous work and suggest directions in which it may be extended.
BibTeX Entry
@inproceedings{SchSej93, author = {Nicol N. Schraudolph and Terrence J. Sejnowski}, title = {\href{http://nic.schraudolph.org/pubs/SchSej93.pdf}{ Unsupervised Discrimination of Clustered Data via Optimization of Binary Information Gain}}, pages = {499--506}, editor = {Stephen Jos{\'e} Hanson and Jack D. Cowan and C. Lee Giles}, booktitle = nips, publisher = {Morgan Kaufmann, San Mateo, CA}, volume = 5, year = 1993, b2h_type = {Top Conferences}, b2h_topic = {>Entropy Optimization}, b2h_note = {In <a href="b2hd-Schraudolph95">Ph.D. thesis</a>}, abstract = { We present the information-theoretic derivation of a learning algorithm that clusters unlabelled data with linear discriminants. In contrast to methods that try to preserve information about the input patterns, we maximize the information gained from observing the output of robust binary discriminators implemented with sigmoid nodes. We derive a local weight adaptation rule via gradient ascent in this objective, demonstrate its dynamics on some simple data sets, relate our approach to previous work and suggest directions in which it may be extended. }}