Learning to Evaluate Go Positions via Temporal Difference Methods

N. N. Schraudolph, P. Dayan, and T. J. Sejnowski. Learning to Evaluate Go Positions via Temporal Difference Methods. In N. Baba and L. C. Jain, editors, Computational Intelligence in Games, Studies in Fuzziness and Soft Computing, pp. 77–98, Springer Verlag, Berlin, 2001.
Earlier version

Download

pdf djvu ps.gz
186.0kB   139.9kB   91.8kB  

Abstract

The game of Go has a high branching factor that defeats the tree search approach used in computer chess, and long-range spatiotemporal interactions that make position evaluation extremely difficult. Development of conventional Go programs is hampered by their knowledge-intensive nature. We demonstrate a viable alternative by training neural networks to evaluate Go positions via temporal difference (TD) learning. Our approach is based on neural network architectures that reflect the spatial organization of both input and reinforcement signals on the Go board, and training protocols that provide exposure to competent (though unlabelled) play. These techniques yield far better performance than undifferentiated networks trained by self-play alone. A network with less than 500 weights learned within 3000 games of 9x9 Go a position evaluation function superior to that of a commercial Go program.

BibTeX Entry

@incollection{SchDaySej01,
     author = {Nicol N. Schraudolph and Peter Dayan
               and Terrence J. Sejnowski},
      title = {\href{http://nic.schraudolph.org/pubs/SchDaySej01.pdf}{
               Learning to Evaluate Go Positions
               via Temporal Difference Methods}},
    chapter =  4,
      pages = {77--98},
     editor = {Norio Baba and Lakhmi C. Jain},
  booktitle = {Computational Intelligence in Games},
  publisher = {\href{http://www.springer.de/}{Springer Verlag}, Berlin},
     series = {Studies in Fuzziness and Soft Computing},
     volume =  62,
       year =  2001,
   b2h_type = {Book Chapters},
  b2h_topic = {Reinforcement Learning},
   b2h_note = {<a href="b2hd-SchDaySej94.html">Earlier version</a>},
   abstract = {
    The game of Go has a high branching factor that defeats the tree search
    approach used in computer chess, and long-range spatiotemporal
    interactions that make position evaluation extremely difficult.
    Development of conventional Go programs is hampered by their
    knowledge-intensive nature.  We demonstrate a viable alternative
    by training neural networks to evaluate Go positions via temporal
    difference (TD) learning.
    Our approach is based on neural network architectures that reflect the
    spatial organization of both input and reinforcement signals on the Go
    board, and training protocols that provide exposure to competent (though
    unlabelled) play.  These techniques yield far better performance than
    undifferentiated networks trained by self-play alone.  A network with
    less than 500 weights learned within 3\,000 games of 9x9 Go a position
    evaluation function superior to that of a commercial Go program.
}}

Generated by bib2html.pl (written by Patrick Riley) on Thu Sep 25, 2014 12:00:33