Information Geometry of Multiple Neural Spike Trains

Shun-ichi Amari, RIKEN Brain Science Institute, Japan
Wednesday, Sep 3, 9.15 - 10.15

Abstract: Information is carried and processed by trains of spikes in an ensemble of neurons. Spikes are stochastic rather than deterministic, and they are correlated spatially and temporally. Information geometry deals with the invariant structure of a manifold of probability distributions and is useful for analyzing stochastic phenomena. The manifold is Riemannian, having a pair of dually coupled affine connections. We apply information geometry to analyze information included in multiple spike trains: We show how pairwise, triplewise and higher-order interactions are separated from firing rates orthogonally. It is also discussed that synchronous firing requires higher-order correlations by using a simple model. Then we show how temporal correlations are separated from firing rates orthogonally by using a simple Markov chain model. Finally we show a simple mixture model as an example of tractable stochastic models, and study its structure of correlations

Shun-ichi Amari Prof. Shun-ichi AMARI, after completing his professorship at The University of Tokyo, moved to The Institute of Physical and Chemical Research - RIKEN where he holds the position of vice-president of Brain Science Institute, director of Brain Style Information Systems Group and team leader of Mathematical Neuroscience Laboratory. He also serves on boards of numerous scientific journals and committees.

Quantum computing: targets, principles and obstacles

Mika Hirvensalo, Department of Mathematics, University of Turku, Finland
Wednesday, Sep 3, 14.00 - 15.00

Abstract: In this talk, we will first present some of the reasons which should drive, and drives researchers to be interested in quantum computing and quantum information. As the second issue, we discuss the fundamental elements of quantum computing not shared by classical algorithms, and as the concluding topic, the difficulties of quantum computing theory are presented. The main emphasize of the talk will be on the theory of quantum computing, whereas the questions related to the physical implementations are treated only shortly.

Mika Hirvensalo Mika Hirvensalo PhD. is interested in Quantum computing, Decision questions, Probabilistic models of computations. He is also author of "Quantum computing".

Bayesian surprise attracts human attention

Pierre Baldi, School of Information and Computer Sciences, University of California, USA
Thursday, Sep 4, 9.00 - 10.00

Abstract: The concept of information is central to science, technology, and other human endeavors. Shannon's theory of information, although eminently successful for the development of modern computer and telecommunication technologies, does not capture subjective and semantic aspects of information that are not related to its transmission but rather to the expectations of the observer. Here we propose a subjective definition of information we call surprise to quantify how data affects an observer by measuring the difference between the prior and posterior distributions of the observer. Surprise requires averaging over the space of models in contrast with Shannon entropy which requires averaging over the space of data. Surprise can be estimated efficiently and, during learning processes, decreases with the number of examples as 1/N. Scoring items by surprise provides a general principle for the detection of unusual events and the construction of saliency maps that can guide the deployment of attention and other rapid filtering mechanisms in natural or synthetic information processing systems. Applications to the development of neural network architectures for machine vision and the analysis of video data and human eye movements will be presented. Using this framework we measure the extent to which humans direct their gaze towards surprising items while watching television and video games. Humans are strongly attracted to locations of high Bayesian surprise, with 72% of all human gaze shifts directed towards locations more surprising than the average, a figure which rises to 84% when considering only gaze targets simultaneously selected by all subjects. The resulting theory of surprise is applicable across different spatio-temporal scales, modalities, and levels of abstraction.

Pierre Baldi Pierre Baldi recieved his PhD in 1986 on California Institute of Technology. His research interests contains following fields: AI, Machine Learning, Data Mining, Chemoinformatics, Bioinformatics, Systems Biology and Communication Networks. He is holder of Lew Allen Research Award, Laurel Wilkening Faculty Innovation Award and Microsoft Faculty Research Award. He is also fellow of AAAI (Association Advancement Artifical Intelligence).

Modeling Embodied Cognition and Emotion

Tom Ziemke, University of Skövde, Sweden
Thursday, Sep 4, 14.30 - 15.30

Abstract: This talk focuses on work in the European project ICEA - Integrating Cognition, Emotion and Autonomy (cf. www.iceaproject.eu). The twofold hypothesis behind the project is (1) that the emotional and bioregulatory mechanisms that come with the organismic embodiment of living cognitive systems also play a crucial role in the constitution of their "high-level" cognitive processes, and (2) that models of these mechanisms can be usefully integrated in artificial cognitive systems architectures, which will constitute a significant step towards more autonomous robotic cognitive systems capable of dealing with issues such as energy management, self- monitoring, self-repair, etc. A central question addressed in the ICEA project is how in embodied neurocomputational cognitive architectures lower-level (e.g. bodily, homeostatic) mechanisms can modulate cognitive processes at higher levels through emotional/affective mechanisms.

Tom Ziemke Tom Ziemke is Professor of Cognitive Science at the University of Skvde, Sweden. He received his doctorate from the University of Sheffield, UK. His main research interests are theories and robotic/neurocomputational models of situated and embodied cognition, in particular the various roles of the body in shaping cognitive processes. Among other things, he is coordinator of ICEA, a European cognitive systems project on bio-inspired cognitive/affective robotics (www.iceaproject.eu), one of the principal investigators of European project ROSSI on concept/language grounding and human-robot interaction (www.rossiproject.eu), editor of a recent two-volume book on embodiment ("Body, Language and Mind"), and associate editor of the journals "New Ideas in Psychology" and "Connection Science".

Are We There Yet?

Nello Cristianini, Departments of Engineering Mathematics and Computer Science, University of Bristol, UK
Friday, Sep 5, 9.00 - 10.00

Abstract: Exactly fifty years ago the conference "Mechanisation of Thought Processes" brought together the entire research community working on Machine Intelligence. Today its Proceedings provide us with a snapshot of where the field was 50 years ago, enabling us to take a historical perspective on the current state of the area. Many similarities, as well as important differences, emerge from the comparison. The great impact of statistical approaches on the quest for machine intelligence is discussed, as well as some of the risks and limitations connected to them.

Nello Cristianini Nello Cristianini is a Professor of Artificial Intelligence at the University of Bristol since March 2006, and a holder of the Royal Society Wolfson Merit Award. He has wide research interests in the area of computational pattern analysis and its application to problems ranging from genomics, to computational linguistics and artificial intelligence systems. He has contributed extensively to the field of kernel methods. Before the appointment to Bristol he has held faculty positions at the University of California, Davis, and visiting positions at the University of California, Berkeley, and in many other institutions. Before that he was a research assistant at Royal Holloway, University of London. He has also covered industrial positions. He has a PhD from the University of Bristol, a MSc from Royal Holloway, University of London, and a Degree in Physics from University of Trieste. Since 2001 has been Action Editor of the Journal of Machine Learning Research (JMLR), and since 2005 also Associate Editor of the Journal of Artificial Intelligence Research (JAIR). He is co-author of the books 'An Introduction to Support Vector Machines' and 'Kernel Methods for Pattern Analysis' with John Shawe-Taylor, and "Introduction to Computational Genomics" with Matt Hahn (all published by Cambridge University Press).

How to Learn a Program?

Jürgen Schmidhuber, Istituto Dalle Molle di Studi sull'Intelligenza Artificiale, Switzerland and TU München, Germany
Friday, Sep 5, 14.00 - 15.00

Abstract: We will discuss novel ways of making robots and other agents smarter through machine learning algorithms. The focus will be on sequence learning as opposed to conventional pattern recognition. We will outline very general, asymptotically optimal problem solvers pioneered in our lab, as well as practical applications based on state-of-the-art adaptive feedback neural networks, with examples ranging from challenging control tasks to handwriting recognition and music composition.

Jürgen Schmidhuber Jürgen Schmidhuber is Co-Director of the Swiss Institute for Artificial Intelligence IDSIA (since 1995), Professor of Cognitive Robotics at TU Munich (since 2004), Professor SUPSI (since 2003), and also adjunct Professor of Computer Science at the University of Lugano, Switzerland (since 2006). He obtained his doctoral degree in computer science from TUM in 1991 and his Habilitation degree in 1993, after a postdoctoral stay at the University of Colorado at Boulder, USA. He helped to transform IDSIA into one of the world's top ten AI labs (the smallest!), according to the ranking of Business Week Magazine (US). His research grants have yielded more than 200 peer-reviewed scientific papers on topics ranging from machine learning and mathematically optimal universal AI and artificial recurrent neural networks to adaptive robotics and complexity theory, digital physics, and the fine arts.

Abstracting Biological Development to Evolve Large-Scale Artificial Neural Networks

Kenneth O. Stanley, University of Central Florida School of Electrical Engineering and Computer Science, USA
Saturday , Sep 6, 9.00 - 10.00

Abstract: In his talk, Stanley will focus on recent work in indirectly encoding neural networks through an abstraction of biological development called Hypercube-based NeuroEvolution of Augmenting Topologies (HyperNEAT). The primary conceptual focus will be on the critical role of abstraction in biologically-inspired systems. In particular, because DNA encodes complexity on a massive scale, interest has grown in recent years in the powerful mapping between genotype and phenotype in nature. Thus, the question, what is the right level of abstraction for the development of the embryo in nature? The conclusion is that it is possible to capture the essential properties of developmental encoding at a much higher level of abstraction than previously thought. This new developmentally-motivated encoding, called Compositional Pattern Producing Networks (CPPNs), is then extended to represent large-scale neural networks by exploiting a surprisingly simple geometric trick. The result is a novel ability to evolve working networks with millions of connections that exploit geometric regularities in several problem domains. Finally, Stanley will conclude with thoughts on prospects for open-ended evolution and unbounded innovation.

Kenneth O. Stanley Kenneth O. Stanley is an Assistant Professor in the School of Computer Science at the University of Central Florida. He graduated with a B.S.E. in Computer Science Engineering and a minor in Cognitive Science from the University of Pennsylvania in 1997 and received a Ph.D. in 2004 at the University of Texas at Austin. He is the inventor of the Neuroevolution of Augmenting Topologies (NEAT) algorithm for evolving increasingly complex artificial neural networks. His main research contributions are in neuroevolution (i.e. evolving neural networks), generative and developmental systems, coevolution, machine learning for video games, and interactive evolution. He has won best paper awards for his work on NEAT (at the 2002 Genetic and Evolutionary Computation Conference) and for his work on NERO (at the IEEE 2005 Symposium on Computational Intelligence and Games), in addition to winning the Independent Games Festival Student Showcase Award and supervising the team that produced AAAI's 2007 Best Student Video Award winner.

The concept of analog network as a unifying principle for the evolutionary synthesis of artificial neural networks

Claudio Mattiussi, Laboratory of Intelligent Systems, Swiss Federal Institute of Technology in Lausanne
Saturday, Sep 6, 10.30 - 11.00 (Invited Workshop Speech)

Abstract: In this talk I will first introduce the concept of analog network. Then, I will show that this concept constitutes an interesting unifying principle for the analysis and synthesis of several kinds of networks of scientific and technological interest, such as artificial neural networks, electronic circuits, and biological networks. Focusing on artificial neural networks, I will illustrate how it is possible to use a few key elements of the analog network concept to describe a neural network and to genetically encode it in view of its evolutionary synthesis. I will then explain how this point of view helps to better understand and design both the familiar direct genetic encodings and the more recently introduced implicit genetic encodings. To illustrate the concept of implicit encoding I will describe the approach called Analog Genetic Encoding (AGE) that we have been developing at LIS since 2001. In particular, I will describe the AGE experiments of evolution of neuromodulatory structures and the AGE experiments of evolution of neural classifiers for biological signals. Finally, I will hint at the possibility of using the lessons drawn from the application of existing genetic encodings and the perspective provided by the concept of analog network to further increase the potential of evolutionary methods for the synthesis of artificial neural networks.

Claudio Mattiussi Claudio Mattiussi is Senior Researcher at the Laboratory of Intelligent Systems (LIS), Swiss Federal Institute of Technology in Lausanne (EPFL), Switzerland. He conducts research on evolutionary computation, neural networks, and machine learning. His research interests include bio-inspired artificial intelligence, systems biology, probabilistic engineering, evolutionary robotics, and the numerical formulation of physical field problems. With Dario Floreano, he recently co-authored the book "Bio-inspired Artificial Intelligence" published by MIT Press (2008).