Tutorial Sessions

June 1 (Sunday)

[ Return to top ]

 Information Theoretic Learning and Kernel Methods

Yiwen Wang*
University of Florida, USA

Entropy and divergence can be effectively used for training nonlinear or linear systems when Renyi's definition of entropy is integrated with Parzen estimation. Besides providing a set of alternative cost functions for learning, ITL has a strong connection to kernel methods, and has a very nice interpretation as statistics in feature space. It is also possible to extend the fundamental definition of similarity in feature space that we called correntropy. Therefore ITL provides a bases for a set of advanced signal processing and machine learning algorithms that will be presented and validated in real world problems. Time permitting a new set of on-line learning algorithms in kernel spaces will be also introduced and contrasted with the traditional way of adapting neural networks in the input space.

* A substitute of Jose C. Principe.

Yiwen Wang received a B.S. in Engineering Science with a minor in Automatic Control from University of Science and Technology of China (USTC, Hefei, Anhui, China) in 2001. In 2004, she received a Master degree in Engineering Science with a minor in Pattern Recognition and Intelligent System from University of Science and Technology of China (USTC, Hefei, Anhui, China). Right then, she joined the Department of
Electrical and Computer Engineering at the University of Florida (Gainesville, FL, USA), and received the PhD degree in 2008. Under the guidance of Dr. Jose C. Principe in the Computational NeuroEngineering Lab, she has investigated the application of advanced signal processing and control methods to neural data for Brain Machine Interfaces (BMIs).  Her research interests are in Brain Machine Interfaces, Statistical modeling on Biomedical Signals, Adaptive Signal Processing, Pattern Recognition, and Information Theoretic Learning.

[ Return to top ]

The Role of Fuzzy Sets and Systems in Science

Rudolf Seising
Medical University of Vienna, Austria

In science we have a traditional division of work: on the one hand we have fundamental, logical and theoretical investigations and on the other hand we have experimental and application side examinations. The theoretical work in science is using logics and mathematics to formulate axioms and laws. It is linked with the philosophical view of rationalism whereas the other aspects of science using experiments to find or prove or refute natural laws have their roots in the philosophical empiricism.

In both directions - from experimental results to theoretical laws or from theoretical laws to experimental proves or refutations - scientists have to bridge the gap that separates theory and practice in science.

Beginning as early as the 17th century, a primary quality factor in scientific work has been a maximal level of exactness. Galileo and Descartes started the process of giving modern science its exactness through the use of the tools of logic and mathematics.

The language of mathematics has served as a basis for the definition of theorems, axioms, definitions, and proofs. The works of Newton, Leibniz, Laplace and many others led to the ascendancy of modern science, fostering the impression that scientists were able to represent all the facts and processes that people observe in the world, completely and exactly. But this optimism has gradually begun to seem somewhat naïve in view of the discrepancies between the exactness of theories and what scientists observe in the real world.

From the empiricist point of view the source of our knowledge is sense experience. John Locke used the analogy of the mind of a newborn as a "tabula rasa" that will be written by the sensual perceptions the baby has later. In Locke's opinion this perceptions provide information about the physical world. Locke's view is called "material empiricism" whereas the so called idealistic empiricism was hold by Berkeley and Hume: there exists no material world, only the perceptions are real.

This epistemological dispute is of great interest for historians of science but it is ongoing till this day and therefore it is of great interest for today's philosophers of science, too. Searching a bridge over the gap between rationalism and empiricism is a slow-burning stove in the history of philosophy of science. Lotfi Zadeh's hierarchy stack of methodologies, fuzzy sets and systems (FSS), computing with words (CW) and the computational theory of perception (CTP), is recommended to build a bridge over this gap.

In this tutorial we 1) will give a presentation of the history of Zadeh's theory of fuzzy sets and systems in 1964/65 to bridge the gap that reflects the fundamental inadequacy of conventional mathematics to cope with the analysis of complex systems. 2) we will embed this new scientific theory into the network of theories of modern science. To this end we also regard the methodologies Zadeh erected on the basic methodology of FSS in the last decade of the 20th century: computing with words (CW) and the computational theory of perceptions (CTP).

Rudolf Seising, born 1961 in Duisburg, Germany, received an MS degree in mathematics from the Ruhr University of Bochum in 1986, a PhD degree in philosophy of science from the Ludwig Maximilians University (LMU) of Munich in 1995, and a postdoctoral lecture qualification (PD) in history of science from the LMU of Munich in 2004 for his book on the history of the theory of fuzzy sets (published in English in 2007: The Fuzzification of Systems, and in German in 2005: Die Fuzzifzierung der Systeme).

He has been scientific assistant for computer sciences at the University of the Armed Forces in Munich from 1988 to 1995 and scientific assistant for history of sciences at the same university from 1995 to 2002. Since 2002, he is scientific assistant in the Core unit for medical statistics and informatics at the University of Vienna Medical School, which in 2004 became the Medical University of Vienna. He has been visiting scholar at the University of California at Berkeley in 2000, 2001 and 2002. Rudolf Seising is Chairman of the IFSA Special Interest Group on History of Fuzzy Sets - IFSA SIG History and of the EUSFLAT Working Group "Philosophical Foundations".

Rudolf Seising teaches the philosophy and history of science, technology, and medicine, medical computer science, and especially the history of the theory of fuzzy sets. His research interests include the philosophy and history of artificial intelligence. Rudolf Seising is the author of the book The Fuzzification of Systems. The Genesis of Fuzzy Set Theory and its Initial Applications - Developments up to the 1970s
(Studies in Fuzziness and Soft Computing, Vol. 216), Springer-Verlag 2007. He will be the editor of the book Fuzzy Set Theory − Philosophy, Logics, and Criticism (Studies in Fuzziness and Soft Computing) Berlin [u.a.]: Springer that will appear in 2008.

He will be guest editor of the special issue on "Quantum Systems - Fuzzy Systems" of the International Journal of General Systems that will appear in 2008.1) will give a presentation of the history of Zadeh's theory of fuzzy sets and systems in 1964/65 to bridge the gap that reflects the fundamental inadequacy of conventional mathematics to cope with the analysis of complex systems. 2) we will embed this new scientific theory into the network of theories of modern science. To this end we also regard the methodologies Zadeh erected on the basic methodology of FSS in the last decade of the 20th century: computing with words (CW) and the computational theory of perceptions (CTP).

[ Return to top ]

Evolutionary Computing: Where We've Been and Where We're Going

 Yuhui Shi
Xi'an Jiao Tong-Liverpool University, China
Russell C. Eberhart
Indiana University Purdue University Indianapolis, USA

This tutorial is aimed at scientists and engineers, as well as faculty and graduate students, who would like to become more familiar with the methodologies of, and recent developments in, the field of evolutionary computing. It is anticipated that many of the participants will be attending WCCI as researchers and developers who focus their activities in the neural network and/or fuzzy logic areas.

The tutorial begins with a review of each of the five main areas of evolutionary computation: genetic algorithms, evolutionary programming, evolution strategies, genetic programming, and swarm intelligence. Concepts, paradigms, algorithms, and implementations of each of the five areas are presented. Examples of practical applications of each area are reviewed.

A canonical version of software (freeware or shareware) for each area will be briefly described, and made available to tutorial participants.

The tutorial concludes with a brief outline of recent developments in each field, and a summary of challenges currently facing researchers and developers in the evolutionary computation field.

 

Yuhui Shi is a Professor of Electrical and Electronic Engineering at Xi'an Jiaotong-Liverpool University (XJTLU), Suzhou, China. He also holds the position of the Director of the Research and Postgraduate Office at XJTLU. Before joining XJTLU, he was with the Electronic Data Systems Corporation (EDS), Indiana, USA, as an Applied Specialist for 9 years. He is an adjunct professor of Indiana University Purdue University Indianapolis, Indiana, USA, and Southeast University, Nanjing, China. He has eight years' experience primarily in the system design and development in automotive industry, and over 18 years experience in algorithm design and implementation. He has extensive knowledge on innovation and creative problem-solving skills. Dr. Shi is an Associate Editor of the IEEE Transactions on Evolutionary Computation, and a member of the editorial board of the Journal of Swarm Intelligence, and the Chair of the IEEE CIS Task Force on Swarm Intelligence. He co-authored a book on Swarm Intelligence together with Dr. James Kennedy and Prof. Russell Eberhart, and another book on Computational Intelligence: Concepts to Implementations together with Prof. Russell Eberhart.

Russell C. Eberhart is Professor of Electrical and Computer Engineering at the Purdue School of Engineering and Technology, Indiana University Purdue University Indianapolis (IUPUI). He is also Vice President and Chief Technology Officer of Computelligence LLC, Indianapolis, Indiana. He received his Ph.D. from Kansas State University in electrical engineering. He is co-editor of a book on neural networks, and co-author of Computational Intelligence PC Tools, published in 1996 by Academic Press. He is co-author of a book with Jim Kennedy and Yuhui Shi entitled Swarm Intelligence, published by Morgan Kaufmann/Academic Press in April 2001. He was awarded the IEEE Third Millenium Medal. In 2001, he became a Fellow of the IEEE, and in 2002 he became a Fellow of the American Institute for Medical and Biological Engineering. He is the co-author, with Yuhui Shi, of a book entitled Computational Intelligence: Concepts to Implementations, published by Morgan Kaufmann/Elsevier in 2007. His areas of research include swarm intelligence and extended analog computing, and the detection of sleepy and inattentive driving.

[ Return to top ]

Non-Standard Learning Methods for Sparse High-Dimensional Data

Vladimir Cherkassky
University of Minnesota, USA

The field of Predictive Learning is concerned with estimating 'good' predictive models from available data. Such problems can be usually stated in the framework of inductive learning, where the goal is to come up with a good predictive model from known observations (or training data samples). In recent years, there has been a growing interest in applying learning methods to sparse high-dimensional data (i.e., in genomics, medical imaging, object recognition, etc.). In such applications, many successful approaches represent minor modifications of existing inductive learning methods (such as neural networks, support vector machines, discriminant analysis etc.) combined with clever preprocessing and feature extraction. At the same time, in the statistical learning community, there is a trend towards development and better understanding of new non-standard learning settings. Examples include (a) several new learning formulations developed in VC-theory: transduction, learning through contradictions, and SVM+ (Vapnik, 1998, 2006); and (b) non-standard settings proposed in machine learning community, such as Multi-Task Learning (Ben-David et al, 2002), Semi-Supervised Learning (Chapelle et al, 2006) etc. These new learning formulations are motivated by practical needs (to improve generalization for learning with sparse high-dimensional data). This tutorial will present an overview of recent non-standard learning formulations, investigate possible connections between these formulations, and discuss application examples illustrating advantages of using these new approaches for sparse high-dimensional data. The presentation will be based, to a large extent, on the conceptual framework developed by Vapnik [1998, 2006].

Vladimir Cherkassky is Professor of Electrical and Computer Engineering at the University of Minnesota.  He received Ph.D. in Electrical Engineering from University of Texas at Austin in 1985. His current research is on methods for predictive learning from data, and he has co-authored a monograph Learning From Data published by Wiley in 1998. Prof. Cherkassky has served on the Governing Board of INNS. He has served on editorial boards of IEEE Transactions on Neural Networks, the Neural Networks Journal, the Natural Computing Journal and the Neural Processing Letters. He served on the program committee of major international conferences on Artificial Neural Networks. He was Director of NATO Advanced Study Institute (ASI) From Statistics to Neural Networks: Theory and Pattern Recognition Applications held in France, in 1993. He presented numerous tutorials on neural network and statistical methods for learning from data. In 2007, he was elected as Fellow of IEEE for 'contribution and leadership in statistical learning and neural networks'.

[ Return to top ]

 Hippocampus, Spatial Navigation, Memory and Dynamics: From Neurophysiological Observations to Computational Models

Colin Molter
RIKEN Brain Science Institute, Japan

The hippocampus' beautiful neuronal architecture and its importance in spatial navigation and in memory formation and consolidation have made it a central topic of research during the past 40 years. The many recent exciting discoveries and development in experimental and in computational studies, are bringing keys to break the neural code of this central brain area. This tutorial aims to provide the current state of the art of our understanding of how the hippocampus manages to perform these two functional roles; memory formation and spatial navigation. To this aim, a state of the art of many neurophysiological evidences will be given. First the very particular type of cells constituting this network (from the place cells discovered 30 years ago to the grid cells discovered two years ago), then the very particular type of dynamics encountered will be reviewed. Different computational models have been proposed for understanding the hippocampal dynamics and their possible functional role in sequence learning, object-place learning, spatial navigation, etc. Selected models will be reviewed and compared. We expect these models to be insightful for researchers in the field of artificial intelligence and neural information processing.

With several peer reviewed papers and awards, Dr. Colin Molter is now an expert in computational neuroscience and more precisely on the fields related to the hippocampus. After studying physics engineering and philosophy, and after completing a PhD in artificial intelligence at the university of Brussels, Dr. Molter joined the laboratory for dynamics of emergent intelligence in 2005.

Rapidly he developed several computational models of the hippocampal formation network, all stressing the important role played by dynamics in information processing. Among others, he simulated the online memory formation of behavioral episodes, the reactivation of specific patterns (such as the reverse replay) and the formation of place cells from the conjunctive activity of grid cells. His models have leaded to collaboration with experimental laboratory for the validation of his predictions. Recently, following the flow of information sent by the hippocampus, he is interested in the exchange of information with the prefrontal cortex and with the ability of this brain area to perform long term storage and to be involved in working memory tasks.

[ Return to top ]

 Evolutionary Multi-criterion Optimization (EMO): Fundamentals, State-of-the-art Methodologies and Future Challenges

Kalyanmoy Deb
Indian Institute of Technology Kanpur, India

Many real-world search and optimization problems are naturally posed as non-linear programming problems having multiple conflicting objectives. Due to lack of suitable solution techniques, such problems are usually artificially converted into a single-objective problem and solved. The difficulty arises because multi-objective optimization problems give rise to a set of Pareto-optimal solutions, each corresponding to a certain trade-off among the objectives. It then becomes important to find not just one Pareto-optimal solution but as many of them as possible. Classical a posteriori MCDM methods are found to be not efficient because they require repetitive applications to find multiple Pareto-optimal solutions and in some occasions repetitive applications do not guarantee finding distinct Pareto-optimal solutions. The population approach of evolutionary algorithms (EAs) allows an efficient way to find multiple Pareto-optimal solutions simultaneously in a single simulation run.

In this tutorial, we shall contrast the differences in philosophies between classical and evolutionary multi-objective methodologies and provide adequate fundamentals needed to understand and use both methodologies in practice. Particularly, major state-of-the-art evolutionary multi-objective optimization (EMO) methodologies will be discussed in detail in the context of their computer implementations. Thereafter, three main application areas of EMO will be discussed with adequate case studies from practice -- (i) applications showing better decision-making abilities through EMO, (ii) applications exploiting the multitude of trade-off solutions of EMO in extracting useful information in a problem, and (iii) applications showing better problem-solving abilities in various other tasks (such as, reducing bloating, solving single-objective constraint handling, and others).

Clearly, EAs have a niche in solving multi-objective optimization problems compared to classical methods. This is why EMO methodologies are getting a growing attention in the recent past. Since this is a comparatively new field of research, in this tutorial, a number of future challenges in the research and application of multi-objective optimization will also be discussed.

This tutorial is aimed for both novices and users of EMO. Those without any knowledge in EMO will have adequate ideas of the procedures and their importance in computing and problem-solving tasks. Those who have been practicing EMO will also have enough ideas and materials for future research, know state-of-the-art results and techniques, and make a comparative evaluation of their research.

Besides the CEC participants, ANN and Fuzzy logic computing researchers often come across multiple objectives in their problem solving tasks and would find this tutorial useful. Some application studies which will be presented in this tutorial uses ANN and fuzzy logic ideas and thus this tutorial will be an ideal topic for discussion for the WCCI-2008 conference.
 

Kalyanmoy Deb holds Deva Raj Chair Professor at Indian Institute of Technology Kanpur in India. He is the recipient of the prestigious MCDM Edgeworth-Pareto award by the Multiple Criterion Decision Making (MCDM) Society, one of the highest awards given in the field of multi-criterion optimization and decision making. He has also received prestigious Shanti Swarup Bhatnagar Prize in Engineering Sciences for the year 2005 from Govt. of India. He has also received the `Thomson Citation Laureate Award' from Thompson Scientific for having highest number of citations in Computer Science during the past ten years in India. He is a fellow of Indian National Academy of Engineering (INAE), Indian National Academy of Sciences, and International Society of Genetic and Evolutionary Computation (ISGEC). He has received Fredrick Wilhelm Bessel Research award from Alexander von Humboldt Foundation in 2003. His main research interests are in the area of computational optimization, modeling and design, and evolutionary algorithms. He has written two text books on optimization and more than 200 international journal and conference research papers. He has pioneered and a leader in the field of evolutionary multi-objective optimization. He is associate editor of two major international journals and an editorial board members of five major journals. More information about his research can be found from http://www.iitk.ac.in/kangal/deb.htm.

[ Return to top ]

 Nonlinear Dimensionality Reduction and Data Visualization

Hujun Yin
University of Manchester, UK

Dimension reduction has long been associated with retinotopic mapping for understanding cortical maps and neural information processing. Multisensory information is perceived, propagated and mapped onto the 2-D cortex in a near-optimal information preserving manner. Data visualization, inspired by this mechanism, is playing an increasingly important role in many practical applications involving feature and data reduction, from biology, neuroscience, decision support, social science, to management science. The topic has also attracted a great deal of attention in computer vision and pattern recognition recently. Classic linear methods include principal component analysis (PCA), factor analysis, projection pursuit and independent component analysis. Recently there have been considerable efforts and advances in developing methodologies and techniques for nonlinear dimensionality reduction. A number of novel projection methods have been proposed from statistics, geometry theory and neural networks. Two fundamental approaches are multidimensional scaling and nonlinear PCA. This tutorial will provide an introduction to this challenging and demanding topic. Various recent methods along these lines such as, self-organizing maps, kernel PCA, principal manifold, metric and non-metric scaling, isomap, local linear embedding, Laplacian eigenmap, as well as spectral clustering will be explained and discussed. It will also attempt to unify these methods under a constrained self-organizing framework. A number of examples and real-world applications will be shown to illustrate the usefulness and strengthen of various methods, as well their weakness and limitation.
 

Hujun Yin is a Senior Lecturer at The University of Manchester, School of Electrical and Electronic Engineering. He received BEng and MSc degrees from Southeast University and PhD degree from University of York in 1983, 1986 and 1996 respectively. His research interests include neural networks, self-organising systems in particular, pattern recognition, image processing, and bio-/neuro-informatics. He has studied, extended and applied the self-organising map (SOM) and related topics (principal manifolds and data visualisation) extensively in the past ten years and proposed a number of extensions including Bayesian SOM and ViSOM. He has served on the Programme Committee for more than twenty international conferences. He has been the Organising Chair, Programme Committee Chair, and General Chair for a number of conferences, such as 2001 International Workshop on Self-Organising Maps (WSOM'01), International Conference on Intelligent Data Engineering and Automated Learning (IDEAL) (2002-2007), 2006 International Symposium on Neural Networks (ISNN'06). He sits on the Steering Committee of the WSOM series. He was a guest editor of Neural Networks: 2002 Special Issue on New Developments in Self-Organising Maps, among other special issues on other international journals. He has received research funding from the EPSRC, BBSRC and DTI. He has published more than 100 peer-reviewed articles. He is a senior member of the IEEE and a member of the EPSRC College. He has also been a regular assessor of the EPSRC, BBSRC, Royal Society, Hong Kong Research Grant Council, Netherlands Organisation for Scientific Research, and Slovakia Research and Development Council. He is an Associate Editor of the IEEE Transactions on Neural Networks and a member of the Editorial Board of the International Journal of Neural Systems.

[ Return to top ]

Learning to Play Games

Simon M. Lucas
University of Essex, UK
 Julian Togelius
Dalle Molle Institute for Artificial Intelligence, Switzerland
Thomas P. Runarsson
University of Iceland, Iceland

This tutorial provides a practical introduction to game strategy learning with function approximation architectures. The tutorial will cover the two main approaches to learning game strategy: evolution (including co-evolution), and temporal difference learning, and also discuss some ways of hybridizing these.

We also look at how the choice of input features and function approximation architecture has a critical impact on what is learned, as well as how it is interfaced to the game (e.g. as a value estimator or as an action selector). Incremental and co-evolutionary methods of learning complex skills are described. In addition to standard MLPs, attention is also given to N-Tuple systems, as these have recently shown great potential to learn quickly and effectively, and to evolutionary methods for selecting subsets of the input vector to use and neural network topologies to process it with.

Each method will be demonstrated with reference to some simple fragments of software, illustrating how the learning algorithm is connected with the game and with the function approximation architecture. Example games will include Othello, Simulated Car Racing, and Ms. Pac-Man.

Simon M. Lucas (SMIEEE) is a reader in computer science at the University of Essex (UK). His main research interests are evolutionary computation, games, and pattern recognition, and he has published widely in these fields with over 120 peer-reviewed papers, mostly in leading international conferences and journals. He was chair of IAPR Technical Committee 5 on Benchmarking and Software (2002 - 2006) and is the inventor of the scanning n-tuple classifier, a fast and accurate OCR method. He was appointed inaugural chair of the IEEE CIS Games Technical Committee in July 2006, has been competitions chair for many international conferences, and co-chaired the first IEEE Symposium on Computational Intelligence and Games in 2005. He was program chair for IEEE CEC 2006, and program co-chair for IEEE CIG 2007, and will be program co-chair for PPSN 2008. He is an associated editor of IEEE Transactions on Evolutionary Computation, and the Journal of Memetic Computing. He was an invited keynote speaker at IEEE CEC 2007.

Thomas P. Runarsson is a professor of engineering with the department of Mechanical and Industrial Engineering, Faculty of Engineering, at the University of Iceland. He is an associated editor for the IEEE Transactions on Evolutionary Computation, a member of the IEEE Computational Intelligence Society, director of the Applied Mathematics group at the Science Institute, and a member of the board for the Icelandic Operations Research Society (ICORS). His research interests include evolutionary computation, constraint handling, global optimization, approximated dynamic programming, statistical learning, and real world applications. He was the general chair for PPSN-IX: The International Conference on Parallel Problem Solving From Nature held in Reykjavik, Iceland, 9-13 September 2006.

[ Return to top ]

 Towards Automated Parameter Calibration of Evolutionary Algorithms

A.E. Eiben
Free University Amsterdam, Netherlands

It is ironic that all EC researchers and practitioners know that choosing good EA parameter values is essential for good EA performance, but nobody really knows how to find such good values. Even after 3 decades of EA applications and much experimental research the defining attributes (e.g., the parent selection method) and the parameter values (e.g., the mutation rate) are still chosen in an ad hoc manner, often led by unverified conventions. In short: parameter calibration of EAs is still a challenge. This tutorial will outline the main problems here and hint towards solutions. Using the widely accepted taxonomy of Eiben, Hinterding, and Michalewicz we will review related work. We will show results indicating that the traditional emphasis on playing around with the parameters of variation operators may be suboptimal and more research should be directed towards other EA defining attributes and parameters. We will also consider the most promising approaches to calibrating EAs in a (semi) automated manner.
 

A.E. Eiben (Gusz) is one of the European EC pioneers with his first EC paper dating back to 1989. He is full professor of computational intelligence on the Free University Amsterdam, The Netherlands. He (co-)authored more than 100 research papers, (co-)edited various journal special issues and conference proceedings and Springer's best selling book Introduction to Evolutionary Computing. He is first author of the award winning seminal paper on EC parameters from 1999. He has participated in numerous EC research projects and teaches an EC course for more than 10 years.