Theory, programming and simulation of neural networks

Description:

Neuronal networks are an important subset of the methods of artificial intelligence. These methods have become increasingly important in the fields that with the more traditional methods of informatics are difficult to tackle, and therefore have been reserved for human intelligence. In addition to being able to replace and to support a human workforce, these methods also provide insight into the structure and methods of human reasoning.
The lectures are organized as follows. Introductory topics are:

• graphical methods and game theory (backtracking and constraint propagation)
• analytical optimization (multidimensional extremal problems, Lagrange multipliers, equilibria, gradient descent)

Focus topics are:

• neuronal networks (biological networks, close-to-biology modeling, spinsystem analogies)
• evolutionary optimization (genetic algorithms and programming)
• expert systems (clustering techniques)

Supplementary Material:

Mathematica Introduction

Hopfield Network Code

Project Proposals:

Genetic Algorithms:

(a) Implement a routine to using genetic algorithms for search/optimization problems as seen in chapter 6 of the script. The code should be written in Java or another language (MATLAB excluded).
(b) Test a variety of fitness and mutation functions.

The specific functions to optimize will be provided!

Neural Networks:

(a) Using an nth degree polynomial for the training data, determine an appropriate measure for the efficiency of the Neural Network regarding the number of neurons in the hidden layer, speed of convergence and accuracy.

Image Invariance:

(a) Implement the Fourier-Mellin (FM) transform and determine the effectiveness of using the FM invariant features in training a Neural Network.

Some source code can be provided!

Gradient Descent:

(a) Implement the gradient descent algorithm and the conjugate gradient descent algorithm. The code should be written in Java or another language (MATLAB excluded).

(b) Extend the above algorithms with a momentum term.

Run your algorithms on various functions and try to find functions which cause each algorithm to perform poorly.

Fuzzy Logic and Neural Networks:

(a) Read and implement the methiods described in the paper “Fuzzy logic and neural networks-applications to analytical chemistry”.

The paper will be provided!

Critical behavior at bifurcation onset (Section 4.4 pages 59-61 of the script):

(a) Find the physically correct measure for the distance to equilibrium.

(b) Understand the phase transition and the difference to the “spin model” transition.

Holographic Neurons:

(a) Determine the sensitivity of the Holographic Neural Network to inputs which have been scaled,translated, and rotated.

Holographic Neurons:

(a) Implement the vector version of the Holographic Neuron

Please choose your project by May 12th. The specifics will be arranged for each person during the exercise session.