# Artificial neural network

An artificial neural network (ANN) is an interconnected group of artificial neurons that uses a mathematical or computational model for information processing based on a connectionist approach to computation.

## Structure

Neurons in artificial neural networks are generally structured in layers, each layer holding several neurons. This structure can be quite different.

Three different structures are shown here:

## Neuron

A artificial neural network is build from several Neurons. A Neuron can be drawn in the following way:

## Activation functions

Neurons can have different activation functions.

Three different functions are described here:

### Hard limit function

A neuron with a hard limit function is also called a McCulloch-Pitts Model.

$\varphi^{\mbox{hlim}}(v) = \begin{cases} 1 & \mbox{for } v \geq 0 \\ 0 & \mbox{for } v < 0 \end{cases}$

### Piecewise linear function

$\varphi^{\mbox{pwl}}(v) = \begin{cases} 1 & \mbox{for } v \geq \frac{1}{2} \\ v + \frac{1}{2} & \mbox{for } -\frac{1}{2} < v < \frac{1}{2} \\ 0 & \mbox{for } v \leq -\frac{1}{2} \end{cases}$

### Sigmoid function

A sigmoid function can have a variable slope parameter $a$

$\varphi_a^{\mbox{sig}}(v) = \frac{1}{1 + \exp(-av)}.$

## Specific types

### Elman Networks

Elman Networks are special artificial neural networks which have a memory and thus are able to represent time in an implicit way.

### Time Delay Neural Networks

Time Delay Neural Networks (TDNNs) are special artificial neural networks which receive input over several time steps. Time is represented in an explicit way. The image shows an two-layer TDNN with neuron activations.