Artificial neural network

From Wikimedia Commons, the free media repository
Jump to: navigation, search

An artificial neural network (ANN) is an interconnected group of artificial neurons that uses a mathematical or computational model for information processing based on a connectionist approach to computation.

Structure[edit]

Neurons in artificial neural networks are generally structured in layers, each layer holding several neurons. This structure can be quite different.

Three different structures are shown here:

Neuron[edit]

A artificial neural network is build from several Neurons. A Neuron can be drawn in the following way:

Activation functions[edit]

Neurons can have different activation functions.

Three different functions are described here:

Hard limit function[edit]

A neuron with a hard limit function


\varphi^{\mbox{hlim}}(v) = \begin{cases}
 1 & \mbox{for } v \geq 0 \\
  0 & \mbox{for } v < 0
 \end{cases}

Piecewise linear function[edit]


\varphi^{\mbox{pwl}}(v) = \begin{cases}
 1 & \mbox{for } v \geq \frac{1}{2} \\
 v + \frac{1}{2} & \mbox{for } -\frac{1}{2} < v < \frac{1}{2} \\
 0 & \mbox{for } v \leq -\frac{1}{2}
 \end{cases}

Sigmoid function[edit]

A sigmoid function is also called a McCulloch-Pitts Model. can have a variable slope parameter a


\varphi_a^{\mbox{sig}}(v) = \frac{1}{1 + \exp(-av)}.


Specific types[edit]

Elman Networks[edit]

Elman Networks are special artificial neural networks which have a memory and thus are able to represent time in an implicit way.

Time Delay Neural Networks[edit]

Time Delay Neural Networks (TDNNs) are special artificial neural networks which receive input over several time steps. Time is represented in an explicit way. The image shows an two-layer TDNN with neuron activations.

See also[edit]