Artificial neural network

From Wikimedia Commons, the free media repository
Jump to: navigation, search

An artificial neural network (ANN) is an interconnected group of artificial neurons that uses a mathematical or computational model for information processing based on a connectionist approach to computation.


Neurons in artificial neural networks are generally structured in layers, each layer holding several neurons. This structure can be quite different.

Three different structures are shown here:


A artificial neural network is build from several Neurons. A Neuron can be drawn in the following way:

Activation functions[edit]

Neurons can have different activation functions.

Three different functions are described here:

Hard limit function[edit]

A neuron with a hard limit function

\varphi^{\mbox{hlim}}(v) = \begin{cases}
 1 & \mbox{for } v \geq 0 \\
  0 & \mbox{for } v < 0

Piecewise linear function[edit]

\varphi^{\mbox{pwl}}(v) = \begin{cases}
 1 & \mbox{for } v \geq \frac{1}{2} \\
 v + \frac{1}{2} & \mbox{for } -\frac{1}{2} < v < \frac{1}{2} \\
 0 & \mbox{for } v \leq -\frac{1}{2}

Sigmoid function[edit]

A sigmoid function is also called a McCulloch-Pitts Model. can have a variable slope parameter a

\varphi_a^{\mbox{sig}}(v) = \frac{1}{1 + \exp(-av)}.

Specific types[edit]

Elman Networks[edit]

Elman Networks are special artificial neural networks which have a memory and thus are able to represent time in an implicit way.

Time Delay Neural Networks[edit]

Time Delay Neural Networks (TDNNs) are special artificial neural networks which receive input over several time steps. Time is represented in an explicit way. The image shows an two-layer TDNN with neuron activations.

See also[edit]