THRESHOLD FUNCTIONS
Threshold Logic Units
• Activation
• In a Threshold Logic Unit (TLU) the output of the unit y in response to a particular input pattern is
calculated in two stages. First the activation is calculated. The activation a is the weighted sum of the
•
• Inputs:
• Where x_i is the ith element of the input vector and w_i is the ith element of the weight vector. The
current activation in the TLU in the demonstration below is represented by the dot on the green plane
in the graph. The green plane shows all the possible activation values as the inputs vary. The current
activation is also marked on the vertical axis. First set the weights to non-zero values. Then alter the
inputs and watch the activation change. The activation value a is displayed in green by the diagram of
the TLU. Do this now.
• Output
• The activation is passed through a threshold function to obtain the output y :
• The red plane in the demonstration represents the threshold of the unit, h . It cuts across the
activation plane. If the activation is higher than or equal to the threshold then the output of the
unit will be 1, otherwise it will be 0. You can see the value of y change as the activation moves
across the threshold. Do this now. The threshold function is also referred to as an activation
function, a transfer function, or an output function. Input Patterns
• The inputs presented to the TLU at the same time are called an input pattern. Input patterns are
also referred to as input vectors, exemplars, and data points.
PIECEWISE LINEAR FUNCTION
In mathematics, a piecewise linear function is a real-valued function defined on the •
real numbers or a segment thereof, whose graph is composed of straight-line
sections.It is a piecewise-defined function, each of whose pieces is an affine function.
Usually – but not always – the function is assumed to be continuous; in that case, its •
graph is a polygonal curve.
• The function defined by:
• is piecewise linear with four pieces. (The graph of this function is shown to the right.) Since the
graph of a linear function is a line, the graph of a piecewise linear function consists of line
segments and rays.
• Other examples of piecewise linear functions include the absolute value function, the square
wave, the sawtooth function, and the floor function.
NOTATION
• The notion of a piecewise linear function makes sense in several different contexts.
Piecewise linear functions may be defined on n-dimensional Euclidean space, or more
generally any vector space or affine space, as well as on piecewise linear manifolds,
simplicial complexes, and so forth. In each case, the function may be real-valued, or it may
take values from a vector space, an affine space, a piecewise-linear manifold, or a
simplicial complex. (In these contexts, the term “linear” does not refer solely to linear
transformations, but to more general affine linear functions.)
SIGMOID FUNCTION
• A sigmoid function is a mathematical function having an "S" shape (sigmoid curve).
Often, sigmoid function refers to the special case of the logistic function shown in the first
figure and defined by the formula
• Other examples of similar shapes include the Gompertz curve (used in modeling systems
that saturate at large values of t) and the ogee curve (used in the spillway of some dams).
A wide variety of sigmoid functions have been used as the activation function of artificial
neurons, including the logistic and hyperbolic tangent functions. Sigmoid curves are also
common in statistics as cumulative distribution functions, such as the integrals of the
logistic distribution, the normal distribution, and Student's t probability density functions.
• Definition
• A sigmoid function is a bounded differentiable real function that is defined for all real
input values and has a positive derivative at each point.[1]
• Properties
• In general, a sigmoid function is real-valued and differentiable, having either a nonnegative or non-positive first derivative[citation needed] which is bell shaped. There are also a
pair of horizontal asymptotes as
The differential equation
• with the inclusion of a boundary condition providing a third degree of freedom,
.epyt siht fo snoitcnuf fo ssalc.
a sedivorp ,
• Examples
• Many natural processes, such as those of complex system learning curves, exhibit a
progression from small beginnings that accelerates and approaches a climax over time. When a
detailed description is lacking, a sigmoid function is often used[2] .
• Besides the logistic function, sigmoid functions include the ordinary arctangent, the hyperbolic
tangent, the Gudermannian function, and the error function, but also the generalised logistic
function and algebraic functions like
• The integral of any smooth, positive, "bump-shaped" function will be sigmoidal, thus the
cumulative distribution functions for many common probability distributions are sigmoidal. The
most famous such example is the error function, which is related to the cumulative distribution
function (CDF) of a normal distribution.