Mathwords logoReference LibraryMathwords

Sigmoid Function

The sigmoid function is a mathematical function that takes any real number as input and outputs a value between 0 and 1, producing a characteristic S-shaped curve. It is widely used in machine learning and calculus to model situations where outputs need to be squeezed into a bounded range.

The sigmoid function, often denoted σ(x)\sigma(x), is defined as σ(x)=11+e−x\sigma(x) = \frac{1}{1 + e^{-x}}. It is a smooth, continuously differentiable function whose output approaches 0 as x→−∞x \to -\infty and approaches 1 as x→+∞x \to +\infty, with σ(0)=0.5\sigma(0) = 0.5. The function is symmetric about the point (0,0.5)(0, 0.5) and has a convenient derivative: σ′(x)=σ(x)(1−σ(x))\sigma'(x) = \sigma(x)(1 - \sigma(x)).

Key Formula

σ(x)=11+e−x\sigma(x) = \frac{1}{1 + e^{-x}}
Where:
  • σ(x)σ(x) = the sigmoid output, always between 0 and 1
  • xx = any real number input
  • ee = Euler's number, approximately 2.718

Worked Example

Problem: Evaluate the sigmoid function at x = 2.
Step 1: Write out the sigmoid formula with x = 2.
σ(2)=11+e−2\sigma(2) = \frac{1}{1 + e^{-2}}
Step 2: Calculate the exponent. Since e ≈ 2.718, we get:
e−2≈12.7182≈17.389≈0.1353e^{-2} \approx \frac{1}{2.718^2} \approx \frac{1}{7.389} \approx 0.1353
Step 3: Add 1 to the result.
1+0.1353=1.13531 + 0.1353 = 1.1353
Step 4: Take the reciprocal to find the sigmoid value.
σ(2)=11.1353≈0.8808\sigma(2) = \frac{1}{1.1353} \approx 0.8808
Answer: The sigmoid of 2 is approximately 0.881. This tells us that an input of 2 maps to a value fairly close to 1.

Visualization

Why It Matters

In neural networks, the sigmoid function serves as an activation function that converts raw scores into probabilities between 0 and 1 — making it especially useful for binary classification tasks (e.g., spam or not spam). In calculus, it appears as the solution to the logistic differential equation, which models population growth with a carrying capacity. Its clean derivative also makes it a popular example when studying the chain rule.

Common Mistakes

Mistake: Forgetting the negative sign in the exponent and writing 11+ex\frac{1}{1 + e^{x}} instead of 11+e−x\frac{1}{1 + e^{-x}}.
Correction: The exponent must be −x-x. Without the negative sign, the function flips: large positive inputs would map near 0 instead of near 1.
Mistake: Assuming the sigmoid output can actually reach 0 or 1.
Correction: The sigmoid asymptotically approaches 0 and 1 but never equals them. For any finite input, the output is strictly between 0 and 1.

Related Terms