Pooling and Activation Functions

After convolution, CNNs use pooling layers to simplify and stabilize the learning process.

Pooling Layer

Pooling reduces the spatial dimensions while keeping the important features.

• Max Pooling: Takes the largest value in each region (e.g., 2×2 window).
• Average Pooling: Takes the average value.

This step helps reduce computation and prevent overfitting by ignoring unimportant variations.

Activation Functions

After pooling, activations bring non-linearity to the network.
Without activation functions, CNNs would behave like linear models — unable to learn complex boundaries.

Sigmoid: Maps input values to a range between 0 and 1, useful for binary classification.

Sigmoid(x) = 1 / (1 + math.exp(-x))

ReLU: Introduces non-linearity:

ReLU(x) = max(0, x)

Tanh: Maps input values to a range between -1 and 1, helping center data around zero.

Tanh(x) = (math.exp(x) - math.exp(-x)) / (math.exp(x) + math.exp(-x))

Softmax: Converts logits into probabilities that sum to 1, ideal for multi-class classification.

output = [math.exp(i) / sum(math.exp(j) for j in inputs) for i in inputs]

Together, convolution, pooling, and activation layers form the backbone of CNNs.