Sigmoid or logistic function is a step function that calculates the
probability of "(the transpose of θ) times x"
It is used in logistic regression to tell which class a point belongs to.
Remember what "(the transpose of θ) times x" is?
X is the dataset and θ is the weight-vector or co-efficient for each feature in
the dataset. Each value of θ signifies the importance of each feature in the data.
When "(the transpose of θ) times x" is large for a point, sigmoid is close to 1.
This means that probability of that point belonging to class 1 is almost 1 (most likely).
When "(the transpose of θ) times x" is small for a point, sigmoid is close to 0.
This means that probability of that point belonging to class 1 is almost 0 (least likely).
def sigmoid_activation(weighted_sum):
return 1.0 / (1.0 + np.exp(-1 * weighted_sum))