Classification and Logistic Regression

In machine learning, we often need to choose between two paths. Linear Regression predicts a continuous number, like the price of a house or tomorrow's temperature. However, many real-world problems are Classification tasks, where the output is a discrete category. Is this tumor malignant or benign? Is this credit card transaction fraudulent or legitimate? While linear regression could technically output any number from negative to positive infinity, classification requires a way to 'squish' those numbers into a range that represents probability: a value between $0$ and $1$.

Once we have a probability, we need to make a final decision. We do this by setting a Decision Boundary or threshold. Usually, we set this at $0.5$. If the Sigmoid output is $\ge 0.5$, we classify it as Class 1 (e.g., Spam). If it is $< 0.5$, we classify it as Class 0 (e.g., Not Spam). By changing this threshold, we can make our model more 'cautious' or 'aggressive' depending on the stakes of the decision.

How do we know if our classifier is actually good? We use a Confusion Matrix, a $2 \times 2$ table that compares the Actual labels with the Predicted labels. It breaks results into four quadrants: - True Positives (TP): Predicted Spam, was actually Spam. - True Negatives (TN): Predicted Not Spam, was actually Not Spam. - False Positives (FP): Predicted Spam, but was actually Not Spam (Type I Error). - False Negatives (FN): Predicted Not Spam, but was actually Spam (Type II Error).