Here’s a 2 minute refresher on Logistic regression for you:
- Logistic Regression is used to model the outcomes of a categorical target variable
- Input features are scaled just as with linear regression, however result is fed as an input to the logistic function.
- In linear regression, coefficients are found by minimizing the sum of squared error terms
- In logistic regression, we do this by maximizing the likelihood of the data
- The likelihood of an observation is the probability of seeing that observation in a particular model. This is called MAXIMUM LIKELIHOOD ESTIMATION
Interpreting Coefficients in Logistic Regression:
- Similar to interpretation of coefficients in Linear Regression
- A unit increase in feature Xi results in multiplying the odds ratio by an amount
- When a coefficient βi is positive, the we multiply the odds ratio by a number greater than 1, so increasing the feature Xi will effectively increase the probability of the output being labeled as class 1.
- Increasing the feature with a negative coefficient shifts the balance toward predicting the class as 0.
Where Logistic Model differs from Linear Model:
- When we change the value of an input feature, the effect is a multiplication on the odds ratio and not on the model output itself, which is the probability of predicting the class 1.
- The change In the output of our model as a result of a change in the input is not constant throughout, but depends on the current value of our input features.
- This is different from linear regression, where no matter what the values of the input features, the regression coefficients always represent a fixed increase in the output per unit increase of an input feature.
