In NLP, logistic regression is the baseline supervised machine learning algorithm for classification.
- discriminative classifier: like logistic regression
- only trying to learn to distinguish the classes.
- directly compute $$P(c|d)$$
- generative classifier: like naive Bayes
- have the goal of understanding what each class looks like.
- makes use of likelihood term $$P(d|c)P©$$
A machine learning system for classification has four components:
- A feature representation of the input
- A classification function that computes $$\hat y$$, the estimated class, via $$p(y|x)$$. Like sigmoid and softmax.
- An objective function for learning, usually involving minimizing error on training examples. Like cross-entropy loss function.
- An algorithm for optimizing the objective function. Like stochastic gradient descent.
