In NLP, logistic regression is the baseline supervised machine learning algorithm for classification.

• discriminative classifier: like logistic regression
  • only trying to learn to distinguish the classes.
  • directly compute $$P(c|d)$$
• generative classifier: like naive Bayes
  • have the goal of understanding what each class looks like.
  • makes use of likelihood term $$P(d|c)P©$$

A machine learning system for classification has four components:

• A feature representation of the input
• A classification function that computes $$\hat y$$, the estimated class, via $$p(y|x)$$. Like sigmoid and softmax.
• An objective function for learning, usually involving minimizing error on training examples. Like cross-entropy loss function.
• An algorithm for optimizing the objective function. Like stochastic gradient descent.

More →