Meanwhile, discriminative models are used for either classification or regression and they return a prediction based on conditional probability. There are a variety of ways to categorize a machine learning model.
A model can be classified as belonging to different categories like: generative models, discriminative models, parametric models, non-parametric models, tree-based models, non-tree-based models.
This article will focus on the differences between generative models and discriminative models. Generative models are those that center on the distribution of the classes within the dataset.
The machine learning algorithms typically model the distribution of the data points. Generative models rely on finding joint probability. Generative models are typically employed to estimate probabilities and likelihood, modeling data points and discriminating between classes based on these probabilities.
Because the model learns a probability distribution for the dataset, it can reference this probability distribution to generate new data instances. Generative models often rely on Bayes theorem to find the joint probability, finding p x,y.
Essentially, generative models model how the data was generated, answer the following question:. While generative models learn about the distribution of the dataset, discriminative models learn about the boundary between classes within a dataset. With discriminative models, the goal is to identify the decision boundary between classes to apply reliable class labels to data instances.
Discriminative models separate the classes in the dataset by using conditional probability, not making any assumptions about individual data points. Examples of discriminative models in machine learning include support vector machines , logistic regression, decision trees, and random forests.
LDA models function by estimating the variance and mean of the data for the each class in the dataset. After the mean and variances for every class has been calculated, predictions can be made by estimating the probability that a given set of inputs belongs to a given class.
Markov chains are used to determine the probability of moving from state j to state i, which can be denoted as p i,j. This is just the joint probability mentioned above. A Hidden Markov Model is where an invisible, unobservable Markov chain is used. The data inputs are given to the model and the probabilities for the current state and the state immediately preceding it are used to calculate the most likely outcome.
Bayesian networks are a type of probabilistic graphical model. They represent conditional dependencies between variables, as represented by a Directed Acyclic Graph. In a Bayesian network, each edge of the graph represents a conditional dependency, and each node corresponds to a unique variable.
The conditional independence for the unique relationships in the graph can be used to determine the joint distribution of the variables and calculate joint probability. In other words, a Bayesian network captures a subset of the independent relationships in a specific joint probability distribution. Once a Bayesian network has been created and properly defined, with Random Variables, Conditional Relationships, and Probability Distributions known, it can be used to estimate the probability of events or outcomes.
One of the most commonly used types of Bayesian Networks is a Naive Bayes model. Support vector machines operate by drawing a decision boundary between data points, finding the decision boundary that best separates the different classes in the dataset.
The SVM algorithm draws either lines or hyperplanes that separate points, for 2-dimensional spaces and 3D spaces respectively. Logistic regression is an algorithm that uses a logit log-odds function to determinant the probability of an input being in one of two states. Probabilities greater than 0. For this reason, logistic regression is typically used in binary classification problems. However, logistic regression can be applied to multi-class problems by using a one vs.
Nodes in a decision tree are where decisions about data points are made using different filtering criteria. The leaves in a decision tree are the data points that have been classified. SE: Generative vs. Discriminative — Lenar Hoyt. That link is broken; here's the correct version: meta.
Add a comment. Active Oldest Votes. Improve this answer. Stompchicken Stompchicken Thanks for the paper. The author is now professor at Stanford and has wonderful resources at stanford. A nice explanation also by Andrew Ng here — clyfe. When staring at the matrices observe that in the first one, all entries sum up to 1.
This will speed the enlightenment and reduce confusion — Maxim Khesin. A note by Andrew Ng here is also very useful: cs Show 6 more comments. Kaushal28 5, 4 4 gold badges 34 34 silver badges 61 61 bronze badges. Carlos Rendon Carlos Rendon 6, 5 5 gold badges 31 31 silver badges 50 50 bronze badges.
This answer confuses me. Both classes of algorithsm fall into the class of supervised learning algorithms, which learn a model of labeled training data to derive a function that predicts other data.
The discrimitive algorithm as you describe it sounds as if it does not create a model, is that correct? I would be glad if you could enhence your answer in that regard. A discriminative algorithm uses the data to create a decision boundary, so you ask it "what side of the decision boundary is this instance on? So a generative model like Naive Bayes, does not have a decision boundary?
So generative models seem like they are better for interpretability? Imagine your task is to classify a speech to a language. You can do it by either: learning each language, and then classifying it using the knowledge you just gained or determining the difference in the linguistic models without learning the languages, and then classifying the speech.
Isn't it the other way round? Considering that you learned the language, you are operating on a conditional distribution and so it should be a discriminative approach? I think it is the other way around as well after reading the answers below - Example from the lecture notes of CS by Ghrua — Mitali Cyrus.
In practice, the models are used as follows. In discriminative models , to predict the label y from the training example x , you must evaluate: which merely chooses what is the most likely class y considering x. So, you are left with which is the equation you use in generative models. Saul Berardo Saul Berardo 2, 2 2 gold badges 18 18 silver badges 21 21 bronze badges. By this reasoning, aren't the generative and the discriminative model equal when applied to the same distribution?
Why is there a difference in classification behavior then? Or are they just equal in this maximum likelihood context? To tell whether they are "equal" or not, we need first define what we mean by that.
The are many things in common, but the strongest difference is the strategy employed: model the distribution generative vs.
Ghrua Ghrua 4, 3 3 gold badges 15 15 silver badges 25 25 bronze badges. Ravi Ravi 2, 13 13 silver badges 28 28 bronze badges. Sriwantha Attanayake Sriwantha Attanayake 7, 4 4 gold badges 37 37 silver badges 44 44 bronze badges. An addition informative point that goes well with the answer by StompChicken above. The fundamental difference between discriminative models and generative models is: Discriminative models learn the hard or soft boundary between classes Generative models model the distribution of individual classes Edit: A Generative model is the one that can generate data.
As a matter of fact, most not sure if all unsupervised learning algorithms like clustering etc can be called generative, since they model P d and there are no classes:P PS: Part of the answer is taken from source. Community Bot 1 1 1 silver badge. Prakhar Agrawal Prakhar Agrawal 11 11 silver badges 20 20 bronze badges. The short answer Many of the answers here rely on the widely-used mathematical definition [1]: Discriminative models directly learn the conditional predictive distribution p y x.
Generative models learn the joint distribution p x,y or rather, p x y and p y. Predictive distribution p y x can be obtained with Bayes' rule. A broader and more fundamental definition [2] seems equally fitting for this general question: Discriminative models learn the boundary between classes. So they can discriminate between different kinds of data instances. Generative models learn the distribution of data. So they can generate new data instances.
Image source A closer look Even so, this question implies somewhat of a false dichotomy [3]. Discriminative: Support vector machine SVM , logistic regression, most deep neural networks. Generative classifiers" Comment on "On Discriminative vs. However, the term generative models is typically reserved for high dimensional data.
Christabella Irwanto Christabella Irwanto 8 8 silver badges 15 15 bronze badges. This really should have more upvotes. Your was the only answer that touched on the "false dichotomy" idea I found. My question os similar to this one: stats. All previous answers are great, and I'd like to plug in one more point. Lerner Zhang Lerner Zhang 4, 1 1 gold badge 35 35 silver badges 53 53 bronze badges. Arun Arun 17 1 1 bronze badge. A discriminative algorithm job is just to classify or differentiate between the 2 outcomes.
Nages Nages 1 1 gold badge 9 9 silver badges 11 11 bronze badges. What I get is generative model is supervised learning based while discriminating model is based on unsupervised learning. Am I Right? WaseemAhmadNaeem Kind of, but kind of not.
Generative seems unsupervised because the first step is to get the complete distribution in all vars, not considering y as special. If you stopped there and don't treat y as special then that part by itself is unsupervised. Thank you in advance — Waseem Ahmad Naeem. Elementary examples Naive Bayes is generative, Logistic Regression is discriminative. More examples of both. In summary, Both are probabilistic models, meaning they both use probability conditional probability , to be precise to calculate classes for the unknown data.
The Overflow Blog. Does ES6 make JavaScript frameworks obsolete? Podcast Do polyglots have an edge when it comes to mastering programming Featured on Meta. Now live: A fully responsive profile. Linked 7. Related Hot Network Questions.
0コメント