Multiclass Bayes Classification Algorithm Decoded

Motivation

We have already seen the Bayes Algorithm for a binary class classification problem here (recommended read before reading this), now let’s extrapolate the same to a multi-class classification setting !!

Simple (=2 features) Multiclass (= K classes)Bayes

Hopefully, you have come here after reading a lot of posts from my account, so you might be aware that we first used to develop mathematical intuition using visualizations considered only a 2 feature dataset.

But in this case, since it is a probability-based algorithm, we will develop this algorithm directly for N features.

Multinomial (=N features) Multiclass (= K classes) Bayes

Case 1 : Assuming all the class conditional probabilities are MND with same covariance !!

Case 2 : Assuming all the class conditional probabilities are MND with different covariance !!

Case 3 : Combination of the above two cases i.e. for few the covariance matrix is same while for others it is different

Other Cases :

It is not necessary that we use only MND as a continuous multivariate probability distribution, we can assume any other continuous multivariate probability distribution, and based on what we use the result might change. But most real-world data follow MND so it’s always preferred to choose that.

It is not advised to choose a discrete multivariate probability distribution because this algorithm will work very badly for a discrete distribution and reasons for the same have been explained in the Bayes algorithm for binary classification, which also holds here !!