Up to this point, we’ve explored a single classification model: the k-nearest neighbour classifier. Typically, classification models assign a given observation to one of K potentia... Up to this point, we’ve explored a single classification model: the k-nearest neighbour classifier. Typically, classification models assign a given observation to one of K potential categories. These models are commonly known as classifiers. In this section, we’ll delve into two specific classifiers, both grounded in the principles of Bayes theorem. For any events A and B, Bayes theorem states that P(B|A) = P(A|B)P(B) / P(A). This can be expanded to the case when B1,B2,…,BK form a partition of the sample space. In this case, Bayes theorem becomes P(Bk|A) = (P(A|Bk)P(Bk)) / ∑Kj=1 P(A|Bj)P(Bj) for each k = 1, 2, …, K. The next section discusses the naive Bayes classifier, the basis of which is the Bayes theorem. The naive Bayes classifier is a popular classifier which is often used in text analysis (e.g. the detection of spam emails, categorisation of website articles etc.) as well as in medical classifications. However, like other classifiers, it is not limited to these applications. For the purposes of this unit, we shall restrict our attention to one predictor variable. Subsection 9.2.1 delves into the application of the naive Bayes classifier in scenarios where the predictor variable is categorical, while subsection 9.2.2 provides insights into its utilisation when dealing with a continuous variable. Read more