In class, we learned a Nave Bayes classi er for binary feature values, i. e., xj 2 f0; 1g where we model the class conditional distribution to be Bernoulli. In this exercise, you are going to extend the result to the case where features that are non-binary. We are given a training set f(x(i); y(i)); i = f1; ;mgg, where x(i) 2 f1; 2; ; sgn and y(i) 2 f0; 1g. Again, we model the label as a biased coin with 0 = P(y(i) = 0) and 1 0 = P(y(i) = 1). We model each non-binary feature value x(i) j (an element of x(i)) as a biased dice for each class. This is parameterized by:
P(zj-s|y = 1) = θ.sly=1
Notice that we do not model P(aj sly = 0) and Pu; = sly = 1) directly. Instead we use the above equations to guarantee all probabilities for each class sum to 1.
A) Using the Naive Bayes (NB) assumption, write down the joint probability of the data: in terms of the parameters θο, θjkly-0 and ,,k y-1. You mlay find the indicator function 1(-) useful.
B) Maximizing the joint probability you get in (a) with respect to o, ,ky-o and jk y-1. Write down your resulting θο, θ, k y 0 and θ.kl,-1 and show intermediate steps.