Logistic Regression and k-Nearest Neighbor

• There will be a final exam summary of a one-hour closed book and offline paper exam.

Other Common Classifiers

• Logistic Regression and k-Nearest Neighbor (kNN) are common classifiers.
• Binary indicators on customer's acceptance of a personal loan is the focal dependent variable for personal loan.
• In logistic regression, variables with at least one star sign are considered significant.
• The estimated coefficient of the independent variable ‘Family’ is 0.5476.
• EducationGraduate is the significant independent variable with the highest absolute magnitude of the estimated coefficient.
• The odds of the independent variable of ‘Family’ are 1.729 (e^0.5476).
• Odds refer to the ratio of the probability of personal loan acceptance (Y=1) to the probability of personal loan rejection (Y=0).
• An increase of one unit of family size multiplies the odds of acceptance of a personal loan by e0.5476 (1.729)
• An increase of one unit of family size is associated with an increase of 72.9% in the odds of acceptance of a personal loan.
• Age does not significantly increase the probability of personal loan acceptance.
• Income does not significantly decrease the probability of personal loan acceptance.
• Experience insignificantly decreases the probability of personal loan acceptance.
• In k-Nearest Neighbor algorithms, the error rate is 75% when k is specified as 1.
• For k-Nearest Neighbor algorithms, the error rate is 25% when k is specified as 3.

Dimension Reduction

• Income, overage, underage and phone variables should be selected for subsequent business analytics using decision tree outcome during feature selection.
• Use Principal Component Analysis (PCA) for feature extraction.
• Four principal component variables are needed to explain at least 99% of the total variance of raw data information.
• In PCA results, Sodium is dominantly explained in principal component 1.
• A high rotation value of sodium in principal component 1 may be due to the unit of measurement affecting PCA, so standardization should be applied first.

Introduction to Clustering

• The Euclidean Distance between John and Emily is √21 when using age, number of credit cards and family size, as John is 38 with 5 credit cards and a family size of 4, while Emily is 34 with 7 credit cards and a family size of 3.
• The Manhattan Distance between John and Emily is 7.
• Manhattan Distance is more robust in high-dimensional spaces.
• In high dimensions, Euclidean distances become less informative because all points tend to be similarly distant from each other.
• Manhattan distance grows linearly with the number of dimensions, whereas Euclidean grows with the square root, making Manhattan more discriminative.

Clustering

• For K-Means Clustering. When k is specified to be 2, the assigned initial records point is nodes 1 and 5.
• Then node 1 and 2 will be classified into cluster 1 after calculating distance by Euclidean distance measure.
• Nodes 3, 4, and 5 will be classified into cluster 2.
• The two-dimension location of center point of cluster 1 is (2.5, 2.5) using Euclidean distance measure.
• The two-dimension location of center point of cluster 2 is (26/3, 5/3) using Euclidean distance measure.
• When k is specified to be 2, the assigned initial records point is nodes 3 and 4.
• Nodes 1, 2, and 3 will be classified into cluster 1 after calculating the distance by Euclidean distance measure.
• Node 4 and 5 will be classified into cluster 2.
• The two-dimension location of the center point of cluster 1 is (12/3, 7/3) using the Euclidean distance measure.
• The two-dimension location of the center point of cluster 2 is (19/2, 3/2) using the Euclidean distance measure.
• For node 3, the within-cluster distance is 5 using the Manhattan distance measure.
• For node 3, the closest neighbor distance is 3 using the Manhattan distance measure.
• For node 3 and cluster 1, it is a bad cluster considering the within-cluster distance and the closest neighbor distance

Hierarchical Clustering

• Hierarchical clustering can be conducted based on nodes and two-dimension locational information using Euclidean distance measure and single linkage.
• Draw a dendrogram with hierarchical clustering results.
• Hierarchical clustering can be conducted based on nodes and two-dimension locational information using Manhattan distance measure and complete linkage.
• An example provides logic for public utilities using agglomerative clustering.
• Five utilities and two measures (Sales and Fuel Cost) are used.
• Join Arizona and Commonwealth, then recalculate the distance of four clusters: {Arizona, Commonwealth}, {Boston}, {Central}, {NY}.
• Recalculate the distance of the four clusters using single linkage.
• Consolidate {Central} with {Arizona, Commonwealth} clusters.
• Recompute the distance matrix of the resulting three clusters: {Arizona, Commonwealth, Central}, {Boston}, {NY}.
• Consolidate {NY} with {Boston}, resulting in two clusters: {Arizona, Commonwealth, Central}, {NY, Boston}.
• Consolidate these two clusters and end the algorithm.
• The document frequency (df) value for the word 'Cent' is 1.
• Fill out the term frequency-inverse document frequency (tf-idf) value for the word 'Cent' for document 1 (3*log(5)).