K-Nearest Neighbors (K-NN) is a type of lazy learning, where the function is only approximated locally and all computation is deferred until classification. It’s one of the simplest algorithms in machine learning used for both classification and regression tasks, but here we’ll focus on its use in classification.
- Step 1: Choose a value for K (number of nearest neighbours), e.g., K = 5.
- Step 2: Take the k nearest neighbors of the new data point, according to Euclidean distance/Manhattan distance/Minkowski distance.
- Step 3: Among these k neighbors, count the n data points in each category. e.g., 3 from the green label and 2 from the red label, out of 5 nearest neighbors.
- Step 4: Assign the new data point to the category where you counted the most neighbours. e.g., 3 from the green label and 2 from the red label, out of 5 nearest neighbors -> the data point falls into the green category.
- Choosing K value: A small K might lead to noisy classification, while a large K might smooth out decision boundaries too much, potentially misclassifying…