K-nearest neighbours (KNN) is one of the most intuitive machine learning methods. To classify a new example, it finds the k most similar examples in the training data and lets them vote. There is no complicated training step at all.
How it works
- Store the training data.
- For a new point, measure its distance to every training point, usually with straight-line (Euclidean) distance.
- Pick the k closest points.
- For classification, take the majority class; for regression, average their values.
Example: to predict whether a learner will choose the weekend batch, KNN looks at the k learners most similar in age, work status and commute distance, and uses their choices.
Choosing k
- Small k (like 1 or 3) follows the data closely and can overfit to noise.
- Large k gives smoother, more stable predictions but may blur real differences.
- Use an odd k for two-class problems to avoid ties, and choose the final value with cross-validation.
Why scaling matters
KNN relies on distance, so a feature measured in large numbers (salary in rupees) will drown out one measured in small numbers (years of experience). Always standardise or normalise features first, ideally inside a pipeline:
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.neighbors import KNeighborsClassifier
model = make_pipeline(StandardScaler(), KNeighborsClassifier(n_neighbors=7))
model.fit(X_train, y_train)
Strengths and weaknesses
- Strengths: simple, no training time, works for unusual decision boundaries, easy to explain with examples.
- Weaknesses: slow predictions on large datasets, sensitive to irrelevant features and scaling, and struggles with many dimensions.
KNN vs k-means
The names are similar but the methods differ: KNN is supervised (it needs labels and predicts them), while k-means is unsupervised (it groups unlabelled data). Interviewers like to ask this.
Related: k-means clustering, train-test split and classification models for beginners.
Machine learning models like this one are covered with projects in our Data Science course in Bangalore.
Book a free demo class at our BTM Layout centre, and ask a counsellor for batch details.
