A single decision tree is easy to explain but tends to overfit. A random forest fixes much of that by building many trees and letting them vote. It is one of the most reliable models for tabular data and a favourite first choice in many projects.
The two sources of randomness
- Random rows (bagging): each tree is trained on a random sample of the training data, drawn with replacement.
- Random features: at each split, a tree may only choose from a random subset of the columns.
Because every tree sees slightly different data and features, the trees make different mistakes. Averaging their predictions cancels much of that noise.
Making a prediction
For classification, each tree votes and the majority wins (or the forest averages the probabilities). For regression, the forest averages the numbers predicted by the trees.
In Python
from sklearn.ensemble import RandomForestClassifier
model = RandomForestClassifier(n_estimators=300, max_depth=None, min_samples_leaf=5, n_jobs=-1, random_state=42)
model.fit(X_train, y_train)
More trees rarely hurt accuracy; they only cost time. min_samples_leaf and max_features are the settings worth tuning first.
Feature importance
model.feature_importances_ shows which columns the forest relied on most. It is a useful first look, but it can favour features with many unique values. For important decisions, confirm with permutation importance.
Strengths and limits
- Strengths: works well with little tuning, handles non-linear patterns, needs no feature scaling.
- Limits: harder to explain than one tree, slower to predict with many trees, and it cannot predict values outside the range seen in training.
Read decision trees explained simply first, then cross-validation in plain words to evaluate your forest properly.
Machine learning models like this one are covered with projects in our Data Science course in Bangalore.
Book a free demo class at our BTM Layout centre, and ask a counsellor for batch details.
