Datasets often have many columns that say similar things, such as several exam scores that rise and fall together. Principal component analysis (PCA) combines correlated columns into a smaller number of new features, called principal components, while keeping as much of the information as possible.
The idea in plain words
Imagine a cloud of points stretched like a cigar. Most of the variation runs along its length. PCA finds that direction and calls it the first principal component. The second component is the next most important direction, at right angles to the first, and so on. Keeping only the first few components often keeps most of the information in far fewer columns.
Explained variance
Each component explains a share of the total variation. If the first three components explain 90% of it, you can often use three columns instead of twenty with little loss. Plot the cumulative explained variance and pick the point where adding components gives little extra.
In Python
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
pca_pipeline = make_pipeline(StandardScaler(), PCA(n_components=0.90))
X_reduced = pca_pipeline.fit_transform(X)
n_components=0.90 keeps enough components to explain 90% of the variance. Always scale first; otherwise columns with large numbers dominate.
When PCA helps
- Visualisation: squeeze many features into two components to plot the data and spot groups.
- Speed: fewer features can make some models train faster.
- Correlated features: it removes redundancy before models like linear regression.
When to be careful
- Components are mixtures of original columns, so they are hard to explain to business users.
- PCA captures linear patterns only.
- Fit PCA on training data only, then apply it to test data, to avoid leakage.
Related: feature engineering basics, k-means clustering and statistics for data science.
Techniques like this are covered with projects in our Data Science course in Bangalore.
Book a free demo class at our BTM Layout centre, and ask a counsellor for batch details.
