What is Feature Scaling?
Feature Scaling is an important pre-processing step for some machine learning algorithms.
Imagine you have three friends of whom you know the individual weight and height.
You would like to deduce Christian’s t-shirt size from David’s and Julia’s by looking at the height and weight.
|Name||Height in m||Weight in kg||T-Shirt size|
One way You could determine the shirt size is to just add up the weight and the height of each friend. You would get:
|Name||Height + weight||T-Shirt size|
Because Christian’s height + weight number is nearer to Julia’s number than to David’s, Christian should wear a small T-Shirt. What?
Feature Scaling Formula
x’ = (x – xmin) / (xmax – xmin)
|Name||Scaled Height||Scaled Weight||Combined Scaled Height + Weight||T-Shirt size|
If we look at the combined scaled properties we see that Christian’s value now is closer to David’s so we deduce that Christian shall wear a large shirt as well.
Implementing feature scaling in python
As a little coding practice we can implement a feature scaling algorithm in Python:
def feature_scaling(arr): ret_arr =  min_val = min(arr) max_val = max(arr) if min_val == max_val: raise ZeroDivisionError() for f in arr: f = (f - min_val) / float((max_val - min_val)) ret_arr.append(f) return ret_arr
MinMaxScaler from sklearn
Instead of writing our own feature scaler
we can we should use the MinMaxScaler from sklearn. It works with numpy arrays by default.
from sklearn.preprocessing import MinMaxScaler import numpy as np weights = np.array([[52.0], [79.0], [64.0]]) scaler = MinMaxScaler() rescaled_weight = scaler.fit_transform(weights) print(rescaled_weight)
Which algorithms are affected by non-properly scaled features?
SVM and k-means are algorithms which are affected. SVM for example calculates distances and when two features differ dramatically in value range, the feature with the greater range will dominate the other. (As seen when adding kilograms and meters)