Gini impurity
Gini is the probability of misclassifying a randomly chosen sample when labels are assigned according to the node's class distribution.
Gini = 1 − Σ pᵢ²
Decision trees · Chapter 2
Gini or entropy?
Both measures answer the same practical question: how mixed are the classes inside this node?
Entropy
Entropy measures uncertainty. It is zero in a pure node and increases as the class distribution becomes more even.
Entropy = −Σ pᵢ log₂(pᵢ)
The practical choice
They often choose the same split. Their numeric scales differ, so compare gains within one criterion rather than comparing raw Gini and entropy values directly.
A balanced binary node
With two samples from Class A and two from Class B, Gini is0.5 and entropy is 1 bit. If all four samples share one class, both measures become zero.
Return to the first lesson and switch the impurity measure. The control runs the training algorithm again; it does not replace the number in the interface.