One decision tree tries to solve a prediction problem in one pass. Gradient boosting asks a different question: what small correction would improve the model we already have?
Begin with one constant prediction. Measure what remains wrong. Train a small tree to generalize that correction, add only part of it, and repeat.
In this week’s interactive article, you can build that sequence yourself for regression and classification. You will see why residual fitting is one special case of a broader idea: gradient descent in the space of predictive functions.