Locally Weighted Linear Regression
Move a local weighted fit along a curved dataset and inspect the bandwidth tradeoff.
Free theory articleMove a local weighted fit along a curved dataset and inspect the bandwidth tradeoff.
Concept & interpretation
Locally Weighted Linear Regression (LOESS or LOWESS) is a non-parametric regression method that combines multiple regression models in a k-nearest-neighbor-based meta-model. Instead of fitting a single global line to the entire dataset, it computes a separate linear fit for every target query point using only the points closest to it, weighted by distance.
Experiment
To understand the effect of bandwidth on local predictions in Locally Weighted Linear Regression:
- Keep Query x at exactly
0. - Set Bandwidth to exactly
0.5. Note the small Effective sample size and a prediction close to 0 (the value of the origin). - Change Bandwidth to exactly
1. Notice how points further away gain more weight and drag the prediction upward. - Change Bandwidth to exactly
2. The prediction rises further as the distant high-y points heavily influence the local fit.