Robust Least Square Regression(Linear/Nonlinear) Methods

Let's point to some robust least square methods and some intuitions (not math) behind them :-)


What is the problem of Ordinary Least Square(OLS)? Why do we need robust methods?
------------------------------------------------------------------------------------------
OLS assumes that scatter of data around the fitted line (or curve) follows a Gaussian distribution. Therefore, if this condition is not maintained the fitted line (or curve) will be heavily affected by the outliers. In other words, the fitted curve will wrongly estimated the model parameters due to the influence of outliers.


Some robust least square methods:
-----------------------------------

1. Iteratively Re-weighted Least Square(IRLS): 
If we assume that all red points of the above figure are outliers, the intuition of IRLS is that in each iteration we will weight the data so that the data come closer to the fitted line (or curve). In other words, we will weight data based on their residuals [distance from curve(or line)].


This blog entry will continue as soon as I understand other methods............................................