Robustness

Another very attractive property of an estimator is robustness against outliers [14]. This means that single, highly erroneous observations do not have much influence on the estimator. To obtain a simple form of robustness called B-robustness, we would like the estimator to have a bounded influence function [14]. Again, we can adapt the results in [18]. It turns out to be impossible to have a completely bounded influence function, but we do have a simpler form of robustness, as stated in the following theorem:

Theorem 3   Assume that the data ${\bf x}$ is whitened (sphered) in a robust manner (see Section 4 for this form of preprocessing). Then the influence function of the estimator $\hat{{\bf w}}$ is never bounded for all ${\bf x}$. However, if h(u)=u g(u) is bounded, the influence function is bounded in sets of the form $\{{\bf x}\:\vert\: \hat{{\bf w}}^T{\bf x}/\Vert{\bf x}\Vert>\epsilon\}$ for every $\epsilon>0$, where g is the derivative of G.

In particular, if one chooses a function G(u) that is bounded, h is also bounded, and $\hat{{\bf w}}$ is rather robust against outliers. If this is not possible, one should at least choose a function G(u) that does not grow very fast when |u|grows.