Outlier handling through iterative reweighting.
How Robustness Works#
Standard LOWESS can be biased by outliers. Robustness iterations downweight points with large residuals:
- Fit initial LOWESS
- Compute residuals
- Assign robustness weights (large residuals → low weight)
- Refit using combined distance × robustness weights
- Repeat steps 2–4
Robustness Methods#
Bisquare (Default)#
Smooth downweighting. Points transition gradually from full weight to zero.
$$w(u) = \begin{cases} (1 - u^2)^2 & |u| < 1 \\ 0 & |u| \geq 1 \end{cases}$$Use when: General purpose, balanced approach.
import fastlowess.Lowess;
import fastlowess.Options;
import fastlowess.Result;
public class Example {
public static void main(String[] args) {
int n = 100;
double[] x = new double[n];
double[] y = new double[n];
for (int i = 0; i < n; i++) {
x[i] = i * 2 * Math.PI / (n - 1);
y[i] = Math.sin(x[i]) + 0.1;
}
Options options = Options.builder()
.iterations(3)
.robustnessMethod("bisquare")
.build();
try (Lowess model = new Lowess(options)) {
Result result = model.fit(x, y);
System.out.println("First smoothed value (bisquare robustness): " + result.y()[0]);
}
}
}First smoothed value (bisquare robustness): 0.38260776436644134Huber#
Linear penalty beyond threshold. Less aggressive than Bisquare.
$$w(u) = \begin{cases} 1 & |u| \leq k \\ k/|u| & |u| > k \end{cases}$$Use when: Moderate outliers, want to retain some influence.
import fastlowess.Lowess;
import fastlowess.Options;
import fastlowess.Result;
public class Example {
public static void main(String[] args) {
int n = 100;
double[] x = new double[n];
double[] y = new double[n];
for (int i = 0; i < n; i++) {
x[i] = i * 2 * Math.PI / (n - 1);
y[i] = Math.sin(x[i]) + 0.1;
}
Options options = Options.builder()
.iterations(3)
.robustnessMethod("huber")
.build();
try (Lowess model = new Lowess(options)) {
Result result = model.fit(x, y);
System.out.println("First smoothed value (huber robustness): " + result.y()[0]);
}
}
}First smoothed value (huber robustness): 0.381704958772818Talwar#
Hard threshold. Points are either fully weighted or completely excluded.
$$w(u) = \begin{cases} 1 & |u| \leq k \\ 0 & |u| > k \end{cases}$$Use when: Extreme outliers, want binary exclusion.
import fastlowess.Lowess;
import fastlowess.Options;
import fastlowess.Result;
public class Example {
public static void main(String[] args) {
int n = 100;
double[] x = new double[n];
double[] y = new double[n];
for (int i = 0; i < n; i++) {
x[i] = i * 2 * Math.PI / (n - 1);
y[i] = Math.sin(x[i]) + 0.1;
}
Options options = Options.builder()
.iterations(3)
.robustnessMethod("talwar")
.build();
try (Lowess model = new Lowess(options)) {
Result result = model.fit(x, y);
System.out.println("First smoothed value (talwar robustness): " + result.y()[0]);
}
}
}First smoothed value (talwar robustness): 0.37877937542419027Comparison#
| Method | Transition | Aggressiveness | Use Case |
|---|---|---|---|
| Bisquare | Smooth | Moderate | General purpose |
| Huber | Gradual | Mild | Preserve influence |
| Talwar | Hard | Strong | Extreme contamination |
Detecting Outliers#
Use robustness weights to identify potential outliers:
import fastlowess.Lowess;
import fastlowess.Options;
import fastlowess.Result;
public class Example {
public static void main(String[] args) {
double[] x = { 1.0, 2.0, 3.0, 4.0, 5.0, 6.0 };
double[] y = { 2.0, 4.0, 6.0, 50.0, 10.0, 12.0 }; // 50.0 is an outlier
Options options = Options.builder()
.fraction(0.7)
.iterations(3)
.returnRobustnessWeights(true)
.build();
try (Lowess model = new Lowess(options)) {
Result result = model.fit(x, y);
double[] weights = result.robustnessWeights().orElseThrow();
for (int i = 0; i < weights.length; i++) {
if (weights[i] < 0.5) {
System.out.printf("Point %d is likely an outlier (weight: %.3f)%n", i, weights[i]);
}
}
}
}
}Point 3 is likely an outlier (weight: 0.000)All examples above use the same 100-point noisy sine wave (see Weight Functions for the generation code), except the outlier-detection example above, which uses a small dataset with a deliberate spike.