Intervals
Confidence and prediction intervals for uncertainty quantification.
Overview
|
Adapter support: Confidence and prediction intervals are available in Batch mode only. Streaming and Online modes do not support intervals. |
Confidence and prediction coverage levels are independent; for example, a 90% confidence interval can be paired with a 99% prediction interval.
| Type | Represents | Width | Use |
|---|---|---|---|
Confidence |
Uncertainty in mean curve |
Narrow |
Where is the true trend? |
Prediction |
Uncertainty for new points |
Wide |
Where will new data fall? |
Confidence Intervals
Estimate uncertainty in the smoothed curve itself.
import fastlowess.IntervalsOptions;
import fastlowess.Lowess;
import fastlowess.Options;
import fastlowess.Result;
public class Example {
public static void main(String[] args) {
int n = 100;
double[] x = new double[n];
double[] y = new double[n];
for (int i = 0; i < n; i++) {
x[i] = i * 2 * Math.PI / (n - 1);
y[i] = Math.sin(x[i]) + 0.1;
}
Options options = Options.builder()
.fraction(0.5)
.intervals(IntervalsOptions.builder().confidence(0.95).build()) // 95% CI
.build();
try (Lowess model = new Lowess(options)) {
Result result = model.fit(x, y);
double[] lower = result.confidenceLower().orElseThrow();
double[] upper = result.confidenceUpper().orElseThrow();
for (int i = 0; i < 3; i++) {
System.out.printf("x=%.2f: y=%.2f [%.2f, %.2f]%n",
result.x()[i], result.y()[i], lower[i], upper[i]);
}
}
}
}
x=0.00: y=0.33 [0.29, 0.37]
x=0.06: y=0.36 [0.32, 0.40]
x=0.13: y=0.39 [0.35, 0.43]
Prediction Intervals
Estimate where new observations might fall.
import fastlowess.IntervalsOptions;
import fastlowess.Lowess;
import fastlowess.Options;
import fastlowess.Result;
public class Example {
public static void main(String[] args) {
int n = 100;
double[] x = new double[n];
double[] y = new double[n];
for (int i = 0; i < n; i++) {
x[i] = i * 2 * Math.PI / (n - 1);
y[i] = Math.sin(x[i]) + 0.1;
}
Options options = Options.builder()
.fraction(0.5)
.intervals(IntervalsOptions.builder().prediction(0.95).build()) // 95% PI
.build();
try (Lowess model = new Lowess(options)) {
Result result = model.fit(x, y);
System.out.printf("Prediction bounds: [%.2f, %.2f]%n",
result.predictionLower().orElseThrow()[0], result.predictionUpper().orElseThrow()[0]);
}
}
}
Prediction bounds: [-0.04, 0.71]
Both Intervals
Request both types simultaneously:
import fastlowess.IntervalsOptions;
import fastlowess.Lowess;
import fastlowess.Options;
import fastlowess.Result;
public class Example {
public static void main(String[] args) {
int n = 100;
double[] x = new double[n];
double[] y = new double[n];
for (int i = 0; i < n; i++) {
x[i] = i * 2 * Math.PI / (n - 1);
y[i] = Math.sin(x[i]) + 0.1;
}
Options options = Options.builder()
.fraction(0.5)
.intervals(IntervalsOptions.builder().confidence(0.95).prediction(0.95).build())
.build();
try (Lowess model = new Lowess(options)) {
Result result = model.fit(x, y);
System.out.printf("First point 95%% CI: [%s, %s]%n",
result.confidenceLower().orElseThrow()[0], result.confidenceUpper().orElseThrow()[0]);
}
}
}
First point 95% CI: [0.2929290861928646, 0.3721536723881129]
Confidence Levels
Common levels and their z-values:
| Level | z-value | Interpretation |
|---|---|---|
0.90 |
1.645 |
90% of intervals contain true value |
0.95 |
1.960 |
95% of intervals contain true value |
0.99 |
2.576 |
99% of intervals contain true value |
import fastlowess.IntervalsOptions;
import fastlowess.Lowess;
import fastlowess.Options;
import fastlowess.Result;
public class Example {
public static void main(String[] args) {
int n = 100;
double[] x = new double[n];
double[] y = new double[n];
for (int i = 0; i < n; i++) {
x[i] = i * 2 * Math.PI / (n - 1);
y[i] = Math.sin(x[i]) + 0.1;
}
// 99% confidence interval
Options options = Options.builder()
.intervals(IntervalsOptions.builder().confidence(0.99).build())
.build();
try (Lowess model = new Lowess(options)) {
Result result = model.fit(x, y);
System.out.println("First lower CI bound (99%): " + result.confidenceLower().orElseThrow()[0]);
}
}
}
First lower CI bound (99%): 0.31746448671917393
Residual Bootstrap
Set bootstrap(n) to at least 2 to replace analytic uncertainty with residual-bootstrap refits. Batch shares its outer seed with CV; Streaming restarts the seed per combined chunk, and Online per full-update window. Online bootstrap requires updateMode("full").
import fastlowess.IntervalsOptions;
import fastlowess.Lowess;
import fastlowess.OnlineLowess;
import fastlowess.OnlineOptions;
import fastlowess.Options;
import fastlowess.PointResult;
import fastlowess.Result;
import fastlowess.StreamingLowess;
import fastlowess.StreamingOptions;
public class Example {
public static void main(String[] args) {
double[] x = new double[30];
double[] y = new double[30];
for (int i = 0; i < x.length; i++) {
x[i] = i * 0.1;
y[i] = Math.sin(x[i]) + 0.1 * Math.cos(7 * x[i]);
}
IntervalsOptions intervals = IntervalsOptions.builder()
.confidence(0.95).prediction(0.95).bootstrap(20).build();
try (Lowess batch = new Lowess(Options.builder().intervals(intervals).seed(42L).build())) {
Result fitted = batch.fit(x, y);
System.out.println("Batch SEs: " + fitted.standardErrors().orElseThrow().length);
}
try (StreamingLowess stream = new StreamingLowess(StreamingOptions.builder()
.chunkSize(x.length).intervals(intervals).seed(42L).build())) {
Result chunk = stream.processChunk(x, y);
System.out.println("Streaming CI present: " + chunk.confidenceLower().isPresent());
}
try (OnlineLowess online = new OnlineLowess(OnlineOptions.builder()
.minPoints(5).updateMode("full").intervals(intervals).seed(42L).build())) {
PointResult last = null;
for (int i = 0; i < 12; i++) {
last = online.addPoint(x[i], y[i]).point().orElse(last);
}
System.out.println("Online PI present: " + last.predictionLower().isPresent());
}
}
}
Batch SEs: 30
Streaming CI present: true
Online PI present: true
Standard Errors
Access standard errors directly (available when intervals are computed):
import fastlowess.Lowess;
import fastlowess.Options;
import fastlowess.Result;
public class Example {
public static void main(String[] args) {
int n = 100;
double[] x = new double[n];
double[] y = new double[n];
for (int i = 0; i < n; i++) {
x[i] = i * 2 * Math.PI / (n - 1);
y[i] = Math.sin(x[i]) + 0.1;
}
Options options = Options.builder().outputs("se").build();
try (Lowess model = new Lowess(options)) {
Result result = model.fit(x, y);
double[] se = result.standardErrors().orElseThrow();
for (int i = 0; i < 3; i++) {
System.out.printf("Point %d: SE = %.4f%n", i, se[i]);
}
}
}
}
Point 0: SE = 0.0246
Point 1: SE = 0.0252
Point 2: SE = 0.0258
Availability
|
Streaming computes intervals per combined chunk and blends them across overlaps. Online requires |
| Feature | Batch | Streaming | Online |
|---|---|---|---|
Confidence intervals |
✓ |
✓ |
✓ (full mode) |
Prediction intervals |
✓ |
✓ |
✓ (full mode) |
Standard errors |
✓ |
✓ |
✓ (full mode) |
Residual bootstrap |
✓ |
✓ |
✓ (full mode) |
All examples above use the same 100-point noisy sine wave (see Weight Functions for the generation code).