Intervals

Confidence and prediction intervals for uncertainty quantification.

Overview

Confidence and Prediction Intervals

Adapter support: Confidence and prediction intervals are available in Batch mode only. Streaming and Online modes do not support intervals.

Confidence and prediction coverage levels are independent; for example, a 90% confidence interval can be paired with a 99% prediction interval.

Type Represents Width Use

Confidence

Uncertainty in mean curve

Narrow

Where is the true trend?

Prediction

Uncertainty for new points

Wide

Where will new data fall?


Confidence Intervals

Estimate uncertainty in the smoothed curve itself.

import fastlowess.IntervalsOptions;
import fastlowess.Lowess;
import fastlowess.Options;
import fastlowess.Result;

public class Example {
    public static void main(String[] args) {
        int n = 100;
        double[] x = new double[n];
        double[] y = new double[n];
        for (int i = 0; i < n; i++) {
            x[i] = i * 2 * Math.PI / (n - 1);
            y[i] = Math.sin(x[i]) + 0.1;
        }

        Options options = Options.builder()
                .fraction(0.5)
                .intervals(IntervalsOptions.builder().confidence(0.95).build()) // 95% CI
                .build();

        try (Lowess model = new Lowess(options)) {
            Result result = model.fit(x, y);

            double[] lower = result.confidenceLower().orElseThrow();
            double[] upper = result.confidenceUpper().orElseThrow();
            for (int i = 0; i < 3; i++) {
                System.out.printf("x=%.2f: y=%.2f [%.2f, %.2f]%n",
                        result.x()[i], result.y()[i], lower[i], upper[i]);
            }
        }
    }
}
x=0.00: y=0.33 [0.29, 0.37]
x=0.06: y=0.36 [0.32, 0.40]
x=0.13: y=0.39 [0.35, 0.43]

Prediction Intervals

Estimate where new observations might fall.

import fastlowess.IntervalsOptions;
import fastlowess.Lowess;
import fastlowess.Options;
import fastlowess.Result;

public class Example {
    public static void main(String[] args) {
        int n = 100;
        double[] x = new double[n];
        double[] y = new double[n];
        for (int i = 0; i < n; i++) {
            x[i] = i * 2 * Math.PI / (n - 1);
            y[i] = Math.sin(x[i]) + 0.1;
        }

        Options options = Options.builder()
                .fraction(0.5)
                .intervals(IntervalsOptions.builder().prediction(0.95).build()) // 95% PI
                .build();

        try (Lowess model = new Lowess(options)) {
            Result result = model.fit(x, y);
            System.out.printf("Prediction bounds: [%.2f, %.2f]%n",
                    result.predictionLower().orElseThrow()[0], result.predictionUpper().orElseThrow()[0]);
        }
    }
}
Prediction bounds: [-0.04, 0.71]

Both Intervals

Request both types simultaneously:

import fastlowess.IntervalsOptions;
import fastlowess.Lowess;
import fastlowess.Options;
import fastlowess.Result;

public class Example {
    public static void main(String[] args) {
        int n = 100;
        double[] x = new double[n];
        double[] y = new double[n];
        for (int i = 0; i < n; i++) {
            x[i] = i * 2 * Math.PI / (n - 1);
            y[i] = Math.sin(x[i]) + 0.1;
        }

        Options options = Options.builder()
                .fraction(0.5)
                .intervals(IntervalsOptions.builder().confidence(0.95).prediction(0.95).build())
                .build();

        try (Lowess model = new Lowess(options)) {
            Result result = model.fit(x, y);
            System.out.printf("First point 95%% CI: [%s, %s]%n",
                    result.confidenceLower().orElseThrow()[0], result.confidenceUpper().orElseThrow()[0]);
        }
    }
}
First point 95% CI: [0.2929290861928646, 0.3721536723881129]

Confidence Levels

Common levels and their z-values:

Level z-value Interpretation

0.90

1.645

90% of intervals contain true value

0.95

1.960

95% of intervals contain true value

0.99

2.576

99% of intervals contain true value

import fastlowess.IntervalsOptions;
import fastlowess.Lowess;
import fastlowess.Options;
import fastlowess.Result;

public class Example {
    public static void main(String[] args) {
        int n = 100;
        double[] x = new double[n];
        double[] y = new double[n];
        for (int i = 0; i < n; i++) {
            x[i] = i * 2 * Math.PI / (n - 1);
            y[i] = Math.sin(x[i]) + 0.1;
        }

        // 99% confidence interval
        Options options = Options.builder()
                .intervals(IntervalsOptions.builder().confidence(0.99).build())
                .build();

        try (Lowess model = new Lowess(options)) {
            Result result = model.fit(x, y);
            System.out.println("First lower CI bound (99%): " + result.confidenceLower().orElseThrow()[0]);
        }
    }
}
First lower CI bound (99%): 0.31746448671917393

Residual Bootstrap

Set bootstrap(n) to at least 2 to replace analytic uncertainty with residual-bootstrap refits. Batch shares its outer seed with CV; Streaming restarts the seed per combined chunk, and Online per full-update window. Online bootstrap requires updateMode("full").

import fastlowess.IntervalsOptions;
import fastlowess.Lowess;
import fastlowess.OnlineLowess;
import fastlowess.OnlineOptions;
import fastlowess.Options;
import fastlowess.PointResult;
import fastlowess.Result;
import fastlowess.StreamingLowess;
import fastlowess.StreamingOptions;

public class Example {
    public static void main(String[] args) {
        double[] x = new double[30];
        double[] y = new double[30];
        for (int i = 0; i < x.length; i++) {
            x[i] = i * 0.1;
            y[i] = Math.sin(x[i]) + 0.1 * Math.cos(7 * x[i]);
        }
        IntervalsOptions intervals = IntervalsOptions.builder()
                .confidence(0.95).prediction(0.95).bootstrap(20).build();

        try (Lowess batch = new Lowess(Options.builder().intervals(intervals).seed(42L).build())) {
            Result fitted = batch.fit(x, y);
            System.out.println("Batch SEs: " + fitted.standardErrors().orElseThrow().length);
        }

        try (StreamingLowess stream = new StreamingLowess(StreamingOptions.builder()
                .chunkSize(x.length).intervals(intervals).seed(42L).build())) {
            Result chunk = stream.processChunk(x, y);
            System.out.println("Streaming CI present: " + chunk.confidenceLower().isPresent());
        }

        try (OnlineLowess online = new OnlineLowess(OnlineOptions.builder()
                .minPoints(5).updateMode("full").intervals(intervals).seed(42L).build())) {
            PointResult last = null;
            for (int i = 0; i < 12; i++) {
                last = online.addPoint(x[i], y[i]).point().orElse(last);
            }
            System.out.println("Online PI present: " + last.predictionLower().isPresent());
        }
    }
}
Batch SEs: 30
Streaming CI present: true
Online PI present: true

Standard Errors

Access standard errors directly (available when intervals are computed):

import fastlowess.Lowess;
import fastlowess.Options;
import fastlowess.Result;

public class Example {
    public static void main(String[] args) {
        int n = 100;
        double[] x = new double[n];
        double[] y = new double[n];
        for (int i = 0; i < n; i++) {
            x[i] = i * 2 * Math.PI / (n - 1);
            y[i] = Math.sin(x[i]) + 0.1;
        }

        Options options = Options.builder().outputs("se").build();

        try (Lowess model = new Lowess(options)) {
            Result result = model.fit(x, y);
            double[] se = result.standardErrors().orElseThrow();
            for (int i = 0; i < 3; i++) {
                System.out.printf("Point %d: SE = %.4f%n", i, se[i]);
            }
        }
    }
}
Point 0: SE = 0.0246
Point 1: SE = 0.0252
Point 2: SE = 0.0258

Availability

Streaming computes intervals per combined chunk and blends them across overlaps. Online requires updateMode("full") and reports intervals for the latest point of each window.

Feature Batch Streaming Online

Confidence intervals

✓

✓

✓ (full mode)

Prediction intervals

✓

✓

✓ (full mode)

Standard errors

✓

✓

✓ (full mode)

Residual bootstrap

✓

✓

✓ (full mode)

All examples above use the same 100-point noisy sine wave (see Weight Functions for the generation code).