Execution Modes

Choose the right adapter for your use case.

Overview

Choose the first row below whose condition applies:

ConditionAdapter
Data too large to fit in memoryStreaming
Fits in memory, need real-time/incremental updatesOnline
Fits in memory, no real-time requirementBatch
ModeUse CaseMemoryFeatures
BatchComplete datasetsFullAll features
StreamingLarge files (>100K)ChunkedResiduals, robustness
OnlineReal-time sensorsFixed windowIncremental updates

Adapter Comparison


Batch Adapter

Standard mode for complete datasets. Supports all features.

When to Use

  • Dataset fits in memory
  • Need intervals, cross-validation, or diagnostics
  • Processing complete files

Example

using FastLOWESS
using Random, Statistics

rng = MersenneTwister(42)
x = collect(range(0, 2π, length=100))
y = sin.(x) .+ randn(rng, 100) .* 0.3

model = Lowess(;
    fraction=0.5,
    iterations=3,
    parallel=true,
    outputs=["diagnostics"],
    intervals=(confidence=0.95, prediction=0.95),
)
result = fit(model, x, y)
println("95% CI at midpoint: [", result.confidence_lower[51], ", ", result.confidence_upper[51], "]")
println("R2: ", result.diagnostics.r_squared)
95% CI at midpoint: [-0.01832864538240274, 0.17819422334560386]
R2: 0.7909162309681064

Streaming Adapter

Process large datasets in chunks with configurable overlap.

When to Use

  • Dataset >100,000 points
  • Memory-constrained environments
  • Batch processing pipelines

Parameters

ParameterDefaultDescription
chunk_size5000Points per chunk
overlapchunk_size / 10Overlap between chunks
merge_strategy"weighted_average"How to merge overlaps

Merge Strategies

StrategyBehavior
"average"Average overlapping values
"weighted_average"Distance-weighted blend
"take_first"Keep left chunk values
"take_last"Keep right chunk values

Example

using FastLOWESS
using Random, Statistics

rng = MersenneTwister(42)
x = collect(range(0, 2π, length=100))
y = sin.(x) .+ randn(rng, 100) .* 0.3

model = StreamingLowess(;
    fraction=0.3,
    iterations=2,
    chunk_size=5000,
    overlap=500,
    merge_strategy="average"
)
process_chunk(model, x, y)
result = finalize(model)
println("Smoothed y[0]: ", result.y[1])
Smoothed y[0]: 0.35383694154729933

Always call finalize()

Always call finalize(model) after processing all chunks to retrieve buffered overlap data.

Online Adapter

Incremental updates with a sliding window for real-time data.

When to Use

  • Data arrives incrementally (sensors, streams)
  • Need real-time smoothed values
  • Fixed memory budget

Parameters

ParameterDefaultDescription
window_capacity1000Max points in window
min_points2Points before output starts
update_mode"incremental"Update strategy

Update Modes

ModeBehaviorSpeed
"incremental"Update only affected fitsFaster
"full"Recompute entire windowMore accurate

Example

using FastLOWESS
using Random, Statistics

rng = MersenneTwister(42)
x = collect(range(0, 2π, length=100))
y = sin.(x) .+ randn(rng, 100) .* 0.3

model = OnlineLowess(;
    fraction=0.2,
    window_capacity=100,
    min_points=5,
    update_mode="incremental"
)
shown = 0
for i in eachindex(x)
    global shown
    result = add_point(model, x[i], y[i])
    if result !== nothing && shown < 5
        println("Current smoothed value: ", result.y)
        shown += 1
    end
end
Current smoothed value: 0.23103531694201596
Current smoothed value: 0.4962273769575686
Current smoothed value: 0.6767458589597115
Current smoothed value: 0.41363571386216635
Current smoothed value: 0.31508056492212055

Feature Comparison

FeatureBatchStreamingOnline
Confidence intervals✓✗✗
Prediction intervals✓✗✗
Cross-validation✓✗✗
Diagnostics✓✓✗
Residuals✓✓✓
Robustness weights✓✓✓
Parallel execution✓✓✗

Next Steps