Skip to contents

Alternative Software

Source: vignettes/alternative-software.Rmd
alternative-software.Rmd

Overview

rfastlowess is presented throughout this package’s documentation as a faster, more feature-rich alternative to base R’s stats::lowess(). This page is for readers who already use stats::lowess() and want to know: can I get the exact same numbers out of rfastlowess, and if not, why not?

In short:

Reproducing stats::lowess()

stats::lowess() and rfastlowess’s Lowess() implement the same underlying algorithm, but several defaults differ:

Option stats::lowess() Lowess() default
fraction f = 2/3 0.67
iterations iter = 3 3
delta 0.01 * diff(range(x)) same (NULL -> auto)
Zero-weight fallback "return_original" "use_local_mean"
Boundary padding None ("noboundary") "extend"
Residual scaling MAR: median(\|r\|) "mad": median(\|r - median(r)\|)

rfastlowess intentionally keeps its default zero-weight fallback aligned with Rust and the historical package behavior; when comparing to R, pass zero_weight_fallback = "return_original" explicitly to match stats::lowess(). Setting the R-parity options to match R, Lowess() reproduces stats::lowess() exactly (up to floating-point rounding):

library(rfastlowess)
set.seed(42)
x <- seq(0, 2 * pi, length.out = 60)
y <- sin(x) + rnorm(60, sd = 0.2)

reference <- stats::lowess(x, y, f = 2 / 3, iter = 3)

model <- Lowess(
    fraction = 2 / 3,
    iterations = 3L,
    zero_weight_fallback = "return_original",
    boundary_policy = "noboundary",
    scaling_method = "mar"
)
result <- fit(model, x, y)

cat("Max abs difference:", max(abs(result$y - reference$y)), "\n")
#> Max abs difference: 0

If your x is not already sorted, add outputs = "sorted" to get results ordered the same way stats::lowess() returns them (ascending by x):

set.seed(1)
x <- seq(0, 2 * pi, length.out = 60)
x_unsorted <- sample(x)
y_unsorted <- sin(x_unsorted) + rnorm(60, sd = 0.2)

reference <- stats::lowess(x_unsorted, y_unsorted, f = 2 / 3, iter = 3)

model <- Lowess(
    fraction = 2 / 3,
    iterations = 3L,
    zero_weight_fallback = "return_original",
    boundary_policy = "noboundary",
    scaling_method = "mar",
    outputs = "sorted"
)
result <- fit(model, x_unsorted, y_unsorted)

cat("x is sorted ascending:", all(diff(result$x) >= 0), "\n")
#> x is sorted ascending: TRUE
cat("Max abs difference:", max(abs(result$y - reference$y)), "\n")
#> Max abs difference: 0

Without outputs = "sorted", Lowess() still fits on sorted x internally (as stats::lowess() requires) but returns values reordered back to match your original input order — useful when you need the fit aligned with other columns in a data frame, but not what you want for a value-by-value comparison against stats::lowess()’s own (always sorted) output.

Why the defaults differ

rfastlowess’s defaults (as opposed to what it’s capable of reproducing) deliberately depart from stats::lowess() in these places:

Zero-weight fallback (default zero_weight_fallback = "use_local_mean" vs. R’s "return_original"). This package keeps the Rust/default behavior unchanged for end users; exact stats::lowess() comparisons should set zero_weight_fallback = "return_original" explicitly.

Boundary padding (default boundary_policy = "extend" vs. no padding). Without padding, the local neighbourhood at the first and last few points is one-sided, which biases the fit toward the interior and increases variance right at the edges. "extend" (and the other padding policies — see vignette("boundary")) mitigate this at the cost of no longer being a direct reproduction of Cleveland’s original algorithm. "noboundary" is kept as an explicit option specifically so reference-matching remains possible.

Residual scaling (default scaling_method = "mad" vs. MAR). MAD (median(|r - median(r)|)) centers residuals at their median before taking the median absolute value, which is a breakdown-point-optimal scale estimator; MAR (median(|r|), what R uses) does not center first, so it can be biased when residuals are systematically skewed. See vignette("scaling") for the full comparison, including "mean".

What this package adds

stats::lowess() doesn’t support:

Feature rfastlowess stats::lowess()
Kernel functions 7 options Tricube only
Robustness weighting 3 options Bisquare only
Boundary padding 4 policies none
Scale estimation MAD, MAR, mean MAR only
Parallel / GPU execution yes no
Confidence / prediction intervals yes no
Cross-validation for fraction K-fold, LOOCV no
Custom per-observation weights yes no
Streaming / online modes yes no

See vignette("concepts") for an overview of these, or the Benchmarks article for performance comparisons.