Cross-Validation

Automated parameter selection via cross-validation.

Overview

Cross-validation helps select optimal parameters (especially fraction) by evaluating performance on held-out data.

---

K-Fold Cross-Validation

Split data into K folds, train on K-1, validate on 1.

const { Lowess } = require('fastlowess');

const n = 100;
const x = Float64Array.from({ length: n }, (_, i) => i * 2 * Math.PI / (n - 1));
const y = Float64Array.from(x, xi => Math.sin(xi) + 0.1);

const model = new Lowess({
    cv: { method: "kfold", k: 5, fractions: [0.2, 0.3, 0.5, 0.7] }
});
const result = model.fit(x, y);

console.log("Selected fraction:", result.fraction_used);
console.log("CV scores:", result.cv_scores);

Selected fraction: 0.3
CV scores: Float64Array(4) [
  0.3442028395147176,
  0.34364341192477843,
  0.39986523402252316,
  0.4760102176301601
]

---

Leave-One-Out (LOOCV)

Each point is held out once. Most thorough but slowest.

const { Lowess } = require('fastlowess');

const n = 100;
const x = Float64Array.from({ length: n }, (_, i) => i * 2 * Math.PI / (n - 1));
const y = Float64Array.from(x, xi => Math.sin(xi) + 0.1);

const model = new Lowess({
    cv: { method: "loocv", fractions: [0.2, 0.3, 0.5, 0.7] }
});
const result = model.fit(x, y);
console.log("Fraction used:", result.fraction_used);

Fraction used: 0.2

---

Seeded Randomization

Set a seed for reproducible fold assignments:

const fl = require('fastlowess');

const n = 100;
const x = Float64Array.from({ length: n }, (_, i) => i * 2 * Math.PI / (n - 1));
const y = Float64Array.from(x, xi => Math.sin(xi) + 0.1);

const model = new fl.Lowess({
    cv: { method: "kfold", k: 5, fractions: [0.3, 0.5, 0.7] },
    seed: 42
});
const result = model.fit(x, y);
console.log("Fraction used:", result.fraction_used);

Fraction used: 0.3

---

Comparison

Method	Folds	Speed	Variance	Bias
KFold(5)	5	Fast	Moderate	Low
KFold(10)	10	Medium	Lower	Lower
LOOCV	N	Slow	Lowest	Lowest

Recommendation

Use 5-fold or 10-fold CV for most applications. LOOCV is only worth it for small datasets (N < 100).

---

CV Metrics

Cross-validation uses MSE (Mean Squared Error) by default:

MSE = mean((y_true - y_pred)^2)

Lower MSE indicates better fit on held-out data.

---

Interpreting Results

const { Lowess } = require('fastlowess');

const n = 100;
const x = Float64Array.from({ length: n }, (_, i) => i * 2 * Math.PI / (n - 1));
const y = Float64Array.from(x, xi => Math.sin(xi) + 0.1);

// Example output
const model = new Lowess({
    cv: { method: "kfold", k: 5, fractions: [0.1, 0.3, 0.5, 0.7] }
});
const result = model.fit(x, y);

// Fraction  | CV Score (MSE)
// 0.1       | 0.0542  ← Undersmoothed
// 0.3       | 0.0231  ← Best
// 0.5       | 0.0298
// 0.7       | 0.0412  ← Oversmoothed
console.log("Fraction used:", result.fraction_used);

Fraction used: 0.3

The fraction with lowest CV score is automatically selected.

---

Availability

Batch Mode Only

Cross-validation is only available in Batch mode.

Feature	Batch	Streaming	Online
K-Fold CV	✓	✗	✗
LOOCV	✓	✗	✗

---

Best Practices

1. Test a range: Include fractions from 0.1 to 0.9
2. Use enough folds: 5-10 folds balance speed and accuracy
3. Set a seed: For reproducible results
4. Check the curve: CV optimizes MSE, but visual inspection matters