Installation

Add the module to your go.mod:

go get github.com/thisisamirv/lowess-project/bindings/go/fastlowess/v5

This package uses cgo, so CGO_ENABLED=1 and a working C compiler (GCC or Clang; on Windows, a MinGW-w64 toolchain) are required at build time. See installation.md for details on providing the native fastlowess_go library.

A first fit

package main

import (
 "fmt"
 "log"
 "math"

 "github.com/thisisamirv/lowess-project/bindings/go/fastlowess/v5"
)

func main() {
 n := 100
 x := make([]float64, n)
 y := make([]float64, n)
 for i := 0; i < n; i++ {
  x[i] = float64(i) / 10
  y[i] = math.Sin(x[i]) + 0.1*math.Sin(float64(i))
 }

 opts := fastlowess.DefaultOptions()
 opts.Fraction = 0.2 // smaller fraction = less smoothing, more local detail

 model, err := fastlowess.NewLowess(opts)
 if err != nil {
  log.Fatal(err)
 }
 defer model.Close()

 result, err := model.Fit(x, y)
 if err != nil {
  log.Fatal(err)
 }

 for i := 0; i < 5; i++ {
  fmt.Printf("x=%.2f  y=%.3f  smoothed=%.3f\n", result.X[i], y[i], result.Y[i])
 }
}
x=0.00  y=0.000  smoothed=0.165
x=0.10  y=0.184  smoothed=0.214
x=0.20  y=0.290  smoothed=0.270
x=0.30  y=0.310  smoothed=0.331
x=0.40  y=0.314  smoothed=0.395

fraction is the most important tuning parameter: it controls the size of the local neighbourhood used at each point.

RangeEffectUse case
0.1-0.3Fine detailRapidly changing signals
0.3-0.5BalancedGeneral purpose
0.5-0.7Heavy smoothingNoisy data
0.7-1.0Very smoothTrend extraction

Choosing a model

  • Lowess (batch): the whole dataset fits in memory and you want every feature (intervals, cross-validation, GPU). Start here.
  • StreamingLowess: the dataset doesn’t fit in memory or arrives in chunks.
  • OnlineLowess: you need a smoothed value immediately as each point arrives (real-time).

See api.md, api-streaming.md, and api-online.md for the full reference of each.

Handling outliers

LOWESS can robustly handle outliers through iterative reweighting:

package main

import (
 "fmt"
 "log"

 "github.com/thisisamirv/lowess-project/bindings/go/fastlowess/v5"
)

func main() {
 x := []float64{1.0, 2.0, 3.0, 4.0, 5.0, 6.0}
 yWithOutlier := []float64{2.0, 4.0, 6.0, 50.0, 10.0, 12.0} // 50.0 is an outlier

 opts := fastlowess.DefaultOptions()
 opts.Fraction = 0.7
 opts.Iterations = 3                 // more iterations for outliers
 opts.RobustnessMethod = "bisquare"  // default, smooth downweighting
 opts.Outputs = []string{"weights"} // see which points were downweighted

 model, err := fastlowess.NewLowess(opts)
 if err != nil {
  log.Fatal(err)
 }
 defer model.Close()

 result, err := model.Fit(x, yWithOutlier)
 if err != nil {
  log.Fatal(err)
 }

 // Outliers will have low robustness weights
 for i, w := range result.RobustnessWeights {
  if w < 0.5 {
   fmt.Printf("Point %d is likely an outlier (weight: %.3f)\n", i, w)
  }
 }
}
Point 3 is likely an outlier (weight: 0.000)

Streaming mode

For datasets too large to fit in memory, stream them in fixed-size chunks with overlap:

package main

import (
 "fmt"
 "log"
 "math"

 "github.com/thisisamirv/lowess-project/bindings/go/fastlowess/v5"
)

func main() {
 n := 5000
 x := make([]float64, n)
 y := make([]float64, n)
 for i := 0; i < n; i++ {
  x[i] = float64(i) * 10.0 * math.Pi / float64(n-1)
  y[i] = math.Sin(x[i]/math.Pi) * math.Exp(-x[i]/30.0)
 }

 opts := fastlowess.DefaultStreamingOptions()
 opts.Fraction = 0.2
 opts.ChunkSize = 1000
 opts.Overlap = 100

 model, err := fastlowess.NewStreamingLowess(opts)
 if err != nil {
  log.Fatal(err)
 }
 defer model.Close()

 for i := 0; i < n; i += opts.ChunkSize {
  end := i + opts.ChunkSize
  if end > n {
   end = n
  }
  if _, err := model.ProcessChunk(x[i:end], y[i:end]); err != nil {
   log.Fatal(err)
  }
 }

 result, err := model.Finalize()
 if err != nil {
  log.Fatal(err)
 }
 fmt.Println("Smoothed", len(result.Y), "points in the final chunk")
}
Smoothed 100 points in the final chunk

Next steps

TopicLink
How LOWESS worksConcepts
All parameters explainedAPI Reference
Batch vs Streaming vs OnlineExecution Modes
Edge handlingBoundary Handling
Outlier handling in depthRobustness
Kernel functionsWeight Functions
Residual scale estimationScaling Methods
Automated parameter selectionCross-Validation
Per-observation weightsCustom Weights
Reconciling streaming chunk overlapsMerge Strategies
Running on the GPUGPU Backend