Data detrending in R

What is data detrending

updated 26th Feb, 2023

Detrending is a technique that serves as a powerful tool in the arsenal of the time series analyst, enabling us to remove the trend component of a signal and reveal the underlying patterns. The detrend function in the R programming language is a particularly elegant implementation of this technique.

Consider a simple example: we generate a time series of 100 data points, each of which is a value on the x-axis plus a random deviation drawn from a normal distribution with a mean of zero and a standard deviation of 10. When we plot this data, we see a clear upward trend due to the increasing x-value. However, this trend is not informative – it tells us nothing about the underlying structure of the data.

This is where the detrend function comes in. By applying this function to the time series data, we remove the trend component and are left with only the residuals. The trend component is typically modeled as a linear function, but other models can also be used.

How to perform detrending in R

It is worth noting that the detrend function is a base R function, and thus requires no additional packages to use. However, detrending may not always be the appropriate method for analyzing time-series data. Other techniques such as differencing or seasonal adjustment may be more suitable in certain cases.

Detrending in R was done using either linear model-based detrending or by using break points (bp) method. The former performs poorly for the samples with clear break points, while a breakpoint-based detrending performs much better for those cases. However, it would not work for short datasets or for very broad peaks.

Linear model-based method of detrending for the entire dataframe

br = 50
break.Points <- seq(from = br, to = dim(data)[1], by = br)
data.bp <- data
library(pracma)
for(i in 1:dim(data)[2]) {
  data.bp[, i] <- detrend(data[, i], tt = 'linear', bp = c())
}