Why is Theil's U2 accuracy not the same in forecast package and DescTools?

423 views Asked by At

Today I was trying to use Theil's U2 from DescTools instead of forecast package. I am just wondering, why are both functions returning different results? As far as I am informed, there should be no difference.

library(forecast)
library(DescTools)

fc_df = data.frame(
fc1 = c(5565,5448,5164,5067,4997,5035,5168,5088,5162,4990,5018,5782),
fc2 = c(2565,2448,2164,2067,1997,1035,2168,2088,2162,1990,1018,2782)
)
act_df = data.frame(
act1 = c(9370,7980,6050,5640,6220,5740,6040,5130,5090,5210,4910,6890),
act2 = c(2900,2160,2400,2020,1630,1660,2210,1930,1960,1590,1730,2440)
)
# forecast 
ts_act <- ts(act_df, frequency = 12)
do.call(what = rbind, args = lapply(1:ncol(fc_df), function(x){
                               forecast::accuracy(fc_df[, x], ts_act[, x])}
                              ))
# DescTools ts 
TheilU(fc_df$fc1, ts_act[, 1])
TheilU(fc_df$fc2, ts_act[, 2])
1

There are 1 answers

1
Rob Hyndman On BEST ANSWER

Unfortunately, there are several statistics known as "Theil's U", partly because Theil himself used the same notation for different statistics in different papers.

Suppose the forecasts are stored in the vector f and the actuals are stored in the vector a, each of length n. Then the forecast package is returning a statistic based on relative changes.

fpe <- f[2:n]/a[1:(n-1)] - 1
ape <- a[2:n]/a[1:(n-1)] - 1
theil <- sqrt(sum((fpe - ape)^2)/sum(ape^2))

The DescTools package returns two types of Theil's U statistic. type=2 is

theil <- sqrt(sum((f-a)^2)/sum(a^2))

while type=1 is given by

theil <- sqrt(sum((f-a)^2/n))/(sqrt(sum(f^2)/n) + sqrt(sum(f^2)/n))