Generating long-term forecasts including prophet and temporal aggregation (thief)

Question

I just started to use {fable} and the {tidyverts} family of tools and so far it's going quite well.

I'm currently interested in producing long term probabilistic forecasts from daily data (with a monthly or quarterly resolution being fine or preferred). My understanding is, that temporal aggregation could help reduce model uncertainty and also propagate known daily influences (especially holiday effects) to the e.g. quarterly level and improve accuracy that way.

For the daily data I plan on using prophet + covariates, for the higher aggregates (monthly to yearly) exponantial smoothing seems appropriate.

While I wonder if this approach seems generally promising, I am not quite sure how to structure the forecast problem using {thief} to arrive at probabilistic forecasts.

PS: I found this helpful post for hourly data, but I'm running into problems implementing it for daily data (e.g. creating meaningful aggregations and combining the forecasts): https://stats.stackexchange.com/questions/352121/how-to-forecast-hourly-as-well-as-daily-data-in-r

Answer 1

To produce probabilistic forecasts using {thief} you'll need to convert the fable into a {forecast} style forecast object. The basic minimal structure of a forecast object (and how it can be produced from a fable distribution, which can be obtained for prophet via {fable.prophet}) is:

library(forecast)
library(distributional)
dist <- dist_normal(1:10)

structure(
  list(
    # A time series of the forecast means
    mean = ts(mean(dist)),
    # A matrix time series of the forecast interval's upper bound
    upper = ts(cbind("80%" = quantile(dist, 0.90), "95%" = quantile(dist, 0.975))),
    # A matrix time series of the forecast interval's lower bound
    lower = ts(cbind("80%" = quantile(dist, 0.10), "95%" = quantile(dist, 0.025))),
    # A vector containing forecast interval levels in order of appearance
    level = c(80, 95)
  ),
  class = "forecast"
)
#>    Point Forecast      Lo 80     Hi 80       Lo 95     Hi 95
#>  1              1 -0.2815516  2.281552 -0.95996398  2.959964
#>  2              2  0.7184484  3.281552  0.04003602  3.959964
#>  3              3  1.7184484  4.281552  1.04003602  4.959964
#>  4              4  2.7184484  5.281552  2.04003602  5.959964
#>  5              5  3.7184484  6.281552  3.04003602  6.959964
#>  6              6  4.7184484  7.281552  4.04003602  7.959964
#>  7              7  5.7184484  8.281552  5.04003602  8.959964
#>  8              8  6.7184484  9.281552  6.04003602  9.959964
#>  9              9  7.7184484 10.281552  7.04003602 10.959964
#> 10             10  8.7184484 11.281552  8.04003602 11.959964

^{Created on 2020-12-10 by the reprex package (v0.3.0)}

You can then use these forecast structures with thief::reconcilethief() to reconcile them.

That said, temporal (and cross-temporal) reconciliation is currently being developed for {fable}. The relevant development issue is here: https://github.com/tidyverts/fabletools/issues/59 There are some complexities with reconciling daily or sub-daily temporal to monthly and annual hierarchies which have not yet been implemented, but the prototype currently works well for >monthly bottom levels. This recent presentation at ISF2020 introduces temporal reconciliation with {fable}: https://www.youtube.com/watch?v=6D7rNHZ5E-Q&t=1120

It's worth noting that the intervals from {thief} are simply the sum of intervals from higher temporal frequencies. These intervals are not optimal, and so will differ from the optimal probabilistic forecast distributions that will be given by {fable}.