create lag variable of xts object using $ vs. [] notation

Question

I am trying to create a lagged vector within an xts object using the lag function. It works when defining the new vector within the xts object using $ notation (e.g. x.ts$r1_lag), but it does when defining the new variable using square brackets, i.e. xts[,"r1_lag"]. See code below:

library(xts)
x <- data.frame(date=seq(as.Date('2015-01-01'), by='days', length=100),
                runif(1e2), runif(1e2), runif(1e2))
colnames(x) <- c("date", "r1", "r2", "r3")

#the following command works
x.ts <- xts(x, order.by=x$date)
x.ts$r1_lag <- lag(x.ts$r1)
# but the following does not (says subscript is out of bounds)
x.ts <- xts(x, order.by=x$date)
x.ts[,"r1_lag"] <- lag(x.ts[,"r1"])

I need to use [] notation rather than $ notation to reference the vectors because if I want to run the lag transformation on vectors in more than one xts object (vectors within a list of multiple xts objects), I can't define the new vectors within the objects using $ notation, i.e. I cant define the new vectors using the notation in the below stylized loop:

for (i in letters) {
  for (j in variables) {
    macro.set.ts$i$paste(j,"_L1",sep="") <- lag(macro.set.ts[[i]][,j])
    macro.set.ts$i$paste(j,"_L2",sep="") <- lag(macro.set.ts[[i]][,j], 2)
    macro.set.ts$i$paste(j,"_L4",sep="") <- lag(macro.set.ts[[i]][,j], 4)
  }
}

Thanks!

Answer 1

You don't need to use [<-.xts. You can use merge instead:

for (i in letters) {
  for (j in variables) {
    # create all lags
    mst_ij <- macro.set.ts[[i]][,j]
    jL <- merge(lag(mst_ij), lag(mst_ij, 2), lag(mst_ij, 4))
    colnames(jL) <- paste(j, c("L1","L2","L4"), sep="_")
    # merge back with original data
    macro.set.ts[[i]] <- merge(macro.set.ts[[i]], jL)
  }
}

Answer 2

This should work:

x.ts <- merge(x.ts,lag(x.ts[,"r1"]))

You will then probably want to rename the last column that was added:

dimnames(x.ts)[[2]][5] <- "r1_lag"

This is the result:

> head(x.ts)
           date         r1           r2           r3             r1_lag      
2015-01-01 "2015-01-01" "0.23171030" "0.44174424" "0.3396816640" NA          
2015-01-02 "2015-01-02" "0.97292220" "0.74909452" "0.2793033421" "0.23171030"
2015-01-03 "2015-01-03" "0.52320743" "0.49288463" "0.0193637393" "0.97292220"
2015-01-04 "2015-01-04" "0.36574297" "0.69571803" "0.6411834760" "0.52320743"
2015-01-05 "2015-01-05" "0.37563137" "0.13841216" "0.3087215754" "0.36574297"
2015-01-06 "2015-01-06" "0.48089356" "0.32702759" "0.3967609401" "0.37563137"

> class(x.ts)
[1] "xts" "zoo"

Hope this helps.

Answer 3

The error is not related to lag function. You get an error because you try assign an xts object with another xts object. This example reproduces the error :

x.date= seq(as.Date('2015-01-01'), 
                        by = 'days' , length = 5)
x1 <- xts(data.frame(c1=runif(5)), order.by=x.date)
x2 <- xts(data.frame(c2=runif(5)), order.by=x.date)
x1[,'r2'] <- x2

## Error in `[<-.default`(`*tmp*`, , "r2", 
##   subscript out of bounds

I find this is coherent within xts logic, because xts are indexed objects. So it is better here to merge objects or join and conserve the indexed nature of your time series.

merge(x1,x2) 

This will cbind the 2 times series and fix any index problem. in fact, cbind is just a merge:

identical(cbind(x1,x2),merge(x1,x2)

That's said I think it is a kind of bug that this works for $<- operator and not with [<- operator.

Answer 4

I got the same output with:

x.ts <- cbind(x.ts,lag(x.ts[,"r1"]))

And

x.ts <- transform(x.ts, r1_lag = lag(x.ts[,'r1']))

But, be careful with the output. It may look the same but with an altered structure.