R - Portfolio Analytics optimization not working

Question

im currently working on creating a minimum Variance portfolio and decided to use the function optimize.portfolio of the PortfolioAnalytics package. Unfortunately, when extracting the weights, all of them are NA, eventhough non of my returns do have any NA value which would be the only reason (from my piont of view) to cause resulting weights to be NA. My dataset consists of a multiple assets (+5000) each with 60 observations (monthly).

   library(PortfolioAnalytics)
   library(ROI)  
   #index returns is an xts object consisting of 3800 stock Ids(columns) and 60 observations in
   # monthly interval: To exemplifiy my problem, I set all values in index_returns to 1, to make 
   # sure that no NA values exist.
   index_returns
   any(is.na(index_returns)) # --> evaluates to FALSE
  port_spec <- portfolio.spec(assets =colnames(index_returns) )
  
  # Add a full investment constraint such that the weights sum to 1
  port_spec <- add.constraint(portfolio = port_spec, type = "full_investment")
  
  # Add a long only constraint such that the weight of an asset is between 0 and 1
  port_spec <- add.constraint(portfolio = port_spec, type = "long_only")
  
  # Add an objective to min portfolio variance
  port_spec <- add.objective(portfolio = port_spec, type = "risk", name = "var")
  
  # Solve the optimization problem
  opt <- optimize.portfolio(R = index_returns, trace=TRUE, portfolio = port_spec,optimize_method = "ROI")
  extractWeights(opt) #evaluates to NA for all assets

Does anyone know why this occurs and has any suggestion how to deal with this issue. I know that this optimsiation problem very likely faces invertibility issues due to far more columns than rows, but apart from this notion Im struggling to make any progress with my problem.
I highly appreciate any help!! Thanks in advance

Answer 1

Your optimization most likely fails because you have way more assets than observations. Then, as you correctly assumed, you can't obtain an inverse of the estimated covariance matrix.

To quote from "A Portfolio Optimization Approach with aLarge Number of Assets: Applications tothe US and Korean Stock Markets" available at: https://onlinelibrary.wiley.com/doi/epdf/10.1111/ajfs.12233

"Many attempts have been made to find an invertible estimator of the covariancematrix when N is larger than T. The pseudoinverse estimators of the covariancematrix are used by Sengupta (1983) and Pappas et al. (2010), and the shrinkageestimators of the covariance matrix are suggested by Ledoit and Wolf (2003). Ledoitand Wolf (2003) propose estimating the co variance matrix by an optimallyweighted average of two existing estimators: the sample covariance matrix and sin-gle-index covariance matrix."

So I'd suggest you take a look at the Ledoit-Wolf shrinkage method as a first step. The R-package RiskPortfolios and might also be useful, see https://joss.theoj.org/papers/10.21105/joss.00171