I am in the process of conducting a time series analysis, a multivariate time series to be precise and before feeding the inputs to my LSTM model, I have scaled them. The metrics that I am using to evaluate my model are the loss and mean absolute error, both for the validation set.
I was able to achieve pretty good results with both my loss and MAE below 1. However, it occurred to me then that since my data has been scaled, it is in the range of say 1 to -1, therefore, these results are not surprising.
In essence, the question I have is, how do you evaluate a model, aimed to perform time series forecasts or even regression, when the data has been scaled? Isn't the loss going to be very low anyway due to the range that the data falls in?
Yes however, the function of the loss here is to usually compare the performance of one model versus another. As long as you are scaling models it on the same data this is useful for that.
However, if you want a better of idea of the un-scaled loss you can compute MAE then inverse scale the loss using the same scaler you fit originally. Finally, I don't recommend you using an error metric to judge "good results" for your model unless you have a pervious benchmark. You should really plot forecasted versus actual values on a graph