PyAlgoTrade custom CSV date error Invalid datetime. It must be bigger than that last one

Question

I am trying to add custom csv data into PyAlgoTrade using the addValuesFromCSV method, and I am getting this error:

2021-10-14 17:50:00 {'Open': 4886.0, 'High': 4888.0, 'Low': 4879.0, 'Close': 4883.0}
2021-10-14 17:55:00 {'Open': 4883.0, 'High': 4887.0, 'Low': 4874.0, 'Close': 4877.0}
2021-10-15 10:30:00 {'Open': 4910.0, 'High': 4922.0, 'Low': 4901.0, 'Close': 4907.0}
---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
/var/folders/60/1yz0pnwd2jv15p28bffgz8p40000gn/T/ipykernel_9157/639233146.py in <module>
      5 feed = csvfeed.Feed("Date", "%Y-%m-%d %H:%M:%S")
      6 feed.addValuesFromCSV("data/5min/fcpo_dec_jan_contract.csv")
----> 7 for dateTime, value in feed:
      8     print(dateTime, value)

~/opt/anaconda3/envs/ml/lib/python3.9/site-packages/pyalgotrade/feed/__init__.py in feed_iterator(feed)
     29     try:
     30         while not feed.eof():
---> 31             yield feed.getNextValuesAndUpdateDS()
     32     finally:
     33         feed.stop()

~/opt/anaconda3/envs/ml/lib/python3.9/site-packages/pyalgotrade/feed/__init__.py in getNextValuesAndUpdateDS(self)
     88                     ds = self.createDataSeries(key, self.__maxLen)
     89                     self.__ds[key] = ds
---> 90                 ds.appendWithDateTime(dateTime, value)
     91         return (dateTime, values)
     92 

~/opt/anaconda3/envs/ml/lib/python3.9/site-packages/pyalgotrade/dataseries/__init__.py in appendWithDateTime(self, dateTime, value)
    136 
    137         if dateTime is not None and len(self.__dateTimes) != 0 and self.__dateTimes[-1] >= dateTime:
--> 138             raise Exception("Invalid datetime. It must be bigger than that last one")
    139 
    140         assert(len(self.__values) == len(self.__dateTimes))

Exception: Invalid datetime. It must be bigger than that last one

Here is my data after 2021-10-15 10:30:00

Date	Open	High	Low	Close
2021-10-15 10:30:00	4910	4922	4901	4907
2021-10-15 10:35:00	4907	4907	4891	4896
2021-10-15 10:40:00	4896	4902	4886	4893
2021-10-15 10:45:00	4893	4901	4887	4897
2021-10-15 10:50:00	4897	4897	4879	4882
...................	....	....	....	....
2021-11-15 17:35:00	4987	4989	4984	4985
2021-11-15 17:40:00	4985	4987	4975	4978
2021-11-15 17:45:00	4978	4990	4978	4987
2021-11-15 17:50:00	4986	4988	4970	4972
2021-11-15 17:55:00	4973	4975	4963	4965

Below is my code for the ingestion:

from __future__ import print_function

from pyalgotrade.feed import csvfeed

feed = csvfeed.Feed("Date", "%Y-%m-%d %H:%M:%S")
feed.addValuesFromCSV("data/5min/fcpo_dec_jan_contract.csv")
for dateTime, value in feed:
    print(dateTime, value)

I don't understand why is it saying that the last date is smaller then previous one. Help would be much appreciated

Thank you

Answer 1

This could be due to the bars in the wrong order of timestamps

Have you tried checking if the data is sorted? maybe some issue with the datasource, maybe you can verify that once?