interactive brokers tick by tick data

Question

I am trying to get one day of tick data from interactivebrokers/TWS this is the function i wrote:

def ticker_data(ticker, start, end='',numberOfTicks=1000, Exchange='SMART', Currency='USD', ip='127.0.0.1', port=7496, clientId=1):
ib = IB()
ib.connect(ip, port, clientId=clientId)

contract = Stock(ticker, Exchange, Currency)

real_end = start
df = pd.DataFrame()

while end != real_end:
    ticks = ib.reqHistoricalTicks(contract,
                                  startDateTime=real_end,
                                  endDateTime=end,
                                  numberOfTicks=numberOfTicks,
                                  whatToShow='TRADES',
                                  useRth=True)

    df = pd.concat([df, util.df(ticks)], ignore_index=True)
    print("Latest time in DataFrame:", df.iloc[-1]['time'])
    print("Updated real_end_1:", real_end)
    print("End time:", end)
    print("length of recived_data:", len(ticks))
    print()


    real_end = datetime.strptime(str(df.iloc[-1]['time']), "%Y-%m-%d %H:%M:%S%z").strftime("%Y%m%d %H:%M:%S")#+' US/Eastern'

ib.disconnect()
return df

this is my way of calling it:

from functions.simple_ib_data import ticker_data
from datetime import datetime, timedelta

import pandas as pd

start = (datetime.now() - timedelta(days=1)).replace(hour=9, minute=30, second=0, microsecond=0).strftime("%Y%m%d %H:%M:%S")
end = (datetime.now() - timedelta(days=1)).replace(hour=23, minute=59, second=59, microsecond=0).strftime("%Y%m%d %H:%M:%S")

data = ticker_data(ticker="NCLH", start=start, end=end)
print(data)

so i thought the it will work like so: i get 1000 of tick data then take time from the latest tick recived make it the start for new request and repeat until the end is reached

the Problem: i think there is one minute witch's start time is same as its end time so i am running into this "cicle" where time from latest tick is same as time from the first so the output looks like this:

Latest time in DataFrame: 2023-12-01 14:33:35+00:00 Updated real_end_1: 20231201 09:30:00 End time: 20231201 23:59:59 length of recived_data: 1007

Latest time in DataFrame: 2023-12-01 14:33:35+00:00 Updated real_end_1: 20231201 14:33:35 End time: 20231201 23:59:59 length of recived_data: 1007

Latest time in DataFrame: 2023-12-01 14:33:35+00:00 Updated real_end_1: 20231201 14:33:35 End time: 20231201 23:59:59 length of recived_data: 1007

(and that repeating forever)

i am using ib_insync

any help is greatly apriciated.

Answer 1

IBAPI is a multiprocess system. It uses 2 threads. 1 to send request to IBAPI, another one is to receive data. Both operations are taking place in concurrent manner. So, you have to send the first request & wait for the data to be received & get the last timestamp from the data received & then you can get the new start time from that last received data. You cant achieve this using While loop. you will have to go for a switching mechanism where you will send request and end the first request using some boolean operation and once the data is received, you can go for next call. So, you will have to relook into your strategy. While loop will endup in blocking thread. You have to split the program & execute it.