Getting all options trades for a given day on polygon.io

Question

Getting all options trades for a day via WebSocket

It's possible to get a record of all options trades in a day via the Trades WebSocket:

https://polygon.io/docs/options/ws_options_t

I've done this before and it works well. I start the script that collects the data before the trading day starts. I stop the WebSocket handler after the trading day has completed. I then serialize the data to JSON.

Getting all options trades for an arbitrary day via REST calls

What's a good approach to getting all the options trades for an arbitrary day in the past?

One approach I've considered is:

Get all tickers:

https://polygon.io/docs/options/get_v3_reference_tickers

Then, for each ticker, get all contracts:

https://polygon.io/docs/options/get_v3_reference_options_contracts

Then for each contract, get all trades for the day of interest:

https://polygon.io/docs/options/get_v3_trades__optionsticker

Python script implementing the above

import os

from polygon import RESTClient

api_key = os.environ['POLYGON_IO_API_KEY']

client = RESTClient(api_key=api_key)

import itertools

# ----------------------------------------------------------------------
# get all tickers
# ----------------------------------------------------------------------
gen = client.list_tickers(limit=1000)

ls = []

while(True):
    slice = itertools.islice(gen, 5)

    items = list(slice)

    if len(items) > 0:
        print([item.ticker for item in items])
        ls.extend(items)
    else:
        break

tickers = ls
# ----------------------------------------------------------------------
# for each ticker : get all contracts
# ----------------------------------------------------------------------

# 11:49
# 01:18
# 1 hour and 29 minutes

contracts_by_ticker = {}

print('Getting contracts')

for ticker in tickers:
    print(ticker.ticker)
    gen = client.list_options_contracts(underlying_ticker=ticker.ticker, as_of='2023-08-04', limit=1000)
    contracts_by_ticker[ticker.ticker] = list(gen)


# number of contracts
# 1,358,037

contracts = [item 
    for sublist in contracts_by_ticker.values() 
        for item in sublist]

number_of_contracts = len(contracts)

# contracts[0]
# ----------------------------------------------------------------------
# get all trades
# ----------------------------------------------------------------------

# 1:42

import time

time_a = time.time()

trades_by_contract = {}

i = 1

for contract in contracts:
    ls = list(client.list_trades(ticker=contract.ticker, timestamp='2023-08-04', limit=50000))

    if len(ls) > 0:
        trades_by_contract[contract.ticker] = ls
        
    elapsed = (time.time() - time_a) / 60

    contracts_per_minute = i / max(0.1, elapsed)
    
    number_of_remaining_contracts = number_of_contracts - i

    estimated_minutes = round(number_of_remaining_contracts / contracts_per_minute)

    print(f'{i:10} of {number_of_contracts:10}', f'{contract.ticker:30}', f'trades: {len(ls):10}', f'elapsed: {elapsed:10.2f}', f'estimated minutes: {estimated_minutes}' )

    i += 1
# ----------------------------------------------------------------------

trades = [
    trade
        for trades in trades_by_contract.values()
            for trade in trades
]

Questions

The above script appears to work. However, it seems that the approach will take a very long time. I estimate it might take over 24-48 hours to collect all the trades for a single day.

Is there a better way of getting all options trades for a particular day in the past?

Or is this the recommended approach?

Answer 1

You're correct in that you need to loop over each contract to get the trades. Then, you can loop over all trades and find the specific day you're interested in. You can, however use the Options Contracts endpoint alone to query all of the Contracts as of a specific date. Here's an example: https://api.polygon.io/v3/reference/options/contracts?as_of=2023-05-01&apiKey=

Once you have all of the Contracts on that date, you would loop through each contract on the Trades endpoint and paginate through that date's response:

https://api.polygon.io/v3/trades/<contract>?timestamp=2023-05-01&limit=50000&apiKey=

While we do not have a quicker method for fetching all trades now, we do have solutions in development.