Getting a "hole" in Twitter data -- am I doing sth wrong?

17 views Asked by At

I’m looking at -and trying to retrieve- data for a specific search query on the full archive (premium) search API (V1) but I’m getting a weird "hole" of about 9 days of data from 7 to 16 January. Results for the rest of January up until now are apparently ok.

Parameters passed to the search endpoint are:

    'query' => '<a longish query string about 750 characters>',
    'fromDate' => '202301070000',
    'toDate' => '202301140830',
    'maxResults' => '500'

but apparently the data is missing from the count endpoint as well, since this is what I’m getting with a bucket=day granularity (this covers all of Jan up to now):

{
    "results": [{
        "timePeriod": "202301010000",
        "count": 525
    },
    
    >>>>> ...EVERYTHING FINE UP TO HERE <<<<<<
    
    {
        "timePeriod": "202301070000",
        "count": 15 <--- THIS IS A PARTIAL RESULT
    }, {
        "timePeriod": "202301080000",
        "count": 0
    }, {
        "timePeriod": "202301090000",
        "count": 0
    }, {
        "timePeriod": "202301100000",
        "count": 0
    }, {
        "timePeriod": "202301110000",
        "count": 0
    }, {
        "timePeriod": "202301120000",
        "count": 0
    }, {
        "timePeriod": "202301130000",
        "count": 0
    }, {
        "timePeriod": "202301140000",
        "count": 0
    }, {
        "timePeriod": "202301150000",
        "count": 0
    }, {
        "timePeriod": "202301160000",
        "count": 195 <--- ALSO A PARTIAL RESULT 
    }, 
    
    {
        "timePeriod": "202301170000",
        "count": 682
    }, 

    >>>>> ...FINE FROM HERE ON <<<<<<

],
    "totalCount": 10456,
    "requestParameters": {
        "bucket": "day",
        "fromDate": "202301010000",
        "toDate": "202301241720"
    }
}

for your enjoyment here is a chart of what I'm (not) getting.

enter image description here

I'm a bit weirded out -- also, premium API access is all but free and is paid upfront, you know.

1

There are 1 answers

1
qqg On

From further research and consultation on the Twitter dev fora, this is platform-wide. There may be possibly be a reindexing in a few days but as of now all searches return no results in the 2nd week of January 2023.