Hello i am trying to scrap search results from youtube like channel usernames or channel ids. I tried using Youtube Data api and even did not crossed thousand. I tried keyword travel
` request = youtube.search().list(
q=keyword,
type='channel',
part='id',
maxResults=50 # Adjust this based on your needs, maximum is 50`
in json result i see ` { 'kind': 'youtube#searchListResponse',
'etag': 'yfBPaJGKNXfioCSMoT6xiCzESUk',
'nextPageToken': 'CDIQAA',
'regionCode': 'PK',
'pageInfo': {'totalResults': 25856, 'resultsPerPage': 50}
}
`
where i see total results are in thousands. I tried going to next page with nextPageToken and after some hundred results it stoped and this totalResults is different on every page. I was only able to get 500 channels id in file.
I tried using selenium to scrap but was only able to scrap 600-700 channels. I tried google search results and was only to scrap few hundreds. How to acheive my scraping goal of thousands. If i can do it using api it is be better but if there is any other solution can you please exaplain how to solve this issue.