Get SerpApi search results into CSV files using Python

Question

I'm fairly new and in the middle of learning how to using API for collecting data and having difficulties when getting the results. I entered the code below and wanted to get the results in CSV file.

from serpapi import GoogleSearch

params = { "api_key": "***", "device": "desktop", "engine": "google_jobs", "google_domain": "google.com", "q": "Engineer", "hl": "en", "gl": "au", "location": "Australia", }

search = GoogleSearch(params) results = search.get_dict()

I found that using pandas is one of the solutions but I can't find the postings with more or less the same case as I experienced. What steps should I go through to get the results in the CSV file?

Answer 1

You're wanting something like this:

from serpapi import GoogleSearch
import pandas as pd

params = { "api_key": "api key here", 
          "device": "desktop", 
          "engine": "google_jobs", 
          "google_domain": "google.com", 
          "q": "Engineer", 
          "hl": "en", 
          "location": "Australia"}

search = GoogleSearch(params) 

results = search.get_dict()

df = pd.DataFrame(results['jobs_results'])

df.to_csv('jobs.csv', index=False)

# df for me: 
                                               title  ...                                          thumbnail
0                                  Software Engineer  ...                                                NaN
1              Mid/Senior Embedded Software Engineer  ...                                                NaN
2                 Principal Product Support Engineer  ...  https://encrypted-tbn0.gstatic.com/images?q=tb...
3                              Mechanical Engineer I  ...  https://encrypted-tbn0.gstatic.com/images?q=tb...
4                       Senior Applications Engineer  ...                                                NaN
5  High Performance Computing (HPC) System Suppor...  ...                                                NaN
6                               Sr Big Data Engineer  ...                                                NaN
7                           Senior Software Engineer  ...                                                NaN
8   Site Reliability Engineer -Full Time (Australia)  ...  https://encrypted-tbn0.gstatic.com/images?q=tb...
9                        ForgeRock Platform Engineer  ...  https://encrypted-tbn0.gstatic.com/images?q=tb...

[10 rows x 11 columns]

It wasn't returning any results with "gl": "au". I think this is because this feature is not available within Australia (source: I live in Australia, I have never seen this feature, and seem to not be able to use it). It does, however, seem to be able to access Australian job listings, despite it not being accessible here.