Workaround for 60 maximum results limit from Google Maps Platform

Question

i've been working with googleway packages to retreive information using place_type key, but i've been stuck with the 60 results limit restriction.

I'm trying a slightly different approach to deal qith this restriction: divide an conquer. In my work, all the spatial analysis is performed using QGIS and one idea pops up. create a buffer area around certain coordinates; lets say a radius of 1km. Then using that buffered area, i apply a hex-bin tessellation to get a set of centroid coordinates that can be used to complete the whole area (1km radius) using small chunks of 60 results (let's say 100mts radius query using googleway packages). Graphically the intuition is as shown on the image attached.

In order to retrieve the 60 results i'm using the excelent solution provided here and everthings is fine when i perform singular searchs. Now i'm trying to use recursively a set of coordinates by just adding a for-loop at the very begining of the code but i doesn't work. I'm not much a programmer (as a matter of fact, i'm a sociologist) and i dont really know what i'm doing wrong. Could someone point me in the right direction please?

Thanks in advance and best regards from Guatemala

here my plain text coordinates file:

coords
"14.5446147628533, -90.84266666418"
"14.5538523714673, -90.84266666418"

And here my code:

###Preamble packages##
library(tidyverse)
library(googleway)
### load coordinates, plain text
dfCoords <- read.csv("~/coords.txt", sep="")
##Added For-loop begin##
for (i in dfCoords$coords) {
#### Original script begin ###
place_type <- "store" 
key <- c("API Key") 
radius <- 100 
location <- i

format_res <- function(res) {
  setNames(
    cbind(
      googleway::access_result(res, "coordinates"),
      googleway::access_result(res, "place_name")
    )
    , c("lat", "long", "name")
  )
}

do_search <- function(place_type, key, location, radius, page_token = NULL) {
  
  google_places(
    place_type = place_type,
    location = location,
    key = key,
    radius = radius,
    page_token = page_token
  )
}

full_search <- function(place_type, key, location, radius) {
  
  counter <- 0
  
  page_token <- NULL ## can start on NULL because it means we're doing the first query
  is_another_page <- TRUE 
  
  
  while( is_another_page ) {
    
    res <- do_search(place_type, key, location, radius, page_token)
    
    if( res$status == "OK" ) { ## check a valid result was returned
      
      if( counter == 0 ) {
        df <- format_res( res )
      } else {
        df <- rbind(df, format_res( res ) )
      }
      
      counter <- counter + 1
    }
    
    page_token <- res[["next_page_token"]]
    is_another_page <- !is.null( page_token )
    Sys.sleep(3)  ## Sleep the function before the next call because there's a time limit
  }
  return(df)
}

df <- full_search(place_type, key, location, radius)

##Original script ends

}
##Added for loop end

str( df )

intuition of the workaround

Answer 1

1. You only need to loop over the locations and call the functions from inside the loop (otherwise you are creating & defining the functions in each iteration)

2. I've added the place_id to the results in format_res() so you get the unique place ids. You'll need this when processing the results because even though you specify a radius, google will still give you results outside this value.

3. You need to assign the results of each iteration of the loop to an object. I've created a list lst_results for this

4. The two example coordinates you've given don't produce any results, so I've added some error handling to account for ZERO_RESULTS returned from google. And I've added a third coordinate pair to show you it working.

Here's the full updated code

library(googleway)

format_res <- function(res) {
  setNames(
    cbind(
      googleway::access_result(res, "coordinates"),
      googleway::access_result(res, "place_name"),
      googleway::access_result(res, "place")       ## store the unique place_id as well
    )
    , c("lat", "long", "name", "place_id")
  )
}

do_search <- function(place_type, key, location, radius, page_token = NULL) {
  
  google_places(
    place_type = place_type,
    location = location,
    key = key,
    radius = radius,
    page_token = page_token
  )
  
}

full_search <- function(place_type, key, location, radius) {
  
  counter <- 0
  
  page_token <- NULL ## can start on NULL because it means we're doing the first query
  is_another_page <- TRUE 
  
  ## initialise a data.frame to store the results
  df <- data.frame(
    lat = vector("numeric", 0L)
    , long = vector("numeric", 0L)
    , name = vector("character", 0L)
    , place_id = vector("character", 0L)
    )
  
  while( is_another_page ) {
    
    res <- do_search(place_type, key, location, radius, page_token)
    
    if( res$status == "OK" ) { ## check a valid result was returned
      
      if( counter == 0 ) {
        df <- format_res( res )
      } else {
        df <- rbind(df, format_res( res ) )
      }
      
      counter <- counter + 1
    } else {
      ## print a message for not-OK results
      print(paste0(res[["status"]], " for ", paste0(location, collapse = ", ") ))
    }
    
    page_token <- res[["next_page_token"]]
    is_another_page <- !is.null( page_token )
    Sys.sleep(3)  ## Sleep the function before the next call because there's a time limit
  }
  return(df)
}

## I've added a 3rd example that actually has results
dfCoords <- data.frame(
  coords = c("14.5446147628533, -90.84266666418" ,"14.5538523714673, -90.84266666418", "-37.816660, 144.967092")
)

key <- secret::get_secret("GOOGLE")
place_type <- "store" 
radius <- 100 

## create a list to store the results
lst_results <- vector("list", length = nrow(dfCoords))
## Using a list will be more efficient that `rbind`-ing a data.frame in each iteration

## loop through the indexes of the coordinates
## this wy we can assign the results to the correct index of the list
for (i in 1:nrow(dfCoords) ) {
  
  location <- dfCoords[i, "coords"]
  
  ## the coordiantes must be a numeric vector
  location <- as.numeric(strsplit(location, ",")[[1]])

  
  lst_results[[ i ]] <- full_search(
    place_type = place_type
    , key = key
    , location = location
    , radius = radius
    )
}

lapply(lst_results, head)

# [[1]]
# [1] lat  long name
# <0 rows> (or 0-length row.names)
# 
# [[2]]
# [1] lat  long name
# <0 rows> (or 0-length row.names)
# 
# [[3]]
#         lat     long                                          name                    place_id
# 1 -37.81681 144.9665           StayCentral Flinders Lane Melbourne ChIJmy5Y5YxD1moRwnnrXIAiejM
# 2 -37.81601 144.9665 EB Games / ZiNG Pop Culture - Swanston Street ChIJz6n71LVC1moR-wgn04JtBjk
# 3 -37.81666 144.9668                     Tiffany Pollard Jewellery ChIJ45afhLZC1moRnyg_JBIEf2o
# 4 -37.81666 144.9668                                 dead & buried ChIJx_udg7ZC1moR2Kw-kXTvRIw
# 5 -37.81670 144.9667                          Citizen Watch Repair ChIJtW1Cx8lC1moRxJsUpo14NAY
# 6 -37.81671 144.9669                            Paris in Melbourne ChIJR_J5hLZC1moRxZ7EIUb5ZQw