when i tried to use goutte inside a while loop the goutte instance is only created once and now reiterated upon 20 times where as i want a new instance every loop. the result of the data filtered out is a repetition of the data on the first instance twenty times where as what i want is the seperate data on all the 20 pages.
while($count <=20) {
$new_url = $url .$count;
$check[] = $new_url;
//get a goutte object of each new url returned after each loop
$crawler = Goutte::request('GET', $new_url);
//get all text from a table data of class narrow
$results = $crawler->filter($lin)->each(function ($node, $i) {
return $node->text();
});
$pattern = 'tr>td.pu>a';
//get all the links inside table data of class a
$links = $crawler->filter($pattern)->each(function ($node, $i) {
$href = $node->extract(array('href')); // This is a DOMElement Object
return $href;
});
//filter the links for the needed one which is always greater than 30 characters
foreach($links as $link){
if(strlen($link[0]) > 30){
$p_links[] = $link;
}
}
for($i =0; $i<count($results)-3; $i++){
$content[] = ['comments' => $results[$i], 'links' => 'http://www.nairaland.com' . $p_links[$i][0]];
}
//add the data to an array
$data[] = $content;
$count++;
$crawler = null;
}
then i returned data outside the while loop
I was eventually able to solve this by moving the whole goutte code inside the loop to another function and then calling the function inside the loop. That worked as each goutte instance was created and used independently per function call inside the loop.