goutte http request not creating multiple instances

419 views Asked by At

when i tried to use goutte inside a while loop the goutte instance is only created once and now reiterated upon 20 times where as i want a new instance every loop. the result of the data filtered out is a repetition of the data on the first instance twenty times where as what i want is the seperate data on all the 20 pages.

   while($count <=20) {
        $new_url = $url .$count;
       $check[] = $new_url;
       //get a goutte object of each new url returned after each loop
        $crawler = Goutte::request('GET', $new_url);
       //get all text from a table data of class narrow
        $results = $crawler->filter($lin)->each(function ($node, $i) {

            return $node->text();
        });
    $pattern = 'tr>td.pu>a';
       //get all the links inside table data of class a
    $links = $crawler->filter($pattern)->each(function ($node, $i) {
        $href = $node->extract(array('href'));    // This is a DOMElement Object
            return $href;
    });
       //filter the links for the needed one which is always greater than 30 characters
foreach($links as $link){
    if(strlen($link[0]) > 30){
        $p_links[] = $link;
    }
}
   for($i =0; $i<count($results)-3; $i++){
        $content[] = ['comments' => $results[$i], 'links' => 'http://www.nairaland.com' . $p_links[$i][0]];
    }
       //add the data to an array
       $data[] = $content;
       $count++;
       $crawler = null;
    }

then i returned data outside the while loop

2

There are 2 answers

0
itsdenty On BEST ANSWER

I was eventually able to solve this by moving the whole goutte code inside the loop to another function and then calling the function inside the loop. That worked as each goutte instance was created and used independently per function call inside the loop.

1
Alexey Shokov On

You are using you own integration (Goutte in Lavavel), so please look at your Goutte::request() to find the reason.

Also, please include only related code in the future, to simplify understanding of a problem (I think most code inside the loop is not connected to the problem in this post, but maybe I'm wrong).