PHP - file_get_contents failed to open stream: Connection closed?

Question

Implementation (url is a valid and running url):

$html = file_get_contents($url);

I am programming a crawler in php and sometimes file_get_contents returns the following error:

failed to open stream: Connection closed

This doesn't always occur, so when it does it confuses me a tad. Would this be an error on my side or the website I am crawling side? Either way is it sensible to keep retrying until an error doesn't occur or is there a better way?

Answer 1

Try this way...

function url_get_contents ($Url) {
    if (!function_exists('curl_init')){ 
        die('CURL is not installed!');
    }
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL, $Url);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    $output = curl_exec($ch);
    curl_close($ch);
    return $output;
}

Answer 2

You need to create a stream for this

Read stream_context_create

<?php
// Create a stream 
$opts = array(
                'http'=>array(
                 'method'=>"GET",
                'header'=>"Accept-language: en\r\n" .
                "Cookie: foo=bar\r\n"
             )
       );

 $context = stream_context_create($opts);

 // Open the file using the HTTP headers set above
  $file = file_get_contents($url, false, $context);
?>

Answer 3

Use php CURL library http://php.net/manual/en/book.curl.php for better management of client requests.. file_get_contents() functions fails due to security restrictions on host server