Make cURL output STDERR to file (or string)

Question

We're trying to debug some cURL errors on the server, and I would like to see the STDERR log. Currently, all we can see for our error is "error code: 7" and that we can't connect to target server. We have contacted the host and made special rule to open the port we need and we're even ignoring the certificate for the time being.

Still, we can't connect. I need to debug this, but I can't see any pertinent information on my end.

The lines mentioning "VERBOSE" and "STDERR" are the most important, I think. Nothing is written to $curl_log. What am I doing wrong? Following the manuals logic, this should be correct...

PHP in use:

<?php
$curl = curl_init();
$curl_log = fopen("curl.txt", 'w');
$url = "http://www.google.com";

curl_setopt_array($curl, array(
    CURLOPT_URL             => $url,        // Our destination URL
    CURLOPT_VERBOSE         => 1,           // Logs verbose output to STDERR
    CURLOPT_STDERR          => $curl_log,   // Output STDERR log to file
    CURLOPT_SSL_VERIFYPEER  => 0,           // Do not verify certificate
    CURLOPT_FAILONERROR     => 0,           // true to fail silently for http requests > 400
    CURLOPT_RETURNTRANSFER  => 1            // Return data received from server
));

$output = fread($curl_log, 2048);
echo $output; // This returns nothing!
fclose($curl_log);

$response = curl_exec($curl);
//...restofscript...
?>

From PHP manual: http://php.net/manual/en/function.curl-setopt.php

CURLOPT_VERBOSE TRUE to output verbose information. Writes output to STDERR CURLOPT_STDERR An alternative location to output errors to instead of STDERR.

It is not a permission issue either, I have set file and script permissions to 777 on server side and my local client is windows and has never cared about permission settings (it's only for dev anyway).

Answer 1

You are making couple mistakes in your example:

1) you have to call curl_exec() prior to reading from the "verbose log", because curl_setopt() doesn't perform any action, so nothing can be logged prior to the curl_exec().

2) you are opening $curl_log = fopen("curl.txt", 'w'); only for write, so nothing could be read, even after you write to the file and rewind the internal file pointer.

So the correct shortened code should look like:

<?php
$curl = curl_init();
$curl_log = fopen("curl.txt", 'rw'); // open file for READ and write
$url = "http://www.google.com";

curl_setopt_array($curl, array(
    CURLOPT_URL             => $url,
    CURLOPT_VERBOSE         => 1,
    CURLOPT_STDERR          => $curl_log,
    CURLOPT_RETURNTRANSFER  => 1
));

$response = curl_exec($curl);

rewind($curl_log);
$output= fread($curl_log, 2048);
echo "<pre>". print_r($output, 1). "</pre>";
fclose($curl_log);

// ...

?>

NOTE: verbose log could be longer than 2048 bytes, so you could "fclose" the $curl_log after curl_exec() and then read the whole file with for example file_get_contents(). In that case, the point 2) should not be considered as mistake :-)

Answer 2

Putting al above answers together, I use this function to make a Curl Post Request with loggin to a file option:

function CURLPostRequest($url, array $post = NULL, array $options = array(), $log_file = NULL){
    $defaults = array(
            CURLOPT_POST => 1,
            CURLOPT_HEADER => 0,
            CURLOPT_URL => $url,
            CURLOPT_FRESH_CONNECT => 1,
            CURLOPT_RETURNTRANSFER => 1,
            CURLOPT_FORBID_REUSE => 1,
            CURLOPT_TIMEOUT => 4,
            CURLOPT_POSTFIELDS => http_build_query($post)
    );

    if (is_resource($log_file)){
        $defaults[CURLOPT_VERBOSE]=1;
        $defaults[CURLOPT_STDERR]=$log_file;
        $defaults[CURLINFO_HEADER_OUT]=1;
    }

    $ch = curl_init();
    curl_setopt_array($ch, ($options + $defaults));
    if( ! $result = curl_exec($ch)){
        throw new Exception(curl_error($ch));
    }

    if (is_resource($log_file)){

        $info = curl_getinfo($ch);

        if (isset($info['request_header'])){
            fwrite($log_file, PHP_EOL.PHP_EOL.'* POST Content'.PHP_EOL.PHP_EOL);
            fwrite($log_file, print_r($info['request_header'],true));
            fwrite($log_file, http_build_query($post));
        }

        fwrite($log_file, PHP_EOL.PHP_EOL.'* Response Content'.PHP_EOL.PHP_EOL);
        fwrite($log_file, $result.PHP_EOL.PHP_EOL);
    }

    curl_close($ch);
    return $result;
}

Hope this help to someone.

Answer 3

From php manual for function curl_setopt:

CURLOPT_FILE The file that the transfer should be written to. The default is STDOUT (the browser window).  

Answer 4

I needed to close the file before being able to read it, this worked for me:

$filename = 'curl.txt';
$curl_log = fopen($filename, 'w'); // open file for write (rw, a, etc didn't help)
curl_setopt($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch, CURLOPT_STDERR, $curl_log);        

$result = curl_exec($ch);

fclose($curl_log);      
$curl_log = fopen($filename, 'r'); // open file for read
$output= fread($curl_log, filesize($filename));
echo $output;

(PHP 5.6.0, Apache/2.2.15)

Answer 5

You should put

$output = fread($curl_log, 2048);
echo $output; // This returns nothing!
fclose($curl_log);

after $response = curl_exec($curl); otherwise, file is closed during curl is executing.

Answer 6

A bit late to the party, but this page still pops up high in Google, so let's go.

It seems that CURLOPT_VERBOSE doesn't log anything if CURLINFO_HEADER_OUT is also set to TRUE.

This is a know bug in PHP (#65348), and due to reasons they decided not to fix it.