Detect any unwanted characters and number of lines in a file using PHP

Question

I have a problem I can't resolve. And I can't have any feedback since I can only run the script directly on a server, without any possible debug.

My files are transfered through FTP to a server and I have to read them one by one to place them on a SQL server database.

My files looks like this :

2024-02-22 19:50:44
17.90
50.40
0.0
0.0
0.0
0.0
0.0
4.17

If I run my PHP code like this, no problem:

foreach ($listWeathStatV2 as $weathStat) {
    $fileToRead = "/data/DB_" . $weathStat . ".txt";
    $file = fopen(__DIR__ . $fileToRead, "r") or die("Unable to open the file!");
    rewind($file);
    while(!feof($file)) {
            $heure       = fgets($file);          // Store date value
            $temp        = fgets($file);          // Store temp value
            $humidity    = fgets($file);          // Store hum value
            $pressure    = fgets($file);          // Store pressure value
            $pluvio      = fgets($file);          // Store pluvio value
            $anemo       = fgets($file);          // Store anemo value
            $gir         = fgets($file);          // Store gir value
            $lumino      = fgets($file);          // Store lumino value
            $battLvl     = fgets($file);          // Store battery level value

            $query    = "INSERT INTO " . $weathStat . " (heure, valTemp, valHumid, valPressure, valPluvio, valAnemo, valGir, valLumino, battLevel) VALUES ('$heure', '$temp', '$humidity', '$pressure', '$pluvio', '$anemo', '$gir', '$lumino', '$battLvl')";
            $result   = mysqli_query($connect, $query) or die("Errant query:  ".$query);
        fclose($file);

        $file= fopen(__DIR__ . $fileToRead, "w");
        rewind($file);
        file_put_contents($fileToRead, "");
        fclose($file);
    }
}

The thing is that sometimes, my files look like this:

2024-02-22 19:55:27
27.19
0.0

0.0&#¤
0.0
0.0
4.07

or

2024-02-22 19:55:27
27.19
0.0
0.0
0.0
0.0
0.0
0.0

So I modified my code to convert the file into a string (using file_get_contents) and only send the data if there are no unwanted characters (using preg_match) or if the number of lines is not correct :

foreach ($listWeathStatV2 as $weathStat) {
    $fileToRead = "/data/DB_" . $weathStat . ".txt";
    $file = fopen(__DIR__ . $fileToRead, "r") or die("Unable to open the file!");
    $fileToString = file_get_contents($file);
    if ($fileToString === false) {
        echo "An error occurred while reading the file.";
    }
    else if (fgets($file) != "" && !preg_match('/^£$%&*()}{@#~?><>,|=_+¬¤ÇÓü]/', $fileToString) && (substr_count($fileToString, "\n" ) == 8)) {
        rewind($file);
        while(!feof($file)) {
            $heure       = fgets($file);          // Store date value
            $temp        = fgets($file);          // Store temp value
            $humidity    = fgets($file);          // Store hum value
            $pressure    = fgets($file);          // Store pressure value
            $pluvio      = fgets($file);          // Store pluvio value
            $anemo       = fgets($file);          // Store anemo value
            $gir         = fgets($file);          // Store gir value
            $lumino      = fgets($file);          // Store lumino value
            $battLvl     = fgets($file);          // Store battery level value

            $query    = "INSERT INTO " . $weathStat . " (heure, valTemp, valHumid, valPressure, valPluvio, valAnemo, valGir, valLumino, battLevel) VALUES ('$heure', '$temp', '$humidity', '$pressure', '$pluvio', '$anemo', '$gir', '$lumino', '$battLvl')";
            $result   = mysqli_query($connect, $query) or die("Errant query:  ".$query);
        }
        fclose($file);

        $file= fopen(__DIR__ . $fileToRead, "w");
        rewind($file);
        file_put_contents($fileToRead, "");
        fclose($file);
    }
}

When I run this code, the only thing I can tell is that it can send data if the file is correct, but it blocks the rest of the script if there any unwanted characters and then, remaining files coming after and not treated. Can someone tell me what's wrong in this code ?

Thank you so much for your help and advices. Laurent

Answer 1

If you read the file with file_get_contents(), you don't need to read it also with fgets().

In my code below I read it once. I check if it contains anything other than numbers and whitespace, and skip the file if that happens.

Then I split the file into lines at newline characters, discarding any empty lines. If it contains the correct number of lines, I assign each line to the corresponding variable.

Then I used a prepared statement to substitute the variables into the query, instead of direct variable interpolation.

foreach ($listWeathStatV2 as $weathStat) {
    $fileToRead = __DIR__ . "/data/DB_" . $weathStat . ".txt";
    $fileToString = file_get_contents($fileToRead);
    if ($fileToString === false) {
        echo "An error occurred while reading $weathStat.";
        continue;
    }
    if (preg_match('/[^-:.\s\d]/', $fileToString)) {
        echo "The file $weathStat contains invalid characters";
        continue;
    }

    $lines = array_filter(preg_split('/[\r\n]+/', $fileToString)); // Split file into lines
    if (count($lines) != 9) {
        echo "The file $weathStat doesn't have 9 lines.";
        continue;
    }
        
    [$heure, $temp, $humidity, $pressure, $pluvio, $anemo, $gir, $lumino, $battLvl] = $lines;
    $stmt = mysqli_prepare($connect, "INSERT INTO $weathStat (heure, valTemp, valHumid, valPressure, valPluvio, valAnemo, valGir, valLumino, battLevel) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)");
    myqli_stmt_bind_param($stmt, "sffffffff", $heure, $temp, $humidity, $pressure, $pluvio, $anemo, $gir, $lumino, $battLvl);
    mysqli_stmt_execute($query);
    file_put_contents($fileToRead, "");
}