unable to load big .pcd files using pcl::io::loadPCDFile

Question

I am trying to load a big ointcloud (56 million points) using pcl. When I load it it gives the correct size but it put 19 million points at [0,0,0]. I check that these points are not located here using cloud compare and other software.

This is my working code (it compiles and runs as expected for me).

#include <pcl/io/pcd_io.h>
#include <pcl/point_types.h>
#include <pcl/common/io.h>
#include <pcl/common/common.h>

#include <string>
#include <iostream>
#include <fstream>

int main()
{
    std::string file_name;
    pcl::PointCloud<pcl::PointXYZ>::Ptr pcl_cloud (new pcl::PointCloud<pcl::PointXYZ>);
    pcl::PointCloud<pcl::PointXYZ>::Ptr cloud_final (new pcl::PointCloud<pcl::PointXYZ>);
    float num_points;

    std::cin>>file_name;
    int x=0;
    pcl::io::loadPCDFile(file_name+".pcd",*pcl_cloud);


    for (auto i = 0; i < pcl_cloud->size(); ++i)
    {
        auto point = pcl_cloud->at(i);
        if ((point.x==0)&&(point.y==0)&&(point.z==0)) //(Eigen::seq(0,2))
        // {
        {
            num_points+=1;
            continue;
        }
    }
    std::cout<<"pointcloud size: "<<pcl_cloud->size()<<" of which "<<num_points<<" are exactly [0,0,0].\n";
    return 0;
}

It returns: pointcloud size: 56579685 of which 1.67772e+07 are exactly [0,0,0]

The first part of the pointcloud is loaded correctly.

Answer 1

So I found a workaround that worked for me, still do not understand what went wrong.

When creating the .pcd file I first included all possible fields. By creating the pointcloud with only dimensions XYZ it worked.

Answer 2

When you use all possible fields, one point is 92 bytes large (3 * 4 + 10 * 8, according to your PCD header). During writing and reading PCD files, the data is partially processed as a binary blob, and this blob is 92*56579685=5205331020 bytes large (~5.2 billion). This overflows the value range of 32 bit int (~2.1 billion) and 32 bit unsigned int (~4.3 billion). You might be able to circumvent this by compiling PCL with a larger index type (https://github.com/PointCloudLibrary/pcl/blob/master/cmake/pcl_options.cmake#L96), or saving your data as a binary PCD instead of ASCII PCD, but it is also possible that there is a bug in PCL.
EDIT: I am also not completely sure whether the problem happens while reading, or already while writing the PCD file.