Can you use read_feather without pyarrow package?

102 views Asked by At

I need to load a feather file in python, preferably with pandas, so I tried to use read_feather, but came across the error

/pandas/io/feather_format.py", line 21, in _try_import
    raise ImportError("pyarrow is not installed\n\n"

And indeed, the codebase I am working in does not have pyarrow integrated. When reading https://pandas.pydata.org/docs/reference/api/pandas.read_feather.html, it appears pyarrow is needed for certain dtype backends, but I am using the default settings so I am not sure why it is needed.

1

There are 1 answers

7
Talha Tayyab On BEST ANSWER

We cannot read_feather without pyarrow package because feather IO functionality that is used in pandas is implemented in pyarrow, and thus pandas needs that library to be able to read or write feather files.

from pyarrow import feather

pa_table = feather.read_table(
            handles.handle, columns=columns, use_threads=bool(use_threads)
        )

Even for dtype_backend == "numpy_nullable"

if dtype_backend == "numpy_nullable":
            from pandas.io._util import _arrow_dtype_mapping

            return pa_table.to_pandas(types_mapper=_arrow_dtype_mapping().get

Link to source code: https://github.com/pandas-dev/pandas/blob/v2.1.3/pandas/io/feather_format.py#L72-L148