os.path.isfile returns True even though specified file does not exist at working directory

Question

I have a project that i wrote with OOP on separated modules. First module that gets imported and called by an object is "DataLibrary".

• DataLibrary module's data source is an .xlsx file which contains a table.
• I use pandas.read_excel to read it and .to_csv to convert it into a .csv file.
• Then i use pandas.read_csv to use the file in pandas (filtering columns, getting rows etc.)

To make sure that .csv file exists in working directory i use try-except-else statements. They do these steps i wrote above.

import pandas
import os.path

class DataLibrary:
    def __init__(self):
        try:
            os.path.isfile("iata-codes.csv")
        except FileNotFoundError:
            self.data_xlsx = pandas.read_excel("iata-codes.xlsx", "Sheet1", dtype=str, index_col=None)
            self.data_xlsx.to_csv("iata-codes.csv", encoding="utf-8", index=False)
            self.df = pandas.read_csv("iata-codes.csv")
        else:
            self.df = pandas.read_csv("iata-codes.csv")

The issue is, even though i know that .csv file does not exist at working directory, try statement returns true which skips except statement(where .csv file supposed to be created at) and jumps into else statement where it fails on pandas.read_csv because like i said, there is no such .csv file exists at working directory.

By the way, i already checked the working directory of DataLibrary with os.getcwd and it is my project's directory. So i didn't use relative/absolute file path in my code because .py .xlsx and .csv files are all at the same folder.

I believe it is due to this os.path.isfile statement. When i searched about it, i saw that even though the file does not exist, it returns True if the file is already open in memory. I did use this .csv file before in a different project folder and i didn't use "with" statement to close it after use. Maybe i should clear my IDE's currently open files memory to solve this problem. Is there any way to do that?

This is a related answer i found, but it doesn't explain what to do if file is still open in memory : https://stackoverflow.com/a/76055876/23261968

Answer 1

Try something like this:

import pandas as pd
import os

path = '{your file directory}'
file = '{your filename}'

filepath = os.path.join(path, file)

try:
    # Check if directory exists
    if os.path.exists(filepath):
        if file[-4:] == '.csv':
            df = pd.read_csv(filepath, **kwargs)
            
        elif file[-5:] == '.xlsx':
            df = pd.read_excel(filepath, **kwargs)

Answer 2

Many thanks to Klaus D. on os.path.isfile() returns True or False, which has no effect on try statement. That is why except statement does not catch any FileNotFoundError. So, Else statement gets executed and since there is no .csv file at working directory, it causes FileNotFoundError that fails my program.

To solve it, i revised my code with a simple if-else statement that checks os.path.isfile. When it returns False (because .csv file does not exist at working directory), else statement creates .csv file by reading .xlsx file and converts it into .csv file and reads it in the end.

import pandas
import os.path

class DataLibrary:
    def __init__(self):
        if os.path.isfile("iata-codes.csv"):
            self.df = pandas.read_csv("iata-codes.csv")
        else:
            self.data_xlsx = pandas.read_excel("iata-codes.xlsx", "Sheet1", dtype=str, index_col=None)
            self.data_xlsx.to_csv("iata-codes.csv", encoding="utf-8", index=False)
            self.df = pandas.read_csv("iata-codes.csv")