Trying to slice a large css file Trying to slice a large css file (1,952,726 rows) to chances using Python

Question

When I run the following command, I get an error:

import pandas as pd
import os
import numpy as np

Endcustomers = "Resources/WESTCON_INTE_LTD_2024_01_GBP.csv"
Endcustomers_df = pd.read_csv(Endcustomers)
Endcustomers_df.head(3)

def split_csv_into_chunks(Endcustomers, 10000):
    # Read the CSV file into a pandas DataFrame
    df = pd.read_csv(Endcustomers)
    
    # Determine the total number of chunks needed
    total_chunks = len(df) // 500000 + 1
    
    # Split the DataFrame into chunks
    chunks = np.array_split(df, total_chunks)
    
    # Save each chunk as a separate CSV file
    for i, chunk in enumerate(chunks):
        chunk.to_csv(f"{input_file}_chunk_{i+1}.csv", index=False)
        print(f"Chunk {i+1} saved.")

Error: Cell In[16], line 1 def split_csv_into_chunks(Endcustomers,10000): ^ SyntaxError: invalid syntax

Can someone help resolve this please? Thank you

Answer 1

You have an error on the definition of the function, you are receiving two arguments "Endcustomers" and "10000", which "10000" can't be an argument.

Fix:

def split_csv_into_chunks(Endcustomers):

Also you are using a variable called "input_file" which doesn't exists in your context.

for i, chunk in enumerate(chunks):
    chunk.to_csv(f"INSERT_NAME_OF_FILE_chunk_{i+1}.csv", index=False)
    print(f"Chunk {i+1} saved.")

And you are not calling split_csv_into_chunks anywhere, do you have more code below?