'float' object is not iterable typerror

Question

I've written a script that takes a large excel spreadsheet of data and strips away unwanted columns, rows that contain zero values in particular columns and then saves out to a csv. The piece that I'm stuck on is I'm also trying to remove rows that are missing cells. The way I was trying this was by way of:

for each_row in row_list :
    if not all(map(len, each_row)) :
        continue
    else :
        UICData.append(row_list)

But this isn't working correctly as I'm getting the error:

File "/Users/kenmarold/PycharmProjects/sweetCrude/Work/sweetCrude.py",
line 56, in PrepareRawData
if not all(map(len, each_row)) :
TypeError: 'float' object is not iterable

I'm not exactly sure how to resolve this, what's the way forward on this? I've also attached the full script below.

#!/usr/bin/env python3
import os
import sqlite3
import csv
import unicodecsv
from datetime import date
from xlrd import open_workbook, xldate_as_tuple
from xlwt import Workbook

orig_xls = 'data/all_uic_wells_jun_2016.xls'
temp_xls = 'data/temp.xls'
new_csv = 'data/gh_ready_uic_well_data.csv'
temp_csv = 'data/temp.csv'

input_worksheet_index = 0             # XLS Sheet Number
output_workbook = Workbook()
output_worksheet = output_workbook.add_sheet('Sweet Crude')
lat_col_index = 13
long_col_index = 14

#### SELECT AND FORMAT DATA

def PrepareRawData(inputFile, tempXLSFile, tempCSVFile, outputFile):

    # 0 = API#              # 7 = Approval Date
    # 1 = Operator          # 13 = Latitude
    # 2 = Operator ID       # 14 = Longitude
    # 3 = Well Type         # 15 = Zone

    keep_columns = [0, 1, 2, 3, 7, 13, 14, 15]

    with open_workbook(inputFile) as rawUICData:
        UICSheet = rawUICData.sheet_by_index(input_worksheet_index)
        UICData = []

        for each_row_index in range(1, UICSheet.nrows - 1, 1):
            row_list = []

            lat_num = UICSheet.cell_value(each_row_index, lat_col_index)     # Get Lat Values
            long_num = UICSheet.cell_value(each_row_index, long_col_index)   # Get Long Values

            if lat_num != 0.0 and long_num != 0.0:      # Find Zero Lat/Long Values

                for each_column_index in keep_columns:
                    cell_value = UICSheet.cell_value(each_row_index, each_column_index)
                    cell_type = UICSheet.cell_type(each_row_index, each_column_index)

                    if cell_type == 3:
                        date_cell = xldate_as_tuple(cell_value, rawUICData.datemode)
                        date_cell = date(*date_cell[0:3]).strftime('%m/%d/%Y')
                        row_list.append(date_cell)
                    else:
                        row_list.append(cell_value)

            for each_row in row_list :
                if not all(map(len, each_row)) :
                    continue
                else :
                    UICData.append(row_list)

            # CreateDB(row_list)  # Send row data to Database

        for each_list_index, output_list in enumerate(UICData):

            for each_element_index, element in enumerate(output_list):
                output_worksheet.write(each_list_index, each_element_index, element)

    output_workbook.save(tempXLSFile)

    #### RUN XLS-CSV CONVERSION

    workbook = open_workbook(tempXLSFile)
    sheet = workbook.sheet_by_index(input_worksheet_index)
    fh = open(outputFile, 'wb')
    csv_out = unicodecsv.writer(fh, encoding = 'utf-8')

    for each_row_number in range(sheet.nrows) :
        csv_out.writerow(sheet.row_values(each_row_number))

    fh.close()

    #### KILL TEMP FILES

    filesToRemove = [tempXLSFile]
    for each_file in filesToRemove:
        os.remove(each_file)
    print("Raw Data Conversion Ready for Grasshopper")

# ---------------------------------------------------
PrepareRawData(orig_xls, temp_xls, temp_csv, new_csv)
# ---------------------------------------------------

Answer 1

This is a dirty patch.

        for each_row in row_list :
            if not isinstance(each_row, list):
                each_row = [each_row]
            if not any(map(len, each_row)) :
                continue
            UICData.append(row_list)

EDIT: If the any/map/len raises it still, then I would try a different route to check if it's empty.

Also I'm not sure why you are appending the entire row_list and not the current row. I changed it to appending each_row.

Option1

for each_row in row_list:
    if not each_row:
        continue
    UICData.append(each_row)

Option2

keep_data = [arow in row_list if arow] # Or w/e logic. This will be faster.
UICData.append(keep_data)

Answer 2

Your row_list contains a set of values, for example:

[1.01, 75, 3.56, ...]

When you call for each_row in row_list:, you assign a float value to each_row for every iteration of the loop.

You then try to do this:

if not all(map(len, each_row)):

Python's map function expects a list as the second argument, and tries to iterate over it to apply the function len to each item in the list. You can't iterate a float.

I'm not entirely sure what you are trying to do here, but if you are wanting to check that none of the items in your row_list are None or an empty string, then you could do:

if None not in row_list and '' not in row_list:
    UICData.append(row_list)

Answer 3

Your overall object appears to be to copy selected columns from all rows of one sheet of an Excel XLS file to a CSV file. Each output row must contain only valid cells, for some definition of "valid".

As you have seen, using map() is not a good idea; it's only applicable if all the fields are text. You should apply tests depending generally on the datatype and specifically on the individual column.

Once you have validated the items in the row, you are in a position to output the data. You have chosen a path which (1) builds a list of all output rows (2) uses xlwt to write to a temp XLS file (3) uses xlrd to read the temp file and unicodecsv to write a CSV file. Please consider avoiding all that; instead just use unicodecsv.writer.writerow(row_list)