Why empty cells are printed in this code?_Python

Question

I have this code to read text files in a folder, do some analysis on columns and output the new columns. Strangely, I get some last rows deflected from where they should be. I found that changing the segment size, affects the last rows, but couldn't find the solution for that.

I really appreciate if someone can have a look into it and tell what I am doing wrong please.

files_ = glob.glob('D:\Test files\*.txt')
tfile_ = loadtxt('times.txt')

def cum_sum(lis):
    total = 0
    for x in lis:
        total += x[1]
        yield total

averages_ = []
d = []

with open ('outfile.csv', 'wb') as outfile:
    writer = csv.writer(outfile)

    for fcount_, fname_ in enumerate(files_):
        tzero_ = int(round(tfile_[fcount_],1))

        data = loadtxt(fname_ , usecols = (1,2,3,4), skiprows = tzero_)

        average_, f_column, average_original, fcol = [], [], [], []
        segment_size = 2
        background_average = mean(data[0:5,0])

        for x in range(0, len(data[:,0]), segment_size):
            sample_means = [x/segment_size] + [mean(((data[x:x+segment_size,i])-(background_average))/10.467) for i in range(4)]
            none_zeros = [x if x >= 0 else x == 0 for x in sample_means]
            float_means = ["%1f" % (x) for x in none_zeros]            
            average_original.append(sample_means)
            average_.append(float_means)

        fcol = list(cum_sum(average_original))
        average_ = [row + [col] for row, col in zip(average_, fcol)]
        averages_.append(average_)

    for row in izip_longest(*averages_, fillvalue = ['']):
        writer.writerow(list(chain.from_iterable(row)))

Input data:

1.txt

2013-08-29T15:11:18.55912   0.019494552 0.110042184 0.164076427 0.587849877 8.5
2013-08-29T15:11:18.65912   0.036270974 0.097213155 0.122628797 0.556928624 8.6
2013-08-29T15:11:18.75912   0.055350041 0.104121094 0.121641949 0.593113069 8.7
2013-08-29T15:11:18.85912   0.057159263 0.107410588 0.198122695 0.591797271 8.8
2013-08-29T15:11:18.95912   0.05288292  0.102476346 0.172958062 0.591139372 8.9
2013-08-29T15:11:19.05912   0.043507861 0.104121094 0.162102731 0.598376261 9
2013-08-29T15:11:19.15912   0.068343545 0.102805296 0.168517245 0.587849877 9.1
2013-08-29T15:11:19.25912   0.054527668 0.105765841 0.184306818 0.587191978 9.2
2013-08-29T15:11:19.35912   0.055678991 0.107739538 0.169997517 0.539165352 9.3
2013-08-29T15:11:19.45912   0.05321187  0.102476346 0.167530397 0.645744989 9.4

2.txt

2013-08-29T15:11:19.35912   0.055678991 0.107739538 0.169997517 0.539165352 9.3
2013-08-29T15:11:19.45912   0.05321187  0.102476346 0.167530397 0.645744989 9.4
2013-08-29T15:11:19.55912   0.055021092 0.103134245 0.158155337 0.604955251 9.5
2013-08-29T15:11:19.65912   0.054363193 0.103463195 0.154207944 0.587191978 9.6
2013-08-29T15:11:19.75912   0.045974982 0.096555256 0.179372577 0.613507938 9.7
2013-08-29T15:11:19.85912   0.054527668 0.103463195 0.166543548 0.587191978 9.8
2013-08-29T15:11:19.95912   0.058310586 0.10050265  0.152727671 0.599692059 9.9
2013-08-29T15:11:20.05912   0.056830314 0.097542105 0.170490942 0.591797271 10
2013-08-29T15:11:20.15912   0.056830314 0.099186852 0.180852849 0.59903416  10.1
2013-08-29T15:11:20.25912   0.056172415 0.102805296 0.152234247 0.609560544 10.2
2013-08-29T15:11:20.35912   0.056994789 0.101818448 0.170984366 0.586534079 10.3
2013-08-29T15:11:20.45912   0.050909224 0.10609479  0.148286854 0.599692059 10.4
2013-08-29T15:11:20.55912   0.081994947 0.099186852 0.162596155 0.597718362 10.5
2013-08-29T15:11:20.65912   0.056830314 0.101160549 0.139898643 0.593770968 10.6
2013-08-29T15:11:20.75912   0.061271131 0.101489498 0.135457826 0.583244584 10.7

3.txt

2013-08-29T15:17:22.15912   0.118672813 5.650572169 4.684839266 2.462863007 372.1
2013-08-29T15:17:22.25912   0.108310905 5.51635861  4.67398387  2.374046541 372.2
2013-08-29T15:17:22.35912   0.135613712 5.414382684 4.676944432 2.366151744 372.3
2013-08-29T15:17:22.45912   0.105843784 5.294643278 4.630562289 2.37602024  372.4
2013-08-29T15:17:22.55912   0.116699116 5.177535575 4.667569318 2.35233585  372.5
2013-08-29T15:17:22.65912   0.119330712 5.056480492 4.711484329 2.357599048 372.6
2013-08-29T15:17:22.75912   0.12097546  4.973913071 4.672503588 2.356283248 372.7
2013-08-29T15:17:22.85912   0.121304409 4.881148118 4.638950548 2.354967449 372.8
2013-08-29T15:17:22.95912   0.122126783 4.783777856 4.665595609 2.367467544 372.9
2013-08-29T15:17:23.05912   0.114560945 4.739369136 4.646351954 2.35167795  373
2013-08-29T15:17:23.15912   0.127718924 4.688381358 4.647832235 2.432599618 373.1
2013-08-29T15:17:23.25912   0.125251803 4.659104509 4.632535997 2.30759867  373.2
2013-08-29T15:17:23.35912   0.127061025 4.630156618 4.621187175 2.345756853 373.3
2013-08-29T15:17:23.45912   0.126238651 4.613708955 4.619706894 2.348388452 373.4

4.txt

2013-08-29T15:17:24.35912   0.131172894 4.675223223 4.498817288 2.361546446 374.3
2013-08-29T15:17:24.45912   0.148936166 4.672591596 4.485988189 2.370099142 374.4
2013-08-29T15:17:24.55912   0.118014914 4.68278915  4.567897059 2.362204346 374.5
2013-08-29T15:17:24.65912   0.116699116 4.691341938 4.472665663 2.379309739 374.6
2013-08-29T15:17:24.75912   0.124758379 4.690355078 4.442566625 2.414178425 374.7
2013-08-29T15:17:24.85912   0.133804489 4.698907866 4.46378398  2.35233585  374.8
2013-08-29T15:17:24.95912   0.130186045 4.705486934 4.444540332 2.348388452 374.9
2013-08-29T15:17:25.05912   0.132653166 4.716342396 4.450954881 2.345756853 375
2013-08-29T15:17:25.15912   0.135284762 4.723908325 4.43861921  2.30891447  375.1
2013-08-29T15:17:25.25912   0.134791338 4.738053323 4.431217808 2.304967072 375.2
2013-08-29T15:17:25.35912   0.132159742 4.753185182 4.3813817   2.32996726  375.3

The problem: The code above gives this output where 6th and 7th rows are moved one columns to the right. This is what I don't understand!

0   0   0.004933    0.006984    0.050252    -0.000307987    0   0.000101    0.004744    0.009797    0.051823    0.000100567 0   0   0.489086    0.433002    0.214708    -0.000531121
1   0.000572    0.005341    0.013042    0.051823    0.000263989 1   0   0.00443 0.011399    0.052232    -0.000223134    1   0.000317    0.467982    0.437104    0.214016    -0.000213706
2   0.000658    0.0052  0.011109    0.05198 0.000922391 2   0.000376    0.004336    0.010315    0.051792    0.000152422 2   0.000467    0.450524    0.433309    0.214425    0.00025299
3   0.00058 0.005514    0.01224 0.049121    0.001502225 3   0.000273    0.004524    0.010787    0.052609    0.00042584  3   0.000412    0.439194    0.432814    0.217379    0.000664687
4   0.000399    0.005106    0.011321    0.057009    0.001901351 4   0.00003 0.004807    0.010127    0.05154 0.000455696 4   0.000891    0.432579    0.430881    0.211125    0.001555651
    5   0.001507    0.004446    0.009325    0.051792    0.001962634 5   0.000899    0.429624    0.430197    0.213199    0.002454472                 
    6   0.000729    0.004572    0.007817    0.050598    0.002691747                                         

BTW, times.txt is:

1.87
2.3
3.21
2.84

Answer 1

It looks like your output is separated by tabs.

If one data entry is more than a tab-width wide (usually 8 characters), it will push all following entries in the row over by that much.