Numpy: Read in an arbitrary number of .txt files and save the data in one numpy array

148 views Asked by At

Excuse me if this has been asked previously, I could not find anything.

I am trying to define a function that takes an arbitrary number of .txt files that look like these

enter image description here

reads them, concatenates all the rows and saves it into one numpy array. This works for one .txt file. As soon as i use two files I get array([nan, nan]) as a return, with three files array([nan, nan, nan]), etc.

import numpy as np

def readInSpectra(*files):
    raw = np.genfromtxt(files[0], skip_header=0, delimiter='\t')
    for i in range(1, len(files)):
        raw_i = np.genfromtxt(files[i], skip_header=0, delimiter='\t')
        raw = np.vstack((raw, raw_i))
    return raw

files = ('./file1.txt', './file2.txt', './file3.txt')

test = readInSpectra(files)
2

There are 2 answers

3
Timus On

I'm not completely sure, but I think the repeated vstack is a problem because the shape of the arrays change. Have you tried:

def readInSpectra(*files):
    
    stacking = tuple(np.genfromtxt(file, skip_header=0, delimiter='\t')
                     for file in files)

    return np.vstack(stacking)

EDIT: I think you should call the function this way

test = readInSpectra(*files)

or

test = readInSpectra('./file1.txt', './file2.txt', './file3.txt')
0
Jiadong On

Both should work, I suggest you do second as suggested by @obchardon

import numpy as np


def readInSpectra_0(*files):
    files = files[0]
    raw = np.genfromtxt(files[0], skip_header=0, delimiter='\t')
    for i in range(1, len(files)):
        raw_i = np.genfromtxt(files[i], skip_header=0, delimiter='\t')
        raw = np.vstack((raw, raw_i))
    return raw

def readInSpectra_1(files):
    
    stacking = tuple(np.genfromtxt(file, skip_header=0, delimiter='\t')
                     for file in files)

    return np.vstack(stacking)

#files = ('file1.txt', 'file2.txt')
files = ('./file1.txt', './file2.txt', './file3.txt')


test = readInSpectra_1(files)