Linked Questions

Popular Questions

How to to remove these (') and ('') in python?

Asked by At

I'm having a problem with (') and (") in Python. I wrote this code:

from __future__ import division
from sklearn.feature_extraction.text import CountVectorizer
import os
import numpy as np

# STEP1 Load the classic4 dataset
fold = os.path.join('classic')

corpus = []
for file in os.listdir(fold):
with open(os.path.join(fold, file)) as f:
    text = f.read()
    text = text.strip()
    text = text.replace('\n',' ')
    corpus.append(text)
    #print(text)

with open('terms.txt') as f:
voc = f.readlines()
voc = [v.replace('\n', '') for v in voc]
vectorizer = CountVectorizer(vocabulary=voc)
X = vectorizer.fit_transform(corpus).todense()
#print(vectorizer.get_feature_names())
#np.save('X.npy', X)
np.savetxt('X.txt', X, fmt="%d")

print(X.shape)

#  STEP2  list of lists aka tfidf_vectorized_list

with open('docbyterm.tfidf.txt') as f:
txt = f.readlines()

data = []
for i in range(7095):
data.append([])
for i in txt:
i_list = i.split(' ')
data[int(i_list[0])-1].append(int(i_list[1]))

with open('tfidf_vectorized_list.txt','w+') as f:
for i in data:
    f.write(str(i)[1:-1])
    f.write('\n')
 #print(data)

 tfidf_vectorized_list = []
 with open('tfidf_vectorized_list.txt') as f:
 for line in f:
    inner_list = []
    for elt in line.split(','):
        stripped_elt = elt.strip()
        if len(stripped_elt) > 0:
            inner_list.append(int(stripped_elt))
    tfidf_vectorized_list.append(inner_list)
 tfidf_vectorized_list = np.array(tfidf_vectorized_list)
 print(tfidf_vectorized_list)

The link for tfidf_vectorized_list is https://github.com/Diallosky/tfidf_vectorized_list. I noticed that my tfidf_vectorized_list is like this:

2, 5, 4, 1, 3
11, 7, 9, 8, 10, 6
16, 12, 14, 13, 15
11, 19, 18, 17, 15
7, 21, 20
11, 23, 22
...
23, 58, 55, 51, 53, 59, 57, 60, 50, 56, 31, 12, 54, 52

The above code produces the following output:

[['2, 5, 4, 1, 3' '']
['11, 7, 9, 8, 10, 6' '']
['16, 12, 14, 13, 15' '']
...
['23, 58, 55, 51, 53, 59, 57, 60, 50, 56, 31, 12, 54, 52' '']

I would like to remove the ' and '' characters in order to get a result like this:

[[2, 5, 4, 1, 3]
[11, 7, 9, 8, 10, 6]
[16, 12, 14, 13, 15]
...
[23, 58, 55, 51, 53, 59, 57, 60, 50, 56, 31, 12, 54, 52]

Related Questions