Python: looking for duplicates in list

Question

I have a list of floats, and I want to know how many duplicates are in it.

I have tried with this:

p = t_gw.p(sma, m1, m2)       #p is a 1d numpy array
p_list = list(p)
dup = set([x for x in p_list if p_list.count(x) > 1])
print dup

I have also tried to use collections.counter, but I always get the same error

TypeError: unhashable type: 'numpy.ndarray'

I've looked around in similar questions, but I can't understand what hashable means, why a list (or numpy array) is not hashable and what kind of type should I use.

Answer 1

Your numpy-array is two-dimensional. So list(p) does not do, what you expect. Use list(p.flat) instead.

Or (mis)use numpy's histogram function:

cnt, bins = numpy.histogram(p, bins=sorted(set(p.flat))+[float('inf')])
dup = bins[cnt>1]

Answer 2

It depends what do you mean by number of duplicates.

An easy way to do this is to use hash:

h = {}
arr = [6, 3, 1, 1, 6, 2, 1]
for i in arr:
    if i in h:
        h[i] += 1
    else:
        h[i] =1

print h

Now if you mean that duplicates are the values that are used more then once in the list, you can do this with:

num = 0
for i in h:
    if h[i] > 1:
        num += 1

print num

I think that it is pretty easy to modify it to numpy.

Answer 3

you want to count something in a list ? why not use the count method of list object ?

number = my_list.count(my_float)