Deleting elements from a list if that value is masked in another list

Question

I have imported data that has the format of a numpy masked array of incrementing integers. The masked elements are irregular and not repeating, e.g printing it yields:

masked = [0,1,--,3,--,5,6,--,--,9,--]

And I have another list of incrementing numbers that doesn't start from zero, and has irregular gaps and is a different size from masked:

data = [1,3,4,6,7,9,10]

I want to remove any element of data if its value is a masked element in masked

So that I get:

result = [1,3,6,9]

As 4, 7 and 10 were masked values in masked.

I think my pseudocode should look something like:

for i in len(masked):
    if masked[i] = 'masked' && data[i] == [i]:
        del data[i]

But I'm having trouble reconciling the different lengths and mis-matched indices of the two arrays,

Thanks for any help!

Answer 1

Make sure data is an array:

data = np.asarray(data)

Then:

data[~masked.mask[data]]

This will be extremely fast, though it does assume that your masked array contains all numbers from 0 to at least max(data).

Answer 2

You can use set function to get sets of the lists and take their intersection. Here goes a demo :-

>>> import numpy as np
>>> import numpy.ma as ma
>>> arr = np.array([x for x in range(11)])
>>> masked = ma.masked_array(arr, mask=[0,0,1,0,1,0,0,1,1,0,1])
>>> masked
masked_array(data = [0 1 -- 3 -- 5 6 -- -- 9 --],
             mask = [False False  True False  True False False  True  True False
  True],
       fill_value = 999999)

>>> data = np.array([1,3,4,6,7,9,10])
>>> result = list(set(data) & set(masked[~masked.mask]))
>>> result
[1, 3, 6, 9]
>>>