Cannot extend a list and take set() in a single step without Non-Iterable error

Question

I'm trying to manage a list of unique items that may (or may not) get additions with each iteration of a loop. Maybe they are just integers produced by doSomething(someData).

So with each iteration I call doSomething and want to add my results to a growing list...

   uniqs = []
   for md in mydata:
       newOnes = doSomething(md)                   # returns a list eg [3,2,3]
       uniqs = list(set(uniqs.extend(newOnes)))    # keep only uniquely new items

But It appears i can't do the extend and the set at the same time without an error

TypeError: 'NoneType' object is not iterable

I can do it in two steps,

   uniqs = []
   for md in mydata:
       newOnes = doSomething(md)        # returns a list eg [3,2,3]
       uniqs.extend(newOnes)            # extend the list will all new items
       uniqs = list(set(uniqs))         # keep only unique items

but I'd have thought that line would be okay, as the extend() would occur before the set() would be applied. Not sure why it isn't.

Can someone clarify why that is?

-Ross

Answer 1

If you are happy for uniqs to be a set rather than a list, you can use update to iterate over the list and add any elements that do not already exist:

   uniqs = set()
   for md in mydata:
       newOnes = doSomething(md)
       uniqs.update(newOnes)

If required, you can convert it to a list at the end with uniqs = list(uniqs). The ordering will be undefined, but as you are already using a set as an intermediate in your calculation, this is already the case.

You could perhaps write the above as a one-liner using functools.reduce:

functools.reduce(set.union, (set(doSomething(md)) for md in mydata))

although the explicit loop is probably more readable.