Combining __setattr__ and __getattr__ causes infinite loop

Question

I'm trying to build a system where a base class is used for every other object. Every base object has a _fields dictionary internally where implementations of the base class can store their information.

Base class implementation is quite simple:

class A(object):
    def __init__(self, fields=dict()):
        self._fields = fields

An implementation of the class can set the field in the __init__ call to its super().

What I'd like to add is that the fields are accessible as properties without having to add the @property decorator to a whole bunch of functions. I've overriden the __getattr__ for this purpose in the base class like so:

class A(object):
    def __init__(self, fields=dict()):
        self._fields = fields

    def __getattr__(self, name):
        if hasattr(self, name):
            return object.__getattribute__(self, name)
        elif name in self._fields:
            return self._fields.get(name)
        else:
            raise AttributeError

Now an implementation for this class can work like this:

class A_impl(A):
    def __init__(self):
        super(A_impl, self).__init__(
                    fields=dict(
                    val_1="foo",
                    val_2="",
                    val_3="bar",
                )
            )

By which creating an implementation of this class gives you the options to do:

test = A_imp()
print test.val_1
print test.val_2
print test.val_3

Which returns

foo

bar

I can even override this via @property decorators, changing the class like so:

class A_impl(A):
    def __init__(self):
        super(A_impl, self).__init__(
                    fields=dict(
                    val_1="foo",
                    val_2="",
                    val_3="bar",
                )
            )

    @property
    def val_1(self):
        return self._fields.get('val_1') + "_getter"

Which allows me to manipulate the data for return. The only issue is that if I want to be able to set one of these field variables I have to implement the descriptor setter functions which also requires me to make the property descriptor which creates a lot of duplicate work (ie I have to define descriptors for ALL my fields which is what I want to avoid here.

I implemented the __setattr__ function for the base class to solve the issue where if I manually implement a descriptor setter function it should be chosen over the default which is self._field[name] = value. The base class now looks like this (similar to the __getattr__):

class A(object):
    def __init__(self, fields=dict()):
        self._fields = fields

    def __getattr__(self, name):
        if hasattr(self, name):
            return object.__getattribute__(self, name)
        elif name in self._fields:
            return self._fields.get(name)
        else:
            raise AttributeError

    def __setattr__(self, name, value):
        if hasattr(self, name):
            object.__setattr__(self, name, value)
        elif name in self._fields:
            self._fields[name] = value
        else:
            raise AttributeError

Now if I run the same code test again:

test = A_imp()
print test.val_1
print test.val_2
print test.val_3

It instantly gets stuck in an infinite loop, it starts at the __setattr__ but jumps into the __getattr__ right after and keeps looping that.

I've been reading a lot of questions on stackoverflow for this and couldn't figure it out, this is why I build this test case to cleanly figure it out. Hopefully someone's able to clarify this for me and help me solve the issue.

Answer 1

There is no way to check whether an object has an attribute other than actually trying to retrieve it. Thus,

hasattr(self, 'val_1')

actually tries to access self.val_1. If self.val_1 isn't found through the normal means, it falls back on __getattr__, which calls hasattr again in an infinite recursion.

hasattr actually catches any exception, including RuntimeError: maximum recursion depth exceeded, and returns False, so the exact way this manifests depends on precisely how many __getattr__ calls are nested. Some __getattr__ calls hit the object.__getattribute__ case and raise an AttributeError; some hit the elif name in self._fields: return self._fields.get(name) case. If self._fields exists, this returns a value, but with your __setattr__, sometimes self._fields doesn't exist!

---

When your __setattr__ tries to handle the self._fields assignment in __init__, it calls hasattr(self, '_fields'), which calls __getattr__. Now some __getattr__ calls make two recursive __getattr__ calls, one in hasattr and one in elif name in self._fields. Since hasattr is catching exceptions, this causes recursive calls exponential in the recursion depth instead of quickly either seeming to work or raising an exception.

---

Don't use hasattr in __getattr__ or __getattribute__. In general, be very careful about all attempts to access attributes inside the methods that handle attribute access.