Python: how to dynamically set function closure environment

Question

I want to declare a function dynamically and I want to wrap any access to global variables OR alternatively define which variables are free and wrap any access to free variables.

I'm playing around with code like this:

class D:
    def __init__(self):
        self.d = {}     
    def __getitem__(self, k):
        print "D get", k
        return self.d[k]
    def __setitem__(self, k, v):
        print "D set", k, v
        self.d[k] = v
    def __getattr__(self, k):
        print "D attr", k
        raise AttributeError

globalsDict = D()

src = "def foo(): print x"

compiled = compile(src, "<foo>", "exec")
exec compiled in {}, globalsDict

f = globalsDict["foo"]
print(f)

f()

This produces the output:

D set foo <function foo at 0x10f47b758>
D get foo
<function foo at 0x10f47b758>
Traceback (most recent call last):
  File "test_eval.py", line 40, in <module>
    f()
  File "<foo>", line 1, in foo
NameError: global name 'x' is not defined

What I want is somehow catch the access to x with my dict-like wrapper D. How can I do that?

I don't want to predefine all global variables (in this case x) because I want to be able to load them lazily.

Answer 1

What you're looking for is object proxying.

Here is a recipe for an object proxy which supports pre- and post- call hooks:

http://code.activestate.com/recipes/366254-generic-proxy-object-with-beforeafter-method-hooks/

Create a subclass that doesn't actually load the object until the first time the _pre hook is called. Anything accessing the object will cause the real object to be loaded, and all calls will appear to be handled directly by the real object.

Answer 2

Try this out

class GlobalDict(object):

    def __init__(self, **kwargs):
        self.d = kwargs

    def __getitem__(self, key):
        print 'getting', key
        return self.d[key]

    def __setitem__(self, key, value):
        print 'setting', key, 'to', value
        if hasattr(value, '__globals__'):
            value.__globals__.update(self.d)
        self.d[key] = value
        for v in self.d.values():
            if v is not value:
                if hasattr(v, '__globals__'):
                    v.__globals__.update(self.d)

    def __delitem__(self, key):
        print 'deling', key
        del self.d[key]
        for v in self.d.values():
            if hasattr(v, '__globals__'):
                del v.__globals__[key]

>>> gd = GlobalDict()
>>> src = 'def foo(): print x'
>>> compiled = compile(src, '<foo>', 'exec')
>>> exec compiled in {}, gd
setting foo to <function foo at 0x102223b18>
>>> f = gd['foo']
getting foo
>>> f
<function foo at 0x102223b18>
>>> f() # This one will throw an error
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "<foo>", line 1, in foo
NameError: global name 'x' is not defined
>>> gd['x'] = 1
setting x to 1
>>> f()
1
>>> del gd['x'] # removes 'x' from the globals of anything in gd
>>> f() # Will now fail again
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "<foo>", line 1, in foo
NameError: global name 'x' is not defined