parse python functions as a string within decorator

Question

I am trying to write a function debug decorator that will look at:

def foo(baz):
  bar = 1
  bar = 2
  return bar

and wrap it to:

def foo(baz):
  bar = 1
  print 'bar: {}'.format(bar)
  bar = 2
  print 'bar: {}'.format(bar)
  return bar

I need to play with the function as text, to grab "\w+(?=\s*[=])", but do not know how to access that. I have a decorator I modified from a blog that works, but I just tried changing it to:

class decorator_string_check(object):

   def __init__(self, func):
        self.func = func
        wraps(func)(self)

   def __call__(self, *args, **kwargs):
        print dir(self.func)
        print dir(self.func.__code__)
        print self.func.__code__.__str__()
        ret = self.func(*args, **kwargs)
        return ret

@decorator_string_check
def fake(x):
    y = 6
    y = x
    return y

y = fake(9)

and am getting nothinng of value, namely:

['__call__', '__class__', '__closure__', '__code__', '__defaults__', '__delattr__', '__dict__', '__doc__', '__format__', '__get__', '__getattribute__', '__globals__', '__hash__', '__init__', '__module__', '__name__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'func_closure', 'func_code', 'func_defaults', 'func_dict', 'func_doc', 'func_globals', 'func_name']
['__class__', '__cmp__', '__delattr__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__le__', '__lt__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'co_argcount', 'co_cellvars', 'co_code', 'co_consts', 'co_filename', 'co_firstlineno', 'co_flags', 'co_freevars', 'co_lnotab', 'co_name', 'co_names', 'co_nlocals', 'co_stacksize', 'co_varnames']
<code object fake at 0x7f98b8b1d030, file "./logging.py", line 48>

How do I work with the actual "func" text, to run regexes on it and find things I need within a decorator class object? Thank you

Answer 1

First of all, I suggest you do not do something like that. It's hard to get working code and very hard to make a correct version.

Besides I don't really know what you want to do exactly. Should this decorator add a print statement after every assignment and show the updated value? Or keep track only of a given subset of variables?

This said, to obtain the source code of something you can use the inspect module in particular the getsource function:

In [1]: def test():
   ...:     a = 1
   ...:     b = 2
   ...:     

In [2]: import inspect

In [3]: inspect.getsource(test)
Out[3]: 'def test():\n    a = 1\n    b = 2\n'
In [4]: print(inspect.getsource(test))
def test():
    a = 1
    b = 2

You could modify and inspect the source code as you wish and finally compile() the new source code.

Note however that:

• You have to be careful when modifying the source code, because it's easy to create syntactically invalid code (think: multiline expressions etc.)
• when compiling you'd like to compile in the same scope as the original function. The inspect module has some functions that allow you to obtain the stackframe where your decorator is called and you can obtain the environments from there. Read here about how to handle the stack.
• The source code may not be available. The code could be compiled in bytecode and the original file may be deleted. In this case getsource would simply raise an OSError.

---

A more "sane" solution would be to not look at the source code, but at the bytecode. You can do this using the dis module. You may try to see when the values of the variable change and insert some bytecode that will print that variable.

Note that the dis module was greatly enhanced in python3.4+, so with previous versions of python this would probably be hard.

You should probably read articles like Python bytecode hacks, gotos revisited before trying this. They give you an idea on how to look at bytecode and work with it.

This is probably safer (e.g. even if the source file doesn't exist on the machine the bytecode will still be accessible), yet I still think what you have in mind is not a good thing to do, except as an exercise.

---

As jsbueno points out the proper way of doing what you want (i.e. a python debugger) is to use sys.settrace.

This function allows you to set a tracing function which will be called for each "code" executed. The function will know when a function is called, a new block is entered etc. It gives you access to the frame where the code will be executed, and thus you should be able to find the values you are interested in.

You should check the lnotab_notes.txt file to understand how the data provided as argument to this function can be mapped to the source code positions to understand when an assignment is performed.

_{When I have time (probably next week end) I'll try to implement something based on this approach to demonstrate it.}