Python from source code to interpreter

and beyond …

When running python source code, the following sequence occurs:

• lexing (generates tokens)
• parsing (generates abstract Syntax Tree Structure)
• compiling (generates code objects)
• interpreting

As we can see, before reaching the interpreter (virtual machine), the source code it is transformed from lines of plain text into byte-code instructions which are themselves “wrapped” into code objects.

Byte-Code

Going further the interpreter works by manipulating the objects in memory, based on instructions. The instructions specify what type of objects to create, which objects to delete, the attribute of an object to be read or modified.

In order to see how the byte-code looks like, we need to inspect Code Objects. Let’s take a simple function and access the code object via __code__ :

<class ‘code’>

def f(x):
   return x/2
fco = f.__code__ # will  give us the code object
# byte code instructions
print(fco.co_code) # will print b'|\x00d\x01\x1b\x00S\x00'

Nothing special until now, but the code object has some useful attributes:

dir(fco) 

Notably co_consts (which represents constants like string literals or numeric values contained within the code object) will allow us furthermore to tinker with the functions behavior.

Code object attributes are read-only, thus we need to inject the new value for co_consts not in the current f.__code__ but instead we will use fco(a copy of the current code object).

from types import CodeType
# create new co_const tuple
newconst = (None,5)
f.__code__ = CodeType(fco.co_argcount, fco.co_kwonlyargcount,
             fco.co_nlocals, fco.co_stacksize, fco.co_flags,
             fco.co_code, newconst, fco.co_names,
             fco.co_varnames, fco.co_filename,
             fco.co_name,
             fco.co_firstlineno, fco.co_lnotab, fco.co_freevars,
             fco.co_cellvars)

If previously , a simple call like f(5) returned the value 2.5, now it will return 1.0 , so we were able to alter the functions behavior without modifying the source code

Hope this small exercise gave a better understanding of how Python’s VM works.

Lots of interesting stuff on the same subject :

18.10.1 Python Byte Code Instructions

types - Dynamic type creation and names for built-in types - Python 3.7.3 documentation

A Python Interpreter Written in Python