Python starting and communicating with processes 101

Direct approach using os module:

os.system("ls -l") which executes the command passed as a string in a sub-shell: os.system.__doc__ 'Execute the command in a subshell.'

Using subprocess module the latter will become:

subprocess.call(["ls","-l"]) or subprocess.call("ls -l", shell=True) the difference being when shell=False is set, there is no system shell started up, so the first argument must be a path to an executable file and if shell=True means system shell \bin\sh will first spin up.

In Python 3.5 as subprocess.run was added as a simplification for subprocess.Popen

x = subprocess.run(["ls","-l"]) # shell=False 
y = subprocess.run("ls -l",shell=True)
proc = subprocess.run("ls -al", shell=True, encoding="utf-8", stdout=subprocess.PIPE) 
output = proc.stdout

The main difference between them is that run executes the command and waits for it to finish while Popen you can continue doing stuff while the process finishes and then just repeatedly call to comunicate yourself to pass and receive data to your process.

import subprocess
# get output as 
s = subprocess.check_output(["echo", "Hello World!"])
print(f"output is {s}:")

The recommended approach to invoking subprocesses is to use the run() function for all use cases it can handle. For more advanced use cases, the underlying Popen interface can be used directly.

subprocess.Popen : direct call to the process constructor, where the command is passed as a list and you can set the stdout, stderr value to PIPE and call comunicate() to get the output

from subprocess import Popen, PIPE

process = Popen(['less', 'test.py'], stdout=PIPE, stderr=PIPE)
stdout, stderr = process.communicate()
#wait for proc to complete and read the output line by line
output = process.stdout.readline()

Photo by Alex Chumak on Unsplash

Using shlex, we can create a function which reads the stdout and check for return code process.poll() and displays the output :