Python Subprosess: operation and examples

Python subprocess is a module that allows you to order, run and assess external programs inside the code. The two main functions of this tool are run() And Popen().

What is the Python module subprocess ?

The Python module subprocess is part of the programming language inventory from version 2.4. This is a very complete and powerful tool that you can use to Execute other programs or orders in your code. It not only allows you to open programs, but also to control and adjust the data flow. Python subprocess Offers many methods and functions, which we will examine the most important in this article and explain them using practical examples.

Python subprocess : operation with run()

Let's start by taking a look at the basic structure and operation of the subprocess of Python. This module is used to execute sub-processes: Python then works according to a parent-child hierarchy, and takes on the role of parent who creates a child process. The most used function in the module is run(). It allows you to launch a process via Python and only begins the following steps when it was fully executed.

Example of operation of subprocess in python with run()

We will now use this function for our first example of the operation of Python subprocess. For this, we first import the modules subprocess And systhen we execute a simple request. The corresponding code looks like this:

import subprocess
import sys
result = subprocess.run([sys.executable, "-c", "print('bonjour')"])

The exit is then as follows:

• subprocess.run : This is the actual function. She receives a list of character strings containing the command to be executed. run() Then performs a new Python program.
• sys.executable : This is the absolute path that leads to the Python file with which you initially called your program. Such a path could look like /local/utilisateur/bin/exemple.
• -c :: -c is a command line option with which the character string mentioned is transmitted for execution. For our example, this is a program that displays the word « Hello ».

Execute python subprocess with a script

To test how you use the module for a script you have created, you can try the following example. First of all create a simple script in format .Py And save it as « exemptionscript.py »:

print("Il fait beau aujourd’hui")

To then run this file with Python subprocessuse the following code:

import subprocess
result = subprocess.run(["python", "exemplescript.py"], capture_output=True, text=True)
print(result.stdout)

The corresponding output will then look like this:

Il fait beau aujourd’hui

Open external programs

In principle, it is possible to open any program with python subprocess and the function run(). The only condition is to know the exact name or the access path of this program on your system. For example, in the following code, we open NOTEPAD:

import subprocess
subprocess.run(["notepad"])

CompletedProcess and capture external exits

After these simple examples, we will now be interested in Capture of an external output. To do this, you run an external program with Python subprocess as above, but you receive an object called in return CompletedProcess. We have already briefly integrated the necessary adaptations in a higher example, we will now detail them. Our starting point is again our first code, but with some adaptations:

import subprocess
import sys
result = subprocess.run([sys.executable, "-c", "print('bonjour')"], capture_output=True, text=True)
print("La sortie par défaut est :", result.stdout)
print("L’erreur par défaut est :", result.stderr)

Once again, we ask the system to display the « hello » character string in a child process. The two keyword arguments capture_output=True And text=Truethat we also go to run()are new. If the instruction is executed and there is no error, you get an object CompletedProcess with a link to result. This object contains information on the exit code of the program you want to execute and transmit it to result.stdout And result.stderr. stdout indicates the default output and stderr any default errors. We use text=True To print the output in the form of a character string. As no standard error is expected, our result is as follows:

La sortie par défaut est : bonjour
L’erreur par défaut est :

We will now change the previous example so that stderr Don't be empty. Here is the corresponding code:

import subprocess
import sys
result = subprocess.run([sys.executable, "-c", "raise ValueError('erreur')"], capture_output=True, text=True)
print("La sortie par défaut est :", result.stdout)
print("L’erreur par défaut est :", result.stderr)

While the standard output remains empty, there is now an outing for stderr ::

La sortie par défaut est :
L’erreur par défaut est : Traceback (most recent call last):
File "", line 1, in 
ValueError: erreur

Execution from a function

If you want to include an order directly in the codePython subprocess offers you this option. In this case, the code could resemble this example:

import subprocess
result = subprocess.run(["C:/Users/name/anaconda3/python", "-c", "print('Cette sortie a été prise directement depuis une fonction')"], capture_output=True, text=True, shell=True)
print("La sortie par défaut est :", result.stdout)

The exit is then as follows:

La sortie par défaut est : Cette sortie a été prise directement depuis une fonction

Stop or finish processes

Another very useful use of Python subprocess is obtained by the interaction of run() With the argument timeout. It allows you to stop an external program if its execution takes too long. To do this, use the function time.sleep. The appropriate code is as follows:

import subprocess
import sys
result = subprocess.run([sys.executable, "-c", "import time; time.sleep(3)"], timeout=1)

The child's process uses time.sleep To stop for three seconds. However, as you asked the system to trigger a timeout after a second via timeout=1the result is an exception TimeoutExpired.

Python subprocess with Popen()

Although run() either the function of subprocess From the most used Python, there are also other important python classes which can be very useful. Among them, we find Popen(). This class is in a way the subset of subprocess and is a little more complex to use than run(). On the other hand, Popen() gives you more control over the execution and allows you to interact with the input and the exit. This class owes its name to a UNIX command and means  » Open pipe « (Literally » open hose « ).

Almost all the arguments you can use with run() are authorized with Popen(). However, unlike run()this function does not wait for the end of a process: She launches a second in parallel. Here is a simple example:

import subprocess
from time import sleep
def poll_and_read(process):
    print(f"Voici la sortie après poll(): {process.poll()}")
    print(f"Voici la sortie par défaut : {process.stdout.read().decode(‘utf-8’)}")
process = subprocess.Popen(["python", "timer.py", "3"], stdout=subprocess.PIPE)
poll_and_read(process)
sleep(2)
poll_and_read(process)
sleep(2) 
poll_and_read(process)
process.wait()
print(f"Code de sortie du processus : {process.returncode}")

Here we use the method .poll() To check if the process is still in progress or if it is already over. As long as it is still running, the « none » value is displayed. Then the method displays the output code. With .read()all bytes that were so far in .stdout must be read. If you run the code, you will first get the « none » value, then the value contained so far in stdout. This will continue until the process is finished. poll() Then receives the value « 0 ».