In this tutorial, we will see how to do multiprocessing
in Python using an example.
What is multiprocessing?
It is a way to run multiple processes at the same time given that the machine has support for multiple processors. The main advantage is of multiprocessing is that the system actually performs multiple independently executable parts of an application at the same time which eventually increases the efficiency and performance of the system. Before moving further let’s see what is a process identifier
.
A Process Identifier or PID, in short, is a number that uniquely identifies each running process in an operating system which can be Unix, Linux, macOS, and MS Windows.
Multiprocessing in Python
Python provides an inbuilt package named multiprocessing
which provide API that effectively supports concurrency, distributing the input data across processes or parallel executions of different processes. In the above section, we have learned about process identifier and the built-in os
module of python provides us with a method named getpid()
which returns the PID of the process.
Example
Let’s create a function that takes an integer and return whether it is positive or not and print the PIDs.
import multiprocessing
import os
def isPositive(n):
print("isPositive function with Process ID:",os.getpid())
if n>=0:
print("Integer passed is Positive:")
else:
print("Integer passed is Negative:")
if __name__ == "__main__":
print("Main function with Process ID:",os.getpid())
isPositive(-1)
isPositive(2)
Output:
Main function has Process ID: 7644
isPositive function has Process ID: 7644
Integer passed is Negative:
isPositive function has Process ID: 7644
Integer passed is Positive:
Note: PID can be different from the above output.
From the output, we see that all the function calls that we have called has the same Process ID which is same as that of the main function.
Next, we will use the components of the multiprocessing module to run the tasks as different processes.
Process Class of multiprocessing module
Whenever we want to create a new process we create an object of the Process class whose constructor takes the following arguments.
- target – It is the callable object to be invoked by the
run()
method and default value isNone
. - args – It is the argument tuple that is passed in the function specified in the target.
- name – It is the process name as a string. the default value is None.
There are some other methods provided by this class which are as follows.
start()
– It will start the process’s activity and is called at most once for each process.run()
– This invokes function passed as the target argument.join([timeout])
– This method blocks until the process whose join() method is called terminates.is_alive()
– ReturnsTrue
if a process is alive elseFalse
.pid
– Return the process Id of a process
Example
We will try to perform multiprocessing on the above example using the Process class.
import multiprocessing
import os
def isPositive(n):
print("isPositive function with Process ID:",os.getpid())
if n>=0:
print("Integer passed is Positive")
else:
print("Integer passed is Negative")
if __name__ == "__main__":
print("Main function with Process ID:",os.getpid())
proc1 = multiprocessing.Process(target=isPositive,args=(8,))
proc2 = multiprocessing.Process(target=isPositive,args=(-6,))
proc1.start()
proc2.start()
print("Process proc1 has Process ID:",proc1.pid)
print("Process proc2 has Process ID:",proc2.pid)
proc1.join()
proc2.join()
Output:
Main function with Process ID: 10124
Process proc1 has Process ID: 14912
Process proc2 has Process ID: 11708
isPositive function with Process ID: 14912
Integer passed is Positive
isPositive function with Process ID: 11708
Integer passed is Negative
From the output above, we can see that in total we have 3 processes running, one main process and two subprocesses of the main process.
So we learned about multiprocessing in python.
Resources
Happy Learning 🙂