Threads & Processes

8 years, 2 months ago

Python's threading and multiprocessing modules can often serve the same purpose. However, the differences cause enough emergent profanity to warrant this post.

0. Why Thread?

The first time I really needed threads was GUI creation. With only one thread, the GUI will freeze "program is not responding"-ly for any time-consuming function. In my case, one function makes some HTTP requests that took about one second. One very noticeable second.

The solution was to use a QThread to do the requests in the background, send the data to the GUI thread when it's ready. One practical reason to thread then: When you want to do two things at once.

1. How's a Thread different to a Process?

# -*- coding: utf-8 -*-
# original name: thread_script.py
# author: dud1337
import sys
from threading import Thread
from time import sleep

class TestThread(Thread):
# Prints a message every n seconds
	def __init__(self, to_print, time_interval):
		Thread.__init__(self)
		self.s = to_print
		self.t = time_interval
		self.daemon = True	# kills thread on exit
	def run(self):
		while True:
			sleep(self.t)
			print self.s

threadA = TestThread('I\'m thread A!', 3)
threadB = TestThread('I\'m thread B.', 4)

print 'ctrl+C to exit'

threadA.start()
threadB.start()

# Preventing program from ending
try:
	while True:
		sleep(1)
except KeyboardInterrupt:
	print '\nSeeya!'
	sys.exit()

This script creates two threads, each printing to stdout over and over.

# -*- coding: utf-8 -*-
# original name: process_script
# author: dud1337
import sys
from multiprocessing import Process, current_process
from time import sleep

def test_function(interval):
# Prints a message every n seconds
	s = 'Hi, I\'m ' + current_process().name
	t = interval
	while True:
		sleep(t)
		print s
	
processA = Process(name='process A!', target=test_function, args=(3,))
processB = Process(name='process B.', target=test_function, args=(4,))

print 'ctrl+C to exit'

processA.start()
processB.start()

# To exit nicely
try:
	while True:
		sleep(1)
except KeyboardInterrupt:
	processA.terminate()
	processB.terminate()
	print '\nSeeya!'
	sys.exit()

This script creates two processes, each printing to stdout over and over. Their output looks the same:

i@pc:~$ python whichever_script.py
ctrl+C to exit
Hi, I'm process A!
Hi, I'm process B.
Hi, I'm process A!
^C
Seeya!

So what's the difference? Run both scripts and then the following

i@pc~$ ps -ef | grep -E "(process|thread)_script"
UID	PID	PPID	...	CMD	# this part won't be grep'd
you	238	138	...	python thread_script.py
you	338	139	...	python process_script.py
you	339	338	...	python process_script.py
you 	340	338	...	python process_script.py

So process_script.py is 3 processes, and thread_script.py is just 1. Why 3? The two processes + the "main" process running the sleepy while loop.

Notice the parent process id (PPID) for the last two is the second one. The two Processes in our code was created by the parent process, the main code. Now with -L

i@pc~$ ps -eLf | grep -E "(process|thread)_script"
UID	PID	PPID	LWP	...	CMD	# no grepperino
you	238	138	238	...	python thread_script.py
you	238	138	239	...	python thread_script.py
you	238	138	240	...	python thread_script.py
you	338	139	338	...	python process_script.py
you	339	338	339	...	python process_script.py
you 	340	338	340	...	python process_script.py

LWP (Light Weight Processes) are threads. As you can see they share the ID-space with regular processes, and their PID is still that of the primary python thread_script.py. Also contrast the PPID, the terminal that spawned the base process, unlike for processes.

Try running kill 239 and kill 339. The former will kill the whole threaded process, the later will just kill one of the subprocesses ("Hi, I'm process A!" will stop).

2. OK. So, how're they really different?

First off: PyInstaller with --onefile on windows does not like multiprocessing!

Source. This is a weird one, but a very annoying one. If you ever want to turn your python files into a single exe, try to use threading.

↑ Top ⌂ Home