This is the first of a series of posts about asyncio, the library that ships with Python 3.4, and how it compares with the two classic ways of doing several things at once: threads and processes.
I want to put the three side by side on performance, on how comfortable they are to write, and on what each one gives you and what it costs you. Same problem, three solutions, numbers on the table.
The series, as I have it planned:
- Fetching web pages (this post)
- Database access
- Slow tasks
- Disk access
Quick summary: pros and cons of each method
Before the numbers, the short version of what each one gets you and where it hurts:
| Method | Pros | Cons |
|---|---|---|
| Threads | Sequential flow of execution. Simple to program. | Not real multitasking. Blocking. Low performance. |
| Processes | Real multiprocessing. Sequential flow of execution. Simple to program. | Loads the system. No threads: it spawns operating system processes. |
| asyncio | Uses coroutines to improve performance. Non blocking by design. Does not load the system. | Only in Python 3.4 and up. The code is not linear. |
Fetching web pages
Talking to a web server is one of the most blocking things a program does. The CPU sits idle while the network answers. That makes it a good first case: any way of “running several things at once” should help here, and we get to see which one helps most.
For the test I wrote three small programs that do the same job, one with threads, one with processes and one with coroutines.
How I tested
The list of URLs is built at run time out of Bing searches, one per number. The two globals are set by the test loop for each run:
# Global configs
MAX_CONCURRENCE = 0 # the test loop sets it: 5, 10 and 15
WEB_PAGES_PRELOADED = [] # the test loop fills it: 50, 100 and 200 URLs
BASE_URL = "http://www.bing.com/search?q=%s&go=&qs=n&form=QBLH&filt=all&pq=hello&sc=8-1&sp=-1&sk=&cvid=3c6b1fd5cbe0456b8c2370b57dc7ad38"
What I measure is how each method behaves when you combine the number of URLs with the number of connections allowed at the same time. Every method gets the same cap: 5, 10 and 15 concurrent connections, over 50, 100 and 200 URLs. The same job, made concurrent three different ways, and the times compared.
These are the imports the three versions share:
import timeit
import asyncio
import urllib.error
import urllib.request as req
import aiohttp
from multiprocessing import Pool
from threading import Thread, Semaphore
And here is the code for each method. The loop that runs them all and times them is at the end.
Threads code
# --------------------------------------------------------------------------
# Threads
# --------------------------------------------------------------------------
def download_threads(url, sem_threads):
try:
response = req.urlopen(url)
data = response.read()
except urllib.error.URLError:
print("Thread error in URL: ", url)
sem_threads.release()
def test_threads():
sem_threads = Semaphore(MAX_CONCURRENCE)
th = []
th_append = th.append
for page in WEB_PAGES_PRELOADED:
sem_threads.acquire()
t = Thread(target=download_threads, args=(page, sem_threads))
t.start()
th_append(t)
# Wait for the threads to finish
for x in th:
x.join()
Processes code
# --------------------------------------------------------------------------
# Processes
# --------------------------------------------------------------------------
def download_processes(url):
try:
response = req.urlopen(url)
data = response.read()
except urllib.error.URLError:
print("Process error in URL: ", url)
def test_processes():
mp = Pool(MAX_CONCURRENCE)
mp.map(download_processes, WEB_PAGES_PRELOADED)
asyncio code
# --------------------------------------------------------------------------
# Python 3 coroutines
# --------------------------------------------------------------------------
@asyncio.coroutine
def download_coroutine(url, sem_coroutines):
with (yield from sem_coroutines):
response = yield from aiohttp.request('GET', url)
data = (yield from response.read())
def test_coroutines():
sem_coroutines = asyncio.Semaphore(MAX_CONCURRENCE)
f = asyncio.wait([download_coroutine(page, sem_coroutines) for page in WEB_PAGES_PRELOADED])
asyncio.get_event_loop().run_until_complete(f)
The loop that times it all
# --------------------------------------------------------------------------
# Time it!
# --------------------------------------------------------------------------
if __name__ == '__main__':
testing_cases = {
"Threads": "test_threads",
"Python 3 coroutines": "test_coroutines",
"Processes": "test_processes",
}
print("[*] Starting test")
for requests in [50, 100, 200]:
print(" " * 3, "- Requesting %s URLs:" % requests)
for concurrence in [5, 10, 15]:
print(" " * 5, "+ concurrence %s:" % concurrence)
WEB_PAGES_PRELOADED = [BASE_URL % w for w in range(requests)]
MAX_CONCURRENCE = concurrence
for case_name, case_function in testing_cases.items():
print(" " * 8, "> ", case_name, "time: ",
timeit.timeit("%s()" % case_function,
setup="from __main__ import %s" % case_function,
number=1),
"seconds")
print("[*] Tests end")
Results
This is what the run printed:
cr0hn.com # python3.4 network_coroutines.py
[*] Starting test
- Requesting 50 URLs:
+ concurrence 5:
> Threads time: 5.643185321998317 seconds
> Python 3 coroutines time: 6.230320422007935 seconds
> Processes time: 5.842057971982285 seconds
+ concurrence 10:
> Threads time: 2.679791122995084 seconds
> Python 3 coroutines time: 2.809677056997316 seconds
> Processes time: 3.4920346940052696 seconds
+ concurrence 15:
> Threads time: 1.9668658949958626 seconds
> Python 3 coroutines time: 2.065839316986967 seconds
> Processes time: 2.3969717799918726 seconds
- Requesting 100 URLs:
+ concurrence 5:
> Threads time: 10.728528372012079 seconds
> Python 3 coroutines time: 10.180934254982276 seconds
> Processes time: 12.539949495985638 seconds
+ concurrence 10:
> Threads time: 6.375440713018179 seconds
> Python 3 coroutines time: 5.942036010994343 seconds
> Processes time: 6.644756149005843 seconds
+ concurrence 15:
> Threads time: 3.7005897909984924 seconds
> Python 3 coroutines time: 3.8714171170140617 seconds
> Processes time: 4.932254218001617 seconds
- Requesting 200 URLs:
+ concurrence 5:
> Threads time: 21.58786587699433 seconds
> Python 3 coroutines time: 20.815504188009072 seconds
> Processes time: 23.112755888025276 seconds
+ concurrence 10:
> Threads time: 11.149541911989218 seconds
> Python 3 coroutines time: 10.273655647004489 seconds
> Processes time: 13.324604407011066 seconds
+ concurrence 15:
> Threads time: 7.853176967008039 seconds
> Python 3 coroutines time: 7.413613215001533 seconds
> Processes time: 9.253623016993515 seconds
[*] Tests end
Two things stand out. Threads and coroutines are neck and neck: with 50 URLs threads are a few tenths ahead, with 100 it is a toss up, and with 200 coroutines win in all three settings, by up to half a second. Processes come last every single time, and the gap grows with the number of URLs: at 200 URLs and 15 connections they need 9.2 seconds against the 7.4 of the coroutines. Starting a process is a lot more expensive than starting a thread or a coroutine, and with this many short requests it shows.
Raising the concurrency helps the three of them about the same. Going from 5 to 15 connections cuts the time to roughly a third whichever method you pick, which is what you would expect when the bottleneck is waiting on the network and not the CPU.
So for plain web fetching there is no big winner between threads and asyncio on raw speed. The difference is in how the code is written and in what each one costs the system, and that is what the rest of the series is for. Next one: database access.
