Understanding concurrency

Editor's Notes

• #10: Why the synchronous version rocks? It is easy! Easy to debug. One train of thought Problems with synchronous version Its dead slow! Although being slower is not necessarily a big issue If it runs rarely If the completion time is within SLA What if your program run frequently(Those daily tasks)? What if it takes hours to run? Then we have to think about something better!
• #11: download_all_sites() changed from calling the function once per site to a more complex structure In this version, you’re creating a ThreadPoolExecutor, which seems like a complicated thing. Let’s break that down: ThreadPoolExecutor = Thread + Pool + Executor. This object is going to create a pool of threads, each of which can run concurrently. The Executor is the part that’s going to control how and when each of the threads in the pool will run. map() method runs the passed-in function on each of the sites in the list. The great part is that it automatically runs them concurrently using the pool of threads it is managing. The other interesting change in our example is that each thread needs to create its own requests.Session() object. Because the operating system is in control of when your task gets interrupted and another task starts, any data that is shared between the threads needs to be protected, or thread-safe. Threading.local() creates an object that look like a global but is specific to each individual thread. How to decide the number of thread in the pool? This require a little hit and trial. You might expect that having one thread per download would be the fastest but, at least on my system it was not. Why is that? There are two main competing force working here: Speed up due to concurrency Overhead of creating & destroying thread On my machine, the optimal value was 10 threads. Another interesting thing is, with 10 threads, the time has reduced slightly more than 1/10th of the original time. Why is that? Why the threading version rocks? Its fast! Or atleast faster than the synchronous version. But why? Allow your program to overlap the waiting times and get the final result faster Problems with threading version Little more code to make it happen and think about how to keep objects and code thread safe. Threads can interact in ways that are subtle and hard to detect. These interaction can cause race conditions which can result random, intermittent bugs that are difficult to debug.
• #14: Why the asyncio version rocks? It is faster than the threading version. It forces you to think about when a given task will get swapped out, which can help you create a better, faster, design. It is much more scalable. With threading version you have to fine tune the number of thread based on problem and hardware you are working with. Problems with asyncio version You need special async versions of libraries to gain the full advantage of asyncio. Had you just used requests for downloading the sites, it would have been much slower because requests is not designed to notify the event loop that it’s blocked. All of the advantages of cooperative multitasking get thrown away if one of the tasks doesn’t cooperate. A minor mistake in code can cause a task to run off and hold the processor for a long time, starving other tasks that need running. There is no way for the event loop to break in if a task does not hand control back to it.
• #15: Code looks is much shorter than asyncio version multiprocessing on high level create a new instance of Python interpreter to run on each CPU and then farming out part of our program to run on it. Why the multiprogramming version rocks? Relatively easy to setup & requires little extra code Takes full advantage of the CPU power on your computer Problems with asyncio version Slower than asyncio and threading Require some setup from synchronous version Spent some time thinking about which variables will be accessed in each process
• #17: This code calls cpu_bound() 20 times with a different large number each time. It does all of this on a single thread in a single process on a single CPU. Unlike the I/O-bound examples, the CPU-bound examples are usually fairly consistent in their run times. Why is that? Clearly we can do better than this. This is all running on a single CPU with no concurrency. How much do you think rewriting this code using threading or asyncio will speed this up?
• #18: In your I/O-bound example, much of the overall time was spent waiting for slow operations to finish. threading and asyncio sped this up by allowing you to overlap the times you were waiting instead of doing them sequentially. On a CPU-bound problem, however, there is no waiting. The CPU is cranking away as fast as it can to finish the problem. One CPU is doing all of the work of the non-concurrent code plus the extra work of setting up threads or tasks.
• #19: Why the multiprocessing version rocks? Easy to setup with very little code Take full advantage of CPU power on your computer Problems with multiprocessing version Splitting your problem up so each processor can work independently can sometimes be difficult. Many solutions require more communication between the processes. This would add more complexity to your synchronous version and sometime these communication overhead can outweigh the benefit.