12 Parallelize your Python.pptx

PA R A L L E L COMPUTING IN PYTHON M U LT I C O R E DESIGN IN MODERN COMPUTERS Multicore Design Multiple independent processing units on a single computing component Example One CPU processor with six physical cores Each core has two logical cores T...

PA R A L L E L COMPUTING IN PYTHON M U LT I C O R E DESIGN IN MODERN COMPUTERS Multicore Design Multiple independent processing units on a single computing component Example One CPU processor with six physical cores Each core has two logical cores Total number of cores: 12  Parallel Computing Approaches in Python Multiprocessing Threading Differences Between Process PA R A L L E L and Thread COMPUTIN Process: Independent execution units G IN Thread: Subset of a PYTHON process  DEFINITION OF PROCESS AND THREAD Process Thread Instance of a program Subprocess within the (e.g., Python interpreter, process Jupyter notebook) Each process can have Created by the operating multiple threads system to run a program Threads share the same Has its own memory memory block within the block process  M E M O RY S H A R I N G I N T H R E A D S Variables or objects for multiple threads in a process are all Shared Variables in shared Threads Changing one variable in one thread will change it for all other threads Independent Variables in different processes are not shared Variables in Changing one variable in one process will not affect other processes Processes Advantages and Processes and threads have their own advantages and disadvantages Disadvantages They can be used in different tasks to maximize benefits PYTHON'S GIL PROBLEM Python's Design and GIL Python was designed before multicore processors were common Global Interpreter Lock (GIL) limits to one native thread at a time Prevents multiple threads from running simultaneously Workarounds and Multiprocessing Multithreading workarounds exist in Python Multiprocessing library is a common solution  D I S A D VA N TA G E S O F U S I N G PA R A L L E L C O M P U T I N G Code Complexity Time Consumption for Small Tasks Overheads required to launch and Initialization and maintenance of new maintain new processes processes take time  INTRODUCTION TO M U LT I P R O C E S S I N G L I B R A R Y Multiprocessing Library Overview Python's standard library for parallel computing Supports parallel computing using processes Features Numerous features available Official documentation recommended for detailed information Getting Started Import the library Print the total number of CPUs available for parallel computing  Pool Class Used for parallel computing USING POOL CLASS FOR PA R A L L E L Common Methods apply map COMPUTIN in Pool Class apply_async map_async G Takes two arguments: func Using map and iterable Applies func to Function each element in iterable Collects the results  pool.apply Accepts more Function arguments POOL.APPL Locks the Y AND pool.map main program until all POOL.MAP Function processes are finished FUNCTIONS Both functions lock the main Common Feature program until all processes are finished P O O L. A P P LY _ A S Y N C A N D POOL.MAP_ASYNC FUNCTIONS Ordered Results Unordered Use pool.apply_async or Results Use pool.apply_async or pool.map_async pool.map_async Submit all processes at once Submit all processes at once Retrieve results as soon as they are Retrieve results as soon as they are finished finished  Execution Time Changes Comparison of serial and EXECUTIO parallel versions N TIME Impact of number of data points CHANGES Serial Version WITH Better performance until D ATA a certain point POINTS Parallel Version Performance improvement with more data points  Execution Time for Small Data Points Serial version is faster for data points below 10000 C O M PA R I S O Parallel version has overhead for N OF SERIAL launching and maintaining processes AND Execution Time for Large Data PA R A L L E L Points VERSIONS Parallel version is faster for data points above 10000 Example: For 107 data points, parallel version takes less than 10 s Serial version takes roughly 50 s for the same data points  Introduction to Joblib Third-party package for parallel computing in Python Useful for daily tasks INTRODUCTIO N TO JOBLIB Installation PA C K A G E Run pip install joblib Usage Perform parallel computing easily Many other usages  Convenience of joblib package Parallel part of the code becomes one line Parallel class Provides a convenient interface for multiprocessing SIMPLIFYIN module Delayed function G PA R A L L E L COMPUTING Captures arguments of the target function WITH JOBLIB CPU usage Use all available CPUs with n_jobs=-1 Use all CPUs but one with n_jobs=-2 Verbose argument Turn on to output status messages  Joblib Backends Multiple backends available Different ways to perform parallel computing JOBLIB Multiprocessing Backend BACKENDS Creates a multiprocessing pool Uses separate Python worker processes Executes tasks concurrently on separate CPUs