Concurrency and Parallelism: Threads vs Coroutines

• Concurrency involves separating tasks for interleaved execution.
• Parallelism allows for simultaneous execution of multiple tasks, speeding up processing.

Threads vs. Coroutines

• Threads are preemptively scheduled by the operating system using a scheduling algorithm in the kernel.
• Coroutines are cooperatively multitasked, with the programmer or programming language determining when to switch between them. This often occurs within a single thread.
• Coroutine switches are cooperative because the programmer dictates when they happen, unlike threads that are preemptively switched by the OS.
• The OS kernel is not involved in coroutine switches.

Kernel Threads and User Threads

• Languages supporting native threads can run user threads on the OS's kernel threads.
• Each process has a minimum of one kernel thread.
• Kernel threads within a process share memory space.
• A process owns resources like memory, file handles, and sockets, which are shared among its kernel threads.
• The OS scheduler, part of the kernel, allocates time slices to each thread.
• If a thread doesn't finish within its time slice, the scheduler preempts it.
• On a multi-processor machine, threads can run in parallel, with each thread potentially scheduled on a separate processor.
• On single-processor machines, threads are concurrent due to timeslicing but cannot run in parallel.

Coroutines and Generators

• Coroutines and generators implement cooperative functions.
• They run in a single thread and yield to other functions as determined by the programmer, instead of being scheduled by the OS.
• Languages like Python and ECMAScript 6 use generators to build coroutines.
• Async/await, found in languages like C#, Python, and Rust, is built on generator functions that yield futures/promises.
• Coroutines may refer to stackful functions, while generators refer to stackless functions.
• Fibers, green threads, and lightweight threads are other terms for coroutines or coroutine-like constructs.
• Green threads, like those in early Java, are scheduled by the virtual machine rather than the OS kernel.
• Green threads do not run in parallel.

Resource Consumption

• Threads consume resources, including stack space, with each thread in the JVM having its own stack, typically 1MB.
• The JVM allows configuring the thread stack size.
• Threads incur overhead due to stack usage, thread-local storage, scheduling, context-switching, and CPU cache invalidation.
• Coroutines are more popular for highly concurrent applications because of performance.
• Operating systems limit the number of threads a process can allocate. For example, MacOS will only allow 2000 threads.

Efficiency of Coroutines

• Coroutines can provide a very high level of concurrency with very little overhead.
• Threads are limited to around 30-50 before scheduling overhead reduces useful work.
• Coroutines manage context-switching directly, avoiding scheduler overhead.
• They allow for thousands of routines working together due to their efficiency.
• Coroutines avoid locking on shared data structures by pre-determining switching points, while threads do need to lock shared data.
• Coroutines have lower memory usage because the number of routines is unrelated to memory usage.
• In languages like Python, where threads run concurrently but not in parallel, coroutines offer low memory and free scheduling overhead.

Sequential vs. Concurrent Processing

• Coroutines are a form of sequential processing, with only one executing at any given time.
• Threads are a form of concurrent processing, with potentially multiple threads executing at the same time.
• True threads can be interrupted at almost any point, while coroutines work at well-rehearsed points.

Cooperation vs. Preemption

• Coroutines cooperate by willingly suspending execution to allow other coroutines to run.
• Threads are preempted, meaning their suspension is transparent and enforced by the hardware.

Concurrent Execution and Race Conditions

• Coroutines can be concurrently executed, and race conditions can occur depending on the system.

Analogy

• Coroutines act like jugglers handing off, whereas threads can be interrupted more randomly.

Implementation

• Some thread implementations are more like coroutines, depending on the language.
• In Lua, coroutines are called threads.
• Coroutines typically implement voluntary yielding, so the programmer decides where to yield control.
• Threads are automatically managed by the OS and can run simultaneously on multicore CPUs.

Summary

• A coroutine is a cooperative function that runs asynchronously, while a thread is a light-weight process that runs in parallel.
• Threads can share memory and resources. Threads have a limited memory, while coroutines don't require memory for execution.
• Coroutines are more useful than threads.