Mono- and Multiprocessor Systems

Mono-Processor Systems

• These systems use a classic computer architecture developed by John von Neumann in 1952.
• Microprocessors receive input, execute processes, and distribute output.
• They are single-core systems, running one task for one user.
• This is known as input-processing-output (IPO) or single instruction, single data (SISD).
• They are subject to the von Neumann bottleneck.

Multiprocessor Systems

• These systems offer a solution to the von Neumann bottleneck.
• The two basic procedures to distribute tasks are Symmetrical multiprocessing (SMP) and Asymmetrical multiprocessing (AMP or ASMP)
• Symmetrical multiprocessing allows any processor or core can execute tasks, useful even if software is not optimized for multiprocessing.
• Asymmetrical multiprocessing designates one processor acts as a manager, distributing tasks using different algorithms.
• The n-way symmetric multiprocessing procedure achieves the best performance and the best resource usage.
• Symmetric multiprocessing systems with many users or tasks provide a good solution to the von Neumann bottleneck by distributing input and generating multiple concurrent output streams.

Multiple Core Processors

• These CPUs have multiple physical processing units that act like multiple CPUs that share a memory cache.
• The total number of cores in a system represents the number of processes that can run in parallel.
• Sharing a memory bus can create a slight performance bottleneck.
• The number of physical cores is key when estimating performance gains in parallel design.

Hyper-Threading (HT)

• It is a proprietary Intel technology improving parallelization on x86 processors.
• First introduced in Xeon server processors in 2002, it allows single-processor chips to run with two virtual cores and execute two tasks simultaneously.

Hardware Threads

• Each logical core is called a hardware thread.
• Hardware threads are scheduled separately by the operating system (OS) scheduler.
• While appearing as separate cores to the OS, only one logical core per physical core executes a software instruction at a time.

Processor Configurations and Task Performance

• Single core, single-core chip: 1 task at a time
• Single processor, HT-enabled single-core chip: 2 tasks at a time
• Single processor, dual-core chip: 2 tasks at a time
• Single processor, HT-enabled dual-core chip: 4 tasks at a time
• Single processor, quad-core chip: 4 tasks at a time
• Single processor, HT-enabled quad-core chip: 8 tasks at a time

Flynn's Taxonomy

• Flynn's Taxonomy classifies computer architectures into four categories. There are based on concurrent instruction and data streams.
• Single Instruction, Single Data (SISD): Single control unit and instruction stream, executes one instruction at a time, like single-core processors.
• Single Instruction, Multiple Data (SIMD): Single instruction stream applied to multiple data streams in parallel. Useful in speculative scenarios with multiple data algorithms.
• Multiple Instructions, Single Data (MISD): Applies multiple instructions in parallel on one data stream, generally used for fault tolerance.
• Multiple Instructions, Multiple Data (MIMD): Multiple instruction and data streams enables true parallelism, common in modern computers.

Serial Computing

• Traditionally, software is written for serial computation.
• Problems are broken into a discrete series of instructions, executed sequentially on a single processor, with only one instruction at a time.

Parallel Computing

• It involves simultaneous use of multiple compute resources for a single problem.
• Problems are broken into discrete parts that can be solved concurrently.
• Each part being further broken down into a series of instructions and execute simultaneously on different processors.
• Employing an overall control/coordination mechanism
• Advantages include saving time and money, solving larger problems, leveraging non-local resources, and better use of computing power.
• Types of parallel computing: Bit-level, instruction-level, data-level, and task parallelism.

Applications of Parallel Computing

• Common use cases are databases and data mining.
• Real-time simulation of systems.
• Science and Engineering.
• Advanced graphics, augmented reality, and virtual reality
• The limitations include complex communication and synchronization, algorithms that handle parallel mechanisms, and programs with low coupling and high cohesion.
• Requires skilled programmers.

Parallel Programming Architecture

• Visual Studio and .NET Framework 4 enhance support using runtimes, class libraries, and diagnostic tools.
• The architecture allows writing efficient, scalable parallel code without direct thread or thread pool manipulation.

Task Parallel Library (TPL)

• The Task Parallel Library (TPL) provides public types and APIs in System.Threading.Tasks namespaces.
• Its purpose is to simplify parallelism and concurrency for developers.
• The process automatically scales concurrency to use available processors efficiently.
• In addition, TPL manages thread scheduling, partitioning, cancellation, and state.
• TPL maximizes code performance by simplifying focused, efficient implementation.

System.Threading.Tasks Namespace key classes

• Task represents an asynchronous operation that can be cancelled or waited on
• Task<TResult> represents an asynchronous operation that can return a value
• TaskFactory class creates and starts tasks.
• TaskScheduler class provides default infrastructure for scheduling threads.
• TaskAsyncEnumerableExtensions configures task-related behaviors.
• TaskCanceledException communicates cancellation.
• TaskCompletionSource provides access to the producer within the Task property.

System.Threading.Tasks.Task Class

• It is a class to execute asynchronously as a thread pool thread, per TAP.
• The non-generic class doesn't return results, and the generic Task<T> class must be used to return any values.
• Tasks can be created using lambda expressions, Action delegates, or delegates.

Task Class Properties & Methods

• Status gets the TaskStatus
• CompletedTask gets success state
• IsCanceled indicates canceled execution
• IsCompleted indicates completion
• IsCompletedSuccessfully indicates that a tasks ran to completition sucessfully
• ContinueWith(Action<Task>) creates asynchronous continuation
• Run(Action) queues specified work
• Start() schedules task execution
• Wait() waits for completion
• WhenAll(Task[]) creates task when all tasks complete

Parallel LINQ (PLINQ)

• Parallel LINQ (PLINQ) implements the Language-Integrated Query (LINQ) pattern in parallel.
• Contains all LINQ functions as extension methods for System.Linq, including parallel operations.
• PLINQ combines LINQ syntax with parallel programming.
• While LINQ queries are sequential and potentially slow, PLINQ performs queries on multiple threads and cores.
• .NET facilitates seamless LINQ to PLINQ conversion using the AsParallel() method.

Parallel query

• They are similar to LINQ to Objects queries.
• PLINQ queries operate on in-memory IEnumerable or IEnumerable<T> collections, with deferred execution.
• PLINQ's primary distinction is its partitioning of data and concurrent execution of queries on worker threads.
• PLINQ offers performance improvements with the AsParallel query operation, but can introduce complexities and is not always faster.

ParallelEnumerable Class

• The ParallelEnumerable class is used in the System.Linq namespace with the System.Core assembly.
• It contains non-standard query operators and performs functions with parallel execution.
• AsParallel specifies query sections should be parallelized if possible.
• AsSequential makes queries run sequentially.
• AsOrdered preserves the source sequence order.
• AsUnordered indicates PLINQ does not need to preserve source sequence order.
• ForAll enables results without merging to consumer thread.
• Aggregate overload aggregates thread-local partitions and results of all partitions.
• WithDegreeOfParallelism sets maximum CPUs PLINQ uses.
• WithMergeOptions provides hints for merging parallel results.
• WithExecutionMode dictates if PLINQ should parallelize query even if default behavior is sequential.

PLINQ Performance Considerations

• PLINQ may have performance overhead, especially partitioning and merging.
• Parallel is Not Always Faster: Parallelization introduces overhead; sequential execution may be better for smaller data with compute-bound operations.
• Avoid I/O operations should be avoided due to thread safety.
• Queries may not run in parallel if Core CLR decides it is sub-optimal, even with AsParallel().

Synchronous Program Execution

• Control does not leave the calling thread in synchronous execution.
• Code executes line by line, with the current thread waiting for function calls to complete before advancing.
• This is a common method and works well with CPU performance increases for quickly completing code.
• With parallel processing, multiple threads can occur concurrently and the main program remains synchronous in the same by using.

Asynchronous Program Execution

• It enables multiple tasks to run concurrently, where a called method is executed and the thread immediately executes the next line of code.
• The asynchronous method may or may not create a thread.
• Results are delivered via callbacks upon completion, and callbacks can be void.
• Asynchronous Programming Model (APM) not supported in .NET Core, Event-Based Asynchronous Pattern (EAP) and Task-Based Asynchronous Pattern (TAP) are implemented for I/O bound and compute-bound tasks.

Implementing Async

• While M1() executes, the thread makes asynchronous calls to M2().
• It passes it a callback function, M3().
• Instead of waiting, the thread executes the rest of the code in M1().
• The CPU executes M2() instantly in another thread or at a later time.
• When M2() completes, it calls M3() with the results.

When to Use Asynchronous Programming

• Use when Direct Memory Access or I/O (files, databases, networks) operations are are used.
• It is implemented via the CPU rather than the main application thread.
• Improve app performance, responsiveness, and thread execution.

Async and Await Implementation

• Async and await keywords are essential in.NET Core.
• These keywords work in the .NET Framework, .NET Core or Windows Runtime to asynchronously create method.
• Methods which define functions by using the async keyword are called async methods.