IT1910: Advanced Architectures

The American Heritage Dictionary defines virtual as:
- Existing or resulting in essence or effect, but not in actual fact or form
- Created, simulated, or carried on by means of a computer or computer network
Virtualization of a computer allows a single computer to appear as multiple computers, each with its own operating system and hardware resources
Multiple computers share the same CPU and I/O facilities, with each simulated machine known as a virtual computer or virtual machine

Preemptive multitasking refers to the ability of the operating system to interrupt a task and switch to another task
Concurrency is the perception of having tasks that run at the same time, while parallelism is the actual running of tasks at the same time
Parallelism is a subset of concurrency
Two problems that can occur when multiple threads access shared memory are:
- Data Race: a writer thread modifies the memory while a reader thread is reading from it
- Race Condition: multiple threads access the same shared data and try to change it simultaneously
Solutions to these problems include:
- Synchronization: ensuring that resources are used by only one thread at a time
- Atomic Operations: using special instructions to ensure that shared data is always in a valid state
- Immutable Data: marking shared data as immutable, so that threads can only read from it

Multicore microprocessor: a microprocessor that contains multiple processors (cores) in a single integrated circuit
Task-level parallelism (process-level parallelism): a method of utilizing multiple processors by running independent programs simultaneously
Flynn's Taxonomy: a categorization of parallel hardware based on the number of instruction streams and data streams
- SISD (Single Instruction stream, Single Data stream): a non-parallel computer
- SIMD (Single Instruction stream, Multiple Data streams): a parallel computer, best suited for specialized problems with regularity
- MISD (Multiple Instruction streams, Single Data stream): a parallel computer, used in multiple frequency filters and cryptography
- MIMD (Multiple Instruction streams, Multiple Data streams): a parallel computer, currently the most common type, used in supercomputers and multi-core PCs

MIMD relies on multiple processes or threads to keep multiple processors busy
Hardware multithreading allows multiple threads to share the functional units of a single processor
Types of multithreading:
- Fine-grained multithreading: switches between threads on each instruction, resulting in interleaved execution of multiple threads
- Coarse-grained multithreading: switches threads only on expensive stalls, such as last-level cache misses
- Simultaneous multithreading (SMT): a variation of hardware multithreading that uses the resources of a multiple-issue, dynamically scheduled pipelined processor to exploit thread-level parallelism and instruction-level parallelism

Vector architecture: a set of vector registers that collect data elements from memory, operate on them sequentially, and write the results back to memory
Instruction set architectures (scalar architectures): a single vector instruction that specifies a great deal of work, equivalent to executing an entire loop
Benefits of vector architecture: independence of computation within a vector, allowing for efficient processing of large datasets

HPC refers to the use of aggregated computing power to handle computation and data-intensive tasks
Clustering: a group of loosely coupled computers that work together as a unit, with each computer (node) having its own CPU, memory, and I/O facility
Characteristics of clustering:
- Increasing computing power by combining individual systems
- Creating fault-tolerant systems
- Creating high-availability systems
- Load-balancing systems with large workloads
Models of clustering:
- Shared-nothing model
- Shared disk model

Cloud computing: a conceptual expansion of client-server computing, where many functions of an organization's data center are moved to services on the Internet
Benefits of cloud computing:
- Simplified data center
- Cost reduction
- Strong support for user collaboration
- Scalability
- Access to files from anywhere the Internet is available
Risks of cloud computing:
- Security concerns
- Dependence on the cloud service provider
- Outages or connectivity issues
- Data leakage or theft

GPU: a processor optimized for 2D and 3D graphics, video, visual computing, and display
GPU computing: computing via a parallel programming language and API, without using the traditional graphics API and graphics pipeline model
APIs used in rendering and computing:
- Direct3D
- OpenGL
- OpenCL
- Vulkan### Multithreading
A gentle introduction to multithreading is available online as a web log post.

A vector processor is a type of microprocessor that is designed to perform vector calculations.
It is a central processing unit (CPU) that is capable of executing the same instruction on multiple pieces of data simultaneously.

High-performance computing (HPC) refers to the use of high-performance computers and algorithms to solve complex problems.
HPC is used to solve complex problems in fields such as science, engineering, and finance.

Computer systems architecture is a field of study that deals with the design and organization of computer systems.
The book "Computer Systems Architecture" by A. Yadin, published in 2016, is a comprehensive resource on this topic.