Parallel Processing Concepts PDF

What is parallel processing Week 3.1 Parallel Processing is particularly useful when running programs that perform complex computations, and it provides a viable option to the quest for cheaper computing alternatives.  Parallel processing is when a computer runs multiple tasks at the...

What is parallel processing Week 3.1 Parallel Processing is particularly useful when running programs that perform complex computations, and it provides a viable option to the quest for cheaper computing alternatives.  Parallel processing is when a computer runs multiple tasks at the same time. This is useful for speeding up complex computations and making computing cheaper. Example: Imagine you’re baking a cake. If you do everything one step at a time (sequential processing), it will take longer. But if you mix the batter while preheating the oven and preparing the frosting, you’ll save time (parallel processing). Importance: Parallel processing speeds up tasks and makes computing more efficient, which is essential for tasks like scientific simulations, graphics rendering, and large-scale data analysis. Master / Slave Configuration is a single-processor system where extra slave processors are working, managed by primary master processor. It is an asymmetrical system.  What is it? In this setup, one main computer (master) controls several other computers (slaves). The slaves follow the master’s instructions. Example: Think of a master chef (master) in a restaurant kitchen giving orders to assistant chefs (slaves). The master chef decides the menu and the assistant chefs help by following specific tasks. Importance: This configuration allows for efficient task distribution, making the system work faster and more efficiently. Master / Slave Configuration Loosely Coupled Configuration In this type of configuration, there are several complete computer systems with their own memory, I/O devices, CPU, and operating system. Loosely Coupled Configuration  What is it? This setup has multiple independent computers, each with its own memory and operating system, working together. Example: Imagine a group project where each student works on their own part independently. Each student has their own materials and workspace but they collaborate to complete the project. Importance: This setup is flexible and can handle tasks independently, which is useful for distributed computing like cloud services. Symmetric Configuration processor scheduling is decentralized. A single copy of the OS and a table listing each process and its status is stored in memory common and accessible to all the processors Symmetric Configuration  What is it? Here, all processors share the same memory and are equally responsible for running tasks. There is no single master; they work together equally. Example: Think of a team where everyone has access to the same resources and they all work together without a designated leader. Importance: This allows for better load balancing and efficient use of resources, reducing bottlenecks and improving performance. Semaphore are integer variables that are used to solve the critical section problem by using two atomic operations, wait and signal that are used for process synchronization.  What is it? A semaphore is a variable used to control access to a common resource in concurrent processing. It helps prevent conflicts when multiple tasks try to access the same resource. Example: Imagine a bathroom with a key (semaphore). If someone has the key (semaphore is 1), the bathroom is occupied. If no one has the key (semaphore is 0), the bathroom is free. Importance: Semaphores help manage access to resources, ensuring smooth and conflict-free operations in parallel processing. Different methods of parallel processing: Process Level Parallel Processing Thread Level Parallel Processing Job level parallel processing Job level parallel processing In job-level parallel processing, the CPU is allocated to each program or job running on a computer system. This allows multiple programs or jobs to run simultaneously, with each program utilizing its own portion of the CPU's processing power.  Job Level Parallel Processing What is it? Each job or program gets its own CPU time. Example: Running a web browser, a music player, and a game at the same time on your computer. Importance: Allows multiple programs to run simultaneously, improving multitasking. Process Level Parallel Processing In process-level parallel processing, the CPU is allocated to different parts of a working set or a task. This means that different sections of a program or task can be executed concurrently, utilizing different CPU resources.  Process Level Parallel Processing What is it? Different parts of a single task run concurrently. Example: Editing a video while rendering it in the background. Importance: Speeds up complex tasks by dividing them into smaller, manageable parts. Thread Level Parallel Processing Thread-level parallel processing involves dividing individual instructions or tasks within a program into smaller subdivisions called threads. These threads can be processed concurrently, allowing multiple instructions to be executed simultaneously.  What is it? A single program is divided into smaller tasks (threads) that run concurrently. Example: A web browser loading a webpage while also playing a video. Importance: Enhances the performance of individual programs by breaking them down into smaller tasks. Types of locking mechanism Test And Set Mechanism Involves A Special Instruction That Is Executed By A Processor In A Single Machine Cycle. It Checks If A Key Or Lock Associated With A Critical Resource Is Available. If The Key Is Available (Represented By A 0), The Instruction Sets It To Unavailable (1) To Indicate That A Process Has Entered The Critical Region.  What is it? Checks and sets a lock in one operation to control access to a resource. Example: Checking if a bathroom is available and locking it in one action. Importance: Ensures that only one task accesses a resource at a time, preventing conflicts. Types of locking mechanism Semaphores A Semaphore Is A Nonnegative Integer Variable That Acts As A Flag, Indicating Whether A Resource Is Free And Available For Use By A Process. It Supports Two Operations: P (Proberen Means To Test) And V (Verhogen Means To Increment)..  What is it? A flag indicating if a resource is available, supporting two operations: test (P) and increment (V). Example: A signal flag indicating if a parking spot is available. Importance: Manages resource allocation effectively, ensuring smooth parallel processing.  C o n c u r r e n t Program m i n g a n d e v i ce M a n a g e m e n t D Week 3.2 Explicit parallelism refers to the concept of executing a group of instructions simultaneously instead of sequentially. it involves the compiler sending a set of instructions to the processor, which then carries out the instructions in parallel. this approach enhances efficiency and performance.  Explicit Parallelism What is it? Programmers specify which tasks to run in parallel. Example: A chef telling assistants exactly what tasks to do simultaneously. Importance: Gives precise control over parallel tasks, enhancing efficiency. Implicit parallelism refers to a programming language's ability to identify and implement parallel execution without explicit instructions from the programmer. The compiler or interpreter recognizes opportunities for parallelization in the code and automatically carries out those operations concurrently.  What is it? The system automatically identifies and runs parallel tasks. Example: A smart kitchen automatically assigning tasks to assistants. Importance: Simplifies programming by letting the system handle parallelism, improving performance. Ada language a modern programming language widely used for developing critical software applications. it is versatile and supports a wide range of systems, from small real-time embedded systems to large-scale enterprise applications.  What is it? Ada is a modern programming language used for developing critical software applications. It is versatile and supports systems ranging from small embedded systems to large-scale enterprise applications. Real-life Application: Air Traffic Control Systems - Ada is used to develop software for air traffic control, ensuring high reliability and safety. Buffer a shared data area used to facilitate communication between hardware devices or program processes that operate at different speeds or have varying priorities. it allows each device or process to function independently without causing delays or disruptions. Producers and consumers  A buffer is a shared data area that helps different hardware devices or program processes communicate smoothly, even if they operate at different speeds or priorities. Real-life Application: Streaming Video - Buffers are used in streaming services to store video data temporarily, allowing for smooth playback without interruptions. Concurrent programming refers to a programming paradigm that allows multiple instructions or tasks to be processed in parallel, enabling efficient utilization of computational resources. it is the opposite of sequential programming, where instructions are executed one at a time in a specific order.  What is it? Concurrent programming allows multiple instructions or tasks to run at the same time, making efficient use of computational resources. This is different from sequential programming, where tasks are executed one after another. Real-life Application: Web Servers - Concurrent programming allows web servers to handle multiple user requests simultaneously, improving response time and efficiency. Java Java is a programming language and software platform that was developed by Sun Microsystems, Inc. in 1995. It introduced the concept of "write once, run anywhere," which means that programmers can write an application in Java once, and it can run on any computer or operating system without major modifications.  What is it? Java is a programming language and software platform developed by Sun Microsystems in 1995. It allows developers to write an application once and run it on any computer or operating system without major changes. Real-life Application: Mobile Apps - Java is widely used for developing Android applications, making it possible for apps to run on different devices without modification. Device Management Device Management Functions Tracking device status Using preset policies Allocating devices 1. Tracking Device Status 2. What is it? Monitoring the current status of devices in a system. 3. Real-life Application: Printer Management - Tracking which printers are online or offline in an office. 4. Using Preset Policies 5. What is it? Applying predefined rules to manage devices. 6. Real-life Application: Network Policies - Automatically allocating more bandwidth to critical applications. 7. Allocating Devices 8. What is it? Assigning devices to tasks or users as needed. 9. Real-life Application: Data Center Management - Allocating servers to different tasks based on demand. Types of Devices Dedicated devices these devices are assigned to only one job at a time and serve that job for its entire duration Types of Devices Shared devices these devices are assigned to several processes simultaneously. Types of Devices Shared devices these devices are assigned to several processes simultaneously. Types of Devices Virtual devices these are dedicated devices that have been transformed into shared devices through a spooling program Sequential Access Storage Media refers to storage devices or media that store data in a sequential manner, where records are stored one after another in a specific order. These devices are contrasted with Direct Access Storage Devices (DASD), which allow direct access to specific data locations. Sequential Access Storage Media refers to storage devices or media that store data in a sequential manner, where records are stored one after another in a specific order. These devices are contrasted with Direct Access Storage Devices (DASD), which allow direct access to specific data locations.  What is it? Storage devices that store data in a sequential manner, where records are stored one after another in a specific order. Real-life Application: Tape Drives - Used for data backup and archiving, where data is read and written in a specific sequence. activity If you were to become a boss or owner of a company. Make your own design/floorplan of your computers in your company by the use hardware’s to regulate or smoothen the flow of process inside the workplace. Essay: if you were going to become a boss. How are you going to manage to achieve a task/goals and what process base from our discussion are you going to use. ( sequential or parallel processing.