Evolution of Computing Platforms

• Computing technology has changed platforms and environments over the last 60 years.
• Distributed computing systems use multiple computers on the Internet to solve large problems, becoming data-intensive and network-centric.
• Internet applications utilizing parallel and distributed computing enhance quality of life and information services.

Internet Computing

• Billions of people use the Internet daily, demanding high-performance computing from supercomputers and data centers.
• The Linpack Benchmark is no longer optimal due to high demand, which increased the need for high-throughput computing (HTC) systems, instead of high-performance computing (HPC)
• HTC systems use parallel and distributed computing, fast servers, storage, and high-bandwidth networks to advance network-based computing and web services.

Platform Evolution

• There have been five generations of computer technology, each lasting 10-20 years with overlaps of about 10 years.
• From 1950-1970, mainframes like IBM 360 and CDC 6400 met the needs of large businesses and governments.
• From 1960-1980, minicomputers such as DEC PDP 11 and VAX Series were more popular in small businesses and colleges.
• From 1970-1990, personal computers with VLSI microprocessors became widespread.
• From 1980-2000, portable computers and pervasive devices appeared commonly.
• Since 1990, HPC and HTC systems hidden in clusters, grids, or Internet clouds have proliferated and are used by both consumers and high-end services.
• The trend is now to leverage shared web resources and large amounts of data over the Internet
• Supercomputers (MPPs) shift into cooperative computer clusters to share resources.
• Peer-to-peer (P2P) networks are formed for distributed file sharing and content delivery using many client machines globally.
• P2P, cloud computing, and web service platforms focus more on HTC than HPC, and clustering and P2P lead to computational grids or data grids.

High-Performance Computing

• HPC systems prioritize raw speed performance, increasing from Gflops in the early 1990s to Pflops in 2010, driven by scientific, engineering, and manufacturing demands.
• Supercomputer users are a small fraction of computer users (~10%). Most users now use desktop computers or servers for Internet searches and market-driven computing tasks.

High-Throughput Computing

• Market-oriented systems are shifting from HPC to HTC, focusing on high-flux computing.
• HTC's key application is simultaneous Internet searches and web services by millions of users, shifting the performance goal to throughput, and not just speed
• HTC technology improves batch processing speed and addresses cost, energy, security, and reliability concerns in data and enterprise computing centers.

New Computing Paradigms

• SOA, Web 2.0, and virtualization enable Internet clouds as a new computing paradigm.
• RFID, GPS, and sensors have enabled the Internet of Things (IoT).
• Clusters, MPPs, P2P networks, grids, clouds, web services, social networks, and IoT may blur in the future, with clouds seen as grids or clusters modified by virtualization.

Computing Paradigm Distinctions

• Distributed computing is the opposite of centralized computing.
• Parallel computing overlaps with distributed and cloud computing
• Centralized computing is a computing paradigm where all the computer resources are centralized into the same physical system with fully shared resources
• Parallel computing involves tightly or loosely coupled processors with centralized/distributed memory with shared memory or message passing. Computer systems are known as parallel computers and programs are referred to as parallel programs
• Distributed computing studies distributed systems that consist of multiple autonomous computers, each with its own private memory, communicating through a computer network and message passing with programs known as distributed programs
• Cloud computing uses the internet for distributed computing and applications are service or utility computing
• Some prefer concurrent computing (parallel and distributed), ubiquitous computing (pervasive devices), and Internet of Things (networked objects supported by Internet clouds)

Distributed System Families

• Technologies for P2P networks and clusters have been consolidated into national projects establishing wide area computing infrastructures.
• There is a surge in interest in exploring Internet cloud resources for data-intensive applications and are the result of moving desktop computing to service-oriented computing, with server clusters and large databases.
• Grids and clouds, are disparity systems, emphasize resource sharing in hardware and software.
• Massively distributed systems use a high degree of parallelism or concurrency among machines, such as the Tianhe-1A system built in China in October 2010 that contained 86016 CPU cores and 3,211,264 GPU cores
• A typical P2P network may involve millions of client machines working simultaneously. Experimental cloud computing clusters have been built with thousands of processing nodes.
• HPC and HTC systems require multicore processors to handle many computing threads.
• HPC and HTC systems emphasize parallelism and distributed computing along with throughput, efficiency, scalability, and reliability
• Key design objectives are Efficiency, Dependability, Adaptation, and Flexibility.
• Efficiency is the utilization rate of resources by exploting massive parallelism in HPC and job throughput in HTC, along with data access, storage and power
• Dependability measures the reliability and self management to ensure high throughput service and QoS
• Adaptation refers to the ability to support billions of requests over massive data sets and virtualized cloud resources
• Flexibility refers to the ability of system to run well in both science and engineering(HPC) and business applications (HTC)

Scalable Computing Trends and New Paradigms

• Technology trends drive computing applications, as seen in Jim Gray's "Rules of Thumb in Data Engineering".
• Moore's law says processor speed doubles every 18 months, but it's difficult to say whether this will continue.
• Gilder's law states that network bandwidth has doubled each year, affecting commodity hardware and adoption of commodity technologies in large-scale computing.
• Distributed systems emphasize resource distribution and concurrency or high degree of parallelism (DoP).

Degrees of Parallelism

• Bit-level parallelism (BLP) converts bit-serial processing to word-level processing.
• Instruction-level parallelism (ILP) executes multiple instructions at once. ILP requires branch prediction, scheduling, speculation, etc.
• Data-level parallelism (DLP) uses SIMD and vector machines. DLP requires hardware/compiler support.
• Task-level parallelism (TLP) has become more relevant with multicore processors. TLP struggles with programming/compilation for efficient multicore execution

Innovative Applications

• HPC and HTC systems require transparency in data access, resource allocation, process location, concurrency, job replication, and failure recovery.

Applications Table

• Science and Engineering domain: Includes Scientific simulations and genomic analysis
• Business, education, services Industry and healthcare domains Include Earthquake prediction, global warming, telecommunication, content delivery, banking and insurance and air traffic control
• Internet/Web services and Government Applications include internet search, data centers, traffic monitory and cybersecurity
• Mission Critical Applications include Military command/control and crisis management

Utility Computing Trends

• Computing paradigms are ubiquitous, reliable, scalable, autonomic, composable with QoS/SLAs, and realize the computer utility vision.
• Utility computing is a business model where customers receive computing resources for a fee and grid/cloud platforms regarded as service providers
• Distributed cloud applications run on any server, facing technological challenges in network efficiency, scalable memory/storage, distributed OS, new programming models, etc.

New Technologies Hype Cycle

• Emerging technologies go through a hype cycle involving inflated expectations, disillusionment, and then gradual enlightenment to a productivity plateau.
• The hype cycle helps in understanding the maturity and adoption timeline of new technologies like clouds, biometric authentication, and interactive TV
• Hollow circles will mainstream in two years, gray circles in 2-5 years, solid circles in 5-10 years and the triangles will take more than 10 years

Internet of Things (IoT)

• The concept of the IoT was introduced in 1999 at MIT and refers to the networked interconnection of everyday objects, tools, devices, or computers
• The IoT is a sensor network that interconnects things varying in time and place using RFID or other electronic technology.

IPv6

• The IPv6 protocol assigns an IP address to all the objects on Earth
• Objects and devices are instrumented and interconnected to communicate in 3 patterns: H2H, H2T, and T2T
• Connections are made at the PC and on the move in the day or night.
• Dynamic connection will have IoT in its infancy with prototypes and cloud computing with cloud researchers to support the prototype .
• A smart Earth must have intelligent cities, abundant resources, efficient telecommunications with green IT and good infrastructure.

Cyber-Physical

• Cyber-physical systems (CPS) integrate computational processes with the physical world, merging cyber aspects with physical objects and merges communication, computation and control.
• The IoT emphasizes connections between physical objects. The CPS emphasizes exploration of virtual reality (VR) applications with a transformation of how we interact