Big Data Systems: Data Center and Cloud Computing PDF

Big Data Systems Martin Boissier Data Center and Cloud Computing Data Engineering Systems Hasso-Plattner-Institut Timeline I Date Tuesday Wednesday 15.10. /16....

Big Data Systems Martin Boissier Data Center and Cloud Computing Data Engineering Systems Hasso-Plattner-Institut Timeline I Date Tuesday Wednesday 15.10. /16.10 Intro / Organizational Use Case - Search Engines 22.10. / 23.10. Performance Management Intro to GitHub Classroom 29.10. / 30.10. Map Reduce I Map Reduce II 5.11. / 6.11. Map Reduce III Exercise 12.11. / 13.11. Data Centers Cloud 19.11 / 20.11. File Systems Exercise 26.11. / 27.11. Key Value Stores I Key Value Stores II 3.12 / 4.12. Key Value Stores III Exercise 10.12. / 11.12. Stream Processing I Stream Processing II 17.12. / 18.12. ML Systems I Exercise Christmas Break 3 This Lecture 1. Data Centers 2. Virtualization 3. Scheduling 4. Cloud Computing Partially based on § Christos Kozyrakis, Matei Zaharia: cs349d.stanford.edu § Matthias Boehm: https://mboehm7.github.io/teaching/ss19_dbs/index.htm § Hans-Arno Jacobsen: Distributed Systems § Indranil Gupta: CS 425 / ECE 428 Distributed Systems 4 Where are we? § Infrastructure Application / Query Language / Analytics / Visualization Application § Covering Data Processing § Hardware Data Management Big Data § Scheduling Systems File System § Virtualization Virtualization / Container § On-prem and cloud-based OS / Scheduling Infrastructure Hardware 5  Google Hardware First computer First production >50 data centers 1996 cluster 1998 today chaddavis.photography from United States, CC BY 2.0, via Wikimedia Commons Christian Heilmann, CC BY 2.0, via Wikimedia Commons Steve Jurvetson, CC BY 2.0, via Wikimedia Commons 6 Data Centers Anatomy of a Data Center Commodity CPU: Rack: Server: Xeon E5-2440: 6/12 cores 16-64 servers + Multiple sockets, Xeon Gold 6148: 20/40 top-of-rack switch RAM, disks cores Data Center: >100,000 servers Cluster: Multiple racks + cluster switch [Google Data Center, Eemshaven, Netherlands] 8 Data Center – Compute Hardware § The basics Nvidia Tesla Google TPU § Multi-core CPU servers § 1 & 2 sockets Microsoft Catapult § Modern additions § GPUs § FPGAs § Custom accelerators (e.g., ASICS for AI or crypto$) § Custom-design servers § Configurations optimized for major app classes § Few configurations to allow reuse across many apps § Roughly constant power budget per volume [Facebook server configurations] 9  Data Center – Storage Hardware § The basics NVMe Flash JBOD disk array § Disk trays § SSD & NVM Flash § NVMe (up to 14.5 GB/s with PCIe Gen 5) § Modern additions § Non-volatile memory† § New archival storage (e.g., glass) NVM DIMM § Distributed with compute or NAS systems § Remote storage access for many use cases † Discontinued: https://www.anandtech.com/show/17515/intel-to-wind-down-optane-memory-business 10 Data Center – Network Hardware § The basics § 10, 25, and 40GbE NICs 40GbE Switch § 40 to 100GbE switches § Latest ethernet standard: 800 GbE § Clos topologies Smart NIC Microsoft Catapult § Modern additions § Software defined networking § Smart NICs § RDMA § FPGAs 11 Operators TCO Example § TCO = capital (CapEx) + operational (OpEx) expenses § Operators perspective § CapEx: building, generators, A/C, compute/storage/net HW § Including spares, amortized over 3 – 15 years § OpEx: electricity (5-7c/KWh), repairs, people, WAN, insurance, … § Users perspective § CapEx: cost of long term leases on HW and services § OpEx: pay per use cost on HW and services, people [Source: James Hamilton] § Hardware dominates TCO, make it cheap § Must utilize it as well as possible 12 Reliability Metrics § Failure in time (FIT) § Failures per billion hours of operation = 109/MTTF § Mean time to failure (MTTF) § Time to produce first incorrect output (typically for non-repairable systems) § Mean time between failures (MTBF) § Time of normal operation between failures § Mean time to repair (MTTR) § Time to detect and repair a failure MTBF MTTR MTBF MTTR Correct Failure Correct Failure Correct § Steady state availability = MTBF / (MTBF + MTTR) 13 Fault Tolerance (Google) § Yearly Data Center Failures § ~0.5 overheating (power down most machines in efficient international trade § #1 Self-contained package of necessary SW and data (read-only image) § #2 Lightweight virtualization w/ resource isolation via cgroups § Docker Inc. the main force behind Docker project § Docker provides a set of open source tools to build, distribute and deploy applications § Docker Hub the public official containers registry § Docker Engine the primary tool for managing the containers § Client-server application for managing and executing Docker objects 27 Docker Image and Running § Image is a read-only template with instructions for creating a Docker container § Image has a layered structure based on Union File Systems § Each layer depends on the underlying layers § Use Dockerfile to create images Your Website Image § Public images are on Docker Hub Framework Image (Apache) OS Image (Ubuntu) vm mainly isolate Usage: VM: ease fully utilizing the Server Hardware and Isolate tenants Docker: ease deployment 28 Scheduling Scheduling § Cluster Schedulers § Container orchestration: scheduling, deployment, and management § Resource negotiation with clients what are the resources i need § Typical resource bundles (CPU, memory, device) § Examples § Kubernetes § Mesos § YARN § Amazon ECS § Microsoft ACS § Docker Swarm 30 Cluster Scheduling § How to schedule concurrent jobs on a cluster? 10 § Require same resources Job 1 § Processor, memory, disk, network 5 Job 2 3 Job 3 § Scheduling goals § Good throughput or response time get as much done as 0 2 5 possible § High utilization of resources we want to have a good Response time Job Length Arrival 1 10 0 2 5 2 § Which job to run when? 3 3 5 31 First-In First-Out Scheduling § Single processor/queue § A.k.a., First-come first-serve 10 Job 1 5 § Maintain jobs in queue in order of arrival Job 2 § When processor free, dequeue head 3 Job 3 Job 1 Job 2 Job 3 0 2 5 0 2 5 10 15 18 Job Length Arrival 1 10 0 § Average completion time high 2 5 2 !"# $ % !"# & % !"# ' $(%$)%$* +' § = = = 14.33 3 3 5 ' ' ' 32 Shortest Task First Scheduling § STF: maintain jobs in queue in decreasing order of run time § When processor free, dequeue head gibt es with and aithout pre-emption, was bedeutet, dass wir einen Job unterbrechen, sobald ein kü rzerer Job reinkommt we can starve Job 3 Job 2 Job 1 0 3 8 18 Job Length Arrival § Shortest avgerage completion time 1 10 0 !"# $ % !"# & % !"# ' $(%(%' &) § = = = 9.66 2 5 0 ' ' ' 3 3 0 § STF is example of priority scheduling § In general, also user provided priority possible 33 Round-Robin Scheduling § Use quantum (time unit) to run portion of task at queue head § Pre-empt processes by saving state and resume later § After pre-empt, add to end of queue Job 1 … 0 2 5 12 Job Length Arrival 1 10 0 2 5 2 3 3 5 § Good for interactive jobs 34 Cloud Scheduling § Multiple users with multiple jobs Client § Users/jobs = tenants Resource Manager § Multi-tenant system § Requirement § Schedule for all jobs Node Node Manager Manager § Fairness every user makes Progress and gets the resources he Paid for § High utilization App Master Container § Yarn § Hadoop Capacity Scheduler Container Container § Hadoop Fair Scheduler 35 Hadoop Capacity Scheduler § Multiple queues we have mutliple tenants, so one Queue, one teneant § Each queue multiple jobs § Each queue portion of cluster (also hierarchies) § Within queue FIFO § Limits in queue § Soft limit: minimum guarantee of capacity § Hard limit: maximum guarantee § No preemption § No stopping jobs, need to wait for jobs to finish to change queue limit § Consider memory requirements § Example § Queue 1: 80% of cluster § Queue 2: 20% of cluster 36 Hadoop Fair Scheduler if we have a single user he will get full cluster § All jobs get equal share of cluster (i.e., memory per default) § Single job -> full cluster § Multiple jobs -> equal number of containers § Divide cluster in pools § One pool per user § Each pool same resources § In pool FIFO, Fair, … § If pool has minimum shares § Minimum % of cluster § If not available, take containers from others § By killing jobs (most recently started) § Set limits on max jobs / pool or user 37  also used by Hadoop fair Scheduler when not only using Memory but e.g. Memory and cpu Dominant Resource Fair Scheduling (DRF) § Mesos scheduler § Give same share of dominant resource to each job § Dominant resource: largest percentage of cluster § Example what are my requirements? § Job 1 per task: 2 CPUs, 8GB RAM: § Job 2 per task: 6 CPUs, 2GB RAM: § Cloud with 18 CPUs, 36GB RAM: & ' ( & § Job 1 requirements: CPU = , RAM = ->RAM dominant '( ) *+ ) + & ' § Job 2 requirements: CPU , RAM = -> CPU dominant '( *+ '( § Job 1 RAM share = Job 2 CPU share § Use linear program or iterative algorithm to solve *∗( & &∗+ & § Job 1 = RAM, Job 2 = CPUs *+ * '( * Paper: Ali Ghodsi, et al.: Dominant Resource Fairness: Fair Allocation of Multiple Resource Types. NSDI’11 38 Cloud Computing Cloud Services § On-demand, remote storage and compute resources, or services § Service Models § Infrastructure as a Service (IaaS): VMs § Platform as a Service (PaaS): MapReduce § Software as a Service (SaaS): Email § Public vs private clouds: § Shared across arbitrary orgs/customers vs internal to one organization § Transforming IT Industry/Landscape § Since ~2010 increasing move from on-prem to cloud resources § System software licenses become increasingly irrelevant § Few cloud providers dominate IaaS/PaaS/SaaS markets (w/ 2018 revenue): Microsoft Azure Cloud ($ 32.2B), Amazon AWS ($ 25.7B), Google Cloud (N/A), IBM Cloud ($ 19.2B), Oracle Cloud ($ 5.3B), Alibaba Cloud ($ 2.1B) 40 Cloud Evolution § The 1950s – Mainframe Computing § 1960 – 1990 § Internet, VMs, VPNs § 1999 – Salesforce, the first SaaS § 2006 – Amazon Web Services § 2008 – Microsoft Azure § 2013 – Google Compute Engine § 2014 – IBM Bluemix §… 41 Cloud – Pros and Cons § Pros § Cons § Fast and efficient access to § Clients need to trust the cloud resources that clients actually providers need § Possible limited access to your § Pay for what you use data § No initial capital expenditure § Possible conflicts with government § Less need for a big administrator regulations and restrictions team § Potential data loss § Self-maintenance and fault § Locked in within the cloud tolerance resources provider’s specification § You may encounter unknown costs (check SLA) more machines spawn, then you wanted to 42 Cloud Economics: For Users § Pay-as-you-go (usage-based) pricing: 100% § Most services charge per minute, per byte, Utilization etc. § No minimum or up-front fee Time § Helpful when apps have variable utilization Service can be upscaled for a limited amount of time § Elasticity: Resources § Using 1000 servers for 1 hour costs the same as 1 server for 1000 hours § Same price to get a result faster! Time Time 43 Cloud Economics: For Providers § Economies of scale: § Purchasing, powering & managing machines at scale gives lower per-unit costs than customers’ § Tradeoff: fast growth vs efficiency § Tradeoff: flexibility vs cost § Speed of iteration: § Software as a service means fast time-to-market, updates, and detailed monitoring/feedback § Compare to speed of iteration with ordinary software distribution 44 Characteristics and Deployment Models § Extended Definition § ANSI recommended definitions for service types, characteristics, deployment models how elastic § Characteristics § On-demand self service: unilateral resource provision § Broad network access: network accessibility § Resource pooling: resource virtualization / multi-tenancy § Rapid elasticity: scale out/in on demand § Measured service: utilization monitoring/reporting § Deployment Models § Private cloud: single org, on/off premises § Community cloud: single community (one or more orgs) § Public cloud: general public, on premise of cloud provider § Hybrid cloud: combination of two or more of the above Source: Teknology Solutions Peter Mell and Timothy Grance: The NIST Definition of Cloud Computing, NIST 2011 45 Example Amazon Services – Pricing (current gen) § Amazon EC2 (Elastic Compute Cloud) § IaaS offering of different vCores Mem node types and generations § On-demand, reserved, and spot instances § Amazon ECS (Elastic Container Service) § PaaS offering for Docker containers EC2 on Demand Pricing § Automatic setup of Docker environment § Amazon EMR (Elastic Map Reduce) § PaaS offering for Hadoop workloads § Automatic setup of YARN, HDFS, and specialized frameworks like Spark § Prices in addition to EC2 prices 46 Cloud Computing Evolution Increasing focus on business logic Functions Containers Virtual Machines Bare Metal Hardware on premise Decreasing concern (and control) over stack implementation § Function-as-a-Service (FaaS) / Serverless computing § Runs functions in a Linux container on events § Used for web apps, IoT apps, stream processing, highly parallel MapReduce and video encoding 47 Serverless Computing § Cloud-native platform for short-running, stateless computation, and event-driven applications § Auto-scalability and maintenance § Pay for what you use (millisecond granularity) § Serverless does not mean no servers, means worry-less servers Azure Functions AWS Lambda Kubernetes Red-Hat Google Functions 48 Summary § Data Centers § Hardware § Characteristics § Virtualization § VM, Containers § Scheduling § FIFO, STF, HFS, DRFS Resources § Cloud Computing § Models § Evolution Time Time 49 Next Part § File Systems Application / Query Language / Application Analytics / Visualization Data Processing Big Data Data Management Systems File System Virtualization / Container OS / Scheduling Infrastructure Hardware 50 Thank you for your attention! § Questions? § In Moodle § Per email: [email protected] § In Q&A sessions 51