Lab #5.1 - Apache Spark Stream Processing - Truck Fleet Lab II PDF
Document Details
Uploaded by PerfectPanda
IE University
Tags
Summary
This document is an introduction to Apache Spark, data architecture, and streaming data. It includes a lab on modern data architectures for big data, along with a truck fleet event generator and big data architecture design.
Full Transcript
MODERN DATA ARCHITECTURES FOR BIG DATA II APACHE SPARK STREAM PROCESSING LAB II EVENT PROCESSING WITH SPARK STREAMING AGENDA Introduction to the lab Truck Fleet Lab II TIME TO TURN OSBDET ON! This is a lab! We'll use the course environment shortly: 1. INTRODUCTION TO THE LAB TRUCK FLEET EVENT GENERA...
MODERN DATA ARCHITECTURES FOR BIG DATA II APACHE SPARK STREAM PROCESSING LAB II EVENT PROCESSING WITH SPARK STREAMING AGENDA Introduction to the lab Truck Fleet Lab II TIME TO TURN OSBDET ON! This is a lab! We'll use the course environment shortly: 1. INTRODUCTION TO THE LAB TRUCK FLEET EVENT GENERATOR We're going to use a data generator producing sensor data. Sensor data → small data produced very quickly (velocity) This is considered streaming data, often times called events We have to rely on technologies dealing with this type of data at scale It produces two types of events, geo events & speed events: BIG DATA ARCHITECTURE This is the Big Data Architecture we're going to build: 2. TRUCK FLEET LAB II LAB KICK OFF Let's jump into OSBDET to kick the lab off: CONGRATS, WE'RE DONE!
