Lab #3.1 - Apache Spark Stream Processing - Truck Fleet Lab.pdf

MODERN DATA ARCHITECTURES FOR BIG DATA II APACHE SPARK STREAM PROCESSING LAB FIRST SPARK STREAMING APPLICATION AGENDA Introduction to the lab Truck Fleet Lab TIME TO TURN OSBDET ON! This is a lab! We'll use the course environment shortly: 1. INTRODUCTION TO THE LAB TRUCK FLEET EVENT GENERATOR We're...

MODERN DATA ARCHITECTURES FOR BIG DATA II APACHE SPARK STREAM PROCESSING LAB FIRST SPARK STREAMING APPLICATION AGENDA Introduction to the lab Truck Fleet Lab TIME TO TURN OSBDET ON! This is a lab! We'll use the course environment shortly: 1. INTRODUCTION TO THE LAB TRUCK FLEET EVENT GENERATOR We're going to use a data generator producing sensor data. Sensor data → small data produced very quickly (velocity) This is considered streaming data, often times called events We have to rely on technologies dealing with this type of data at scale It produces two types of events, geo events & speed events: SIMPLIFIED BIG DATA ARCHITECTURE This is the Big Data Architecture we're going to build: 2. TRUCK FLEET LAB LAB KICK OFF Let's jump into OSBDET to kick the lab off: CONGRATS, WE'RE DONE!