IoT Lecture 5: Data Access and Machine Learning

Data Access for IoT

• The main purpose of Data Access for IoT is to enable efficient and seamless data exchange between devices and applications in the Internet of Things (IoT) ecosystem.

Data Formats

• Text Format is considered the simplest format to use due to its human-readable nature and ease of implementation.
• In CSV Format, an escape character is used to distinguish between column separators and actual data values, ensuring accurate data parsing.
• The name 'CSV Format' might be considered misleading, as it stands for Comma Separated Values, but other separators like semicolons or tabs can also be used.
• The Python module 'csv' has limitations in handling CSV format, such as performance issues with large files and limited customization options.

Data Handling

• Data producers must share the same storage and produce data at very high throughput sometimes to ensure real-time processing and analysis of massive data volumes.
• The Python module 'numpy' supports csv format but can handle only numeric values.

Data Formats in Practice

• Excel files (xlsx) are one of the most popular formats for data and require the use of third-party packages like OpenPyXl or pandas in Python.
• JSON (JavaScript Object Notation) is the most commonly used data format in web and mobile applications due to its lightweight, human-readable, and language-independent nature.

Machine Learning

• Machine Learning (ML) is a subset of Artificial Intelligence (AI) that can learn and improve from experience by feeding more data to the algorithm.
• In supervised learning, the model learns the relationship between input data (X) and expected output (Y) from a given set of paired data.
• Unsupervised learning is typically used for data exploration, pattern discovery, and anomaly detection.
• The goal of regression in a given input-output pair (x, y) is to predict the continuous output value based on the input data.
• IoT devices generate a vast amount of data, which can help ML algorithms get more accurate results by providing more training data.