Understanding Data Science Applications

Study Notes

Data science is the study of data for meaningful business insights
It encompasses data gathering, analysis, and decision-making
Key goals: finding patterns in data, making predictions, and discovering hidden information
Data Science enables companies to make better decisions, perform predictive analysis, and discover patterns in their data

Data Science is used across various industries like banking, consultancy, healthcare, and manufacturing.
It’s valuable in route planning, forecasting delays, creating promotions, optimizing delivery timing, forecasting revenue, analyzing training benefits, and predicting election outcomes.

Data science in Finance helps improve decision-making, reduce risk, and increase efficiency.
Key Applications:
- Fraud detection and prevention
- Credit allocation
- Risk management and analysis
- Customer analytics and segmentation
- Pricing optimization

Python is the predominant programming language in finance.
It’s object-oriented and open-source, utilized by numerous corporations including Google.
Python imports financial data, such as stock quotes, using the Pandas framework.

Data wrangling cleans, structures, and transforms raw data into a usable format for analysis.
This includes handling missing or inconsistent data, formatting data types, and merging different datasets.

Discover: Identifying data sources, assessing data quality, and analyzing data structure and format.
Structure: Reshaping, handling missing values, and converting data types.
Clean: Removing or correcting inaccurate data, handling duplicates, and addressing anomalies impacting data reliability.
Enrich: Merging datasets, extracting relevant features, or incorporating external data sources.
Validate: Ensuring data quality and reliability.
Publish: Documenting data lineage, sharing metadata, and preparing data for storage or integration into data science tools.

A part of data wrangling that focuses on preparing data for analysis and modeling.
Key tasks:
- Scaling/Normalization: Adjusting numerical data values to a specific range.
- Encoding: Converting categorical data into numerical formats.
- Handling Missing Values: Deciding how to deal with missing data (like removal, filling with averages, or predicting missing data).
- Splitting Data: Dividing data into training and testing sets for machine learning.