Data Science and AI Overview

Data Science

• Data science is an interdisciplinary field.
• It uses scientific methods, processes, algorithms, and systems.
• It extracts knowledge and insights from structured and unstructured data.
• It's related to data mining, deep learning, and big data.
• Data science aims to unify statistics, data analysis, machine learning, and domain knowledge.
• This allows understanding and analyzing phenomena using data.

Artificial Intelligence, Machine Learning, and Deep Learning

• Artificial intelligence (AI) is machine intelligence.
• It's unlike natural intelligence in humans and animals.
• Examples include computer chess (1994), chatbots (2017 Google), and 2048 games.
• AI encompasses machine learning (ML).
• ML automatically learns and improves from experience.
• It doesn't need explicit programming.
• ML focuses on creating computer programs to access and learn from data.
• Deep learning (DL) is an AI function that mimics the human brain.
• It processes data, creating patterns for decision-making.
• Neurons are fundamental to deep learning.
• The concept of biomimicry is tied to deep learning
• An example of biomimicry is the Aero plane based on Birds.
• Perception is a concept within deep learning.
• An example related to perception is 'going to the movie'.

Machine Learning

• Machine learning is an application of AI.
• It allows systems to learn and improve from experience.
• It doesn't need explicit programming.
• The process begins with observation or data, and uses example-driven instruction to identify patterns.
• It aims to enable computers to learn automatically, without human interaction or assistance.

Deep Learning

• Deep learning is an AI function that imitates the human brain.
• It works by processing data and generating patterns for decision-making.
• Deep learning is equivalent to basic human thought processes.
• Neuroscience is implied by the role of neurons in this process.
• Biomimicry is a concept related to deep learning

How to Learn Data Science

• Fundamental elements (FE) include data wrangling and Exploratory Data Analysis (EDA).
• Data analysis involves techniques, like classification, regression and reinforcement learning among others.
• Data visualization utilizes tools like Tableau, Power BI, Matplotlib, ggplot, and Seaborn.
• Programming languages are essential; Python, R, and Java are examples
• Web scripting tools such as Beautiful Soup, Scrapy and URLLIB are important.
• Mathematical concepts such as statistics, linear algebra, and differential calculus are crucial.

Machine Learning (continued)

• Topics like what machine learning is, types of machine learning, applications, and algorithms should be covered.

Steps of Machine Learning

• Data preprocessing
• Modeling
• Algorithm refinement
• Data sorting
• Training
• Testing

Classifications of Machine Learning

• Supervised learning
• Unsupervised learning
• Reinforcement learning

Supervised Machine Learning

• Learns from labeled examples to predict future events.
• Algorithms produce an inferred function.
• The system provides targets corresponding to input.
• Learning algorithms compare output with expected values, adjusting the model accordingly.

Decision Tree

• Non-parametric supervised learning algorithm for classification and regression.
• Hierarchical tree structure with root node, branches, internal nodes, and leaf nodes.
• Attributes are selected to split data based on a measure like entropy or Gini impurity.

K-Means

• Clustering method in vector quantization.
• Partitions observations into clusters.
• Each observation is assigned to the cluster with the nearest mean.
• The method attempts to minimize variance within clusters.

Naive Bayes Algorithm

• Algorithm that uses Bayes' theorem to classify objects.
• Assumes independence between data attributes, resulting in a 'naive' approach.
• Essential formula for the algorithm is p(c) / x) = p(x/c) * p(c)

Applications of Naive Bayes Algorithm

• Predicting in real-time
• Applicable to multi-class prediction
• Used in text classification, spam filtering and sentiment analysis

Where is Naive Bayes Used?

• Simple and fast technique for classifying large datasets.
• Suitable for predicting probabilities of different classes, using various attributes.

How Naive Bayes Algorithm Works

• Steps:
  • Calculate prior probabilities
  • Determine likelihood probabilities per class.
  • Implement Bayes' formula to calculate posterior probability.
  • Identify higher probability class based on input data.

Linear Regression

• Method for predicting variable values from other variables.
• Estimates coefficients of a linear relationship.
• Aims to minimize discrepancies between predicted and actual values.
• Uses a "least squares" method to find the best fit line.

Real-life Examples of Linear Regression

• Predicting property prices from features
• Estimating impact of SAT/GRE scores
• Gauging sales based on input data
• Short-term weather forecasting
• Used in finance and investment

Python Modules

• Modules are python files containing code.
• Python has built-in modules.
• Python code must be written in python files.

Natural Language Processing (NLP)

• Uses computational methods to understand and model human language.
• Focuses on the properties of written language for understanding and generating the language.
• Important part of AI and machine learning.
• Key component for human-computer dialogue systems and machine translation.
• Main components of NLP include:
  • Investigating properties of written language
  • Application involving language processing.
  • Automating construction/adaptive machine dictionaries
  • Modeling human desires/beliefs

Human-Computer Dialogue Systems

• Focuses on communication between humans and computers in conversational style.

Machine Translation

• Aims at translating text or speech from one language to another.
• Facilitates communication for different languages.
• Allows accessing foreign language information.