Intro to AI: Large Scale Data & Data Mining

• Introduction to Artificial Intelligence, Machine Learning, and Intelligent Agents
• Intro to AI and Data Science NGN 112 – Fall 2024, College of Engineering, American University of Sharjah.
• Prepared by Dr. Salam Dhou, CSE. Last Updated on: 22nd of August 2024

Large Scale Data

• There has been enormous data growth in both commercial and scientific databases due to advances in data generation and collection technologies
• Gather whatever data you can whenever and wherever possible
• Gathered data will have value either for the purpose collected or for a purpose not envisioned.

Data Mining Commercial Viewpoint

• Large amounts of data are being collected and warehoused
• Companies like Meta and Amazon use this data
• Meta has billions of active users
• Amazon handles millions of visits/day
• Computers have become cheaper and more powerful
• Competitive pressure exists to provide customized services

Data Mining Scientific Viewpoint

• Data collected and stored at enormous speeds
• NASA EOSDIS archives over petabytes of earth science data per year
• High-throughput biological data is collected
• Scientific simulations now generate terabytes of data in a few hours
• Data mining helps scientists with automated analysis of massive datasets and hypothesis formation

Artificial Intelligence

• Artificial Intelligence is intelligence demonstrated by machines
• AI is a wide-ranging branch of computer science focused on building smart machines that can perform tasks that typically require human intelligence

Importance of AI

• AI has uses from boosting vaccine development to automating detection of potential fraud
• AI companies raised $66.8 billion in funding in 2022, according to CB Insights research*, more than doubling the amount raised in 2020
• AI is making waves in a variety of industries due to its fast paced adaptation
• AI has positive impact on different sectors, such as: better medicine, safer banking, innovative media, agriculture, and renewable energy

AI in Medicine

• Integrating AI into the healthcare industry could lead to benefits such as more informed health policy and improvements in diagnosing patients
• AI improving medical diagnosis
• AI speeding up drug discovery
• AI transforming patient experience
• AI performing robotic surgery

AI Improving Medical Diagnosis

• Roughly 400,000 hospitalized patients suffer preventable harm annually
• There are roughly 100,000 deaths are year
• Incomplete medical histories and large caseloads can lead to deadly human errors
• AI can predict and diagnose disease at a faster rate
• Many healthcare companies use AI to reduce errors and save lives

AI Speeding Up Drug Discovery

• The drug development industry is affected by high development cost and lengthy research times
• Putting each drug through clinical trials costs an estimated average of $1.3 billion
• Only 10 percent of drugs are successfully brought to market
• Biopharmaceutical companies are leveraging the efficiency, accuracy and knowledge AI can provide
• Companies like to use AI to develop the next generation of medicines

AI Transforming Patient Experience

• Time is money in the healthcare industry
• Providing seamless patient experience allows hospitals, clinics and physicians to treat more patients on a daily basis
• Hospitals whose patients have positive experiences have higher profits

Machine Learning

• Machine learning is an application and subset of AI that provides systems the ability to automatically learn and improve from experience without being explicitly programmed
• Traditional programming involves a series of directions to transform input data into desired output, often using IF-THEN structures
• Machine learning enables machines to learn and take actions based on past observations
• Machine Learning is useful for tasks that are too complex for humans to code directly/explicitly

AI vs Machine Learning

• While AI and machine learning are often used interchangeably, they are different concepts
• AI is the broader concept of machines making decisions, learning new skills, and solving problems
• Machine learning is a subset of AI that enables intelligent systems to autonomously learn new data

Common types of Machine Learning

• Supervised Learning
• Unsupervised Learning
• Reinforcement Learning

Supervised Learning

• Supervised learning models make predictions based on labeled/annotated training data and the most common and popular approach to machine learning
• These models need manually labeled sample data to learn from.
• For example, detecting spam needs correctly identified samples of what is and is not spam
• There are 2 types of Supervised Learning; Classification and Regression

Unsupervised Learning

• Unsupervised learning algorithms uncover insights and relationships in unlabeled data
• The models must find patterns on their own as they are not trained with the "right answer"

Clustering

• Clustering involves grouping samples based on similarity
• Similarity can be how close the points are to each other in a Euclidean space
• Different similarity metrics can be used

Clustering Applications

• Custom profiling for targeted marketing
• Group related documents for browsing
• Group genes and proteins that have similar functionality
• Group stocks with similar price fluctuations
• Used to reduce the size of large data sets

Association Rule Mining

• Used to predict occurrence of an item based on occurrences of other items
• Algorithms extract association rules from datasets

Association Rule Mining Applications

• Market-basket analysis; shops place associated items next to each others so customers can see and buy
• Medicine; used in the medical domain to find combination of patient symptoms and test results associated with certain diseases

Reinforcement Learning

• Is concerned with how a software agent ought to act in a situation to maximize the reward
• Models attempt to determine the best possible path they should take in a given situation through trial and error
• Since there is no training data, machines learn from their own mistakes and choose the actions that lead to the best solution or maximum reward
• Is used in robotics and gaming

Deep Learning

• Deep learning is a subset of machine learning based on Artificial Neural Networks (ANN)
• Deep learning algorithms are built with multiple layers of interconnected neurons, allowing multiple systems to work together simultaneously, and step-by-step
• Deep learning models can be supervised, semi-supervised or unsupervised
• Advanced machine learning algorithms are used by tech giants, like Google, Microsoft, and Amazon; they are used in self driving cars and smart assistants

Deep Learning vs Machine Learning

• Feature extraction is automatically integrated within some layers of the model itself in deep learning algorithms, in contrast, classical machine learning algorithms require feature extraction to be done separately

Feature Extraction

• Feature extraction transforms raw data into numerical features so they can be processed by machine learning algorithms
• Extracted features should describe the samples to help distinguish the classes

Semi-Supervised Learning

• Uses a small amount of labeled data and a larger set of unlabeled data
• Semi-supervised learning model makes effective use of all the available data
• Can require the use of or inspiration from unsupervised methods; examples include clustering and density estimation
• Once patterns are discovered, supervised methods or ideas from supervised learning may be used to label the unlabeled examples
• Becoming popular and is more cost-effective plus faster to set up

Intelligent Agents

• Individual program or entity that interacts with its environment by perceiving via sensors and acting through actuators
• Attributes associated with having a mental state. For example; belief, desire, and intentions
• Intelligent Agents operate within a cycle of perceiving, thinking, and acting

Agent Definitions

• Sensors: devices that detect the change in the environment and then transmit the information to the agent
• Actuators: components of machines that convert energy into motion
• Only responsible for moving and controlling a system.
• Effectors: devices which affect the environment

Agent Examples

• Human: utilize natural attributes such as eyes, ears, hands, legs, and natural intelligence
• Robotic: utilize camera, servo motors, and artificial intelligence
• Software: utilize file contents, keystrokes as sensory input and artificial intelligence

Examples Of AI Intelligent Agents

• Self-driving cars: analyze the environment using cameras and then apply actuators like steering to respond. The road, pedestrians, & roadsigns are the environment
• Cleaning robots: use sensors like cameras and dirt detectors, use actuators like wheels and brushes. The environment is a room that needs cleaning
• Part-picking/Bin-picking Robot: uses sensors like cameras to detect the position of parts. It then uses jointed arms to move the parts and organize appropriately