Chapter 1. Big Data and Artificial Intelligence Systems PDF

Chapter 1. Big Data and Artificial Intelligence Systems Inroduction to AI Dr. Feras Al-Obeidat Human Brain The human brain is one of the most sophisticated machines in the universe. It has evolved for thousands of years to its current state. As a result of...

Chapter 1. Big Data and Artificial Intelligence Systems Inroduction to AI Dr. Feras Al-Obeidat Human Brain The human brain is one of the most sophisticated machines in the universe. It has evolved for thousands of years to its current state. As a result of continuous evolution, we are able to make sense of nature's inherent processes and understand cause and effect relationships. Humans brain are able to learn from nature and devise similar machines and mechanisms to constantly evolve and improve our lives. For example, the video cameras we use derived from the understanding of the human eye. Human brain & informtion processing Fundamentally, human intelligence works on the paradigm of sense, store, process, and act. Through the sensory organs, we gather information about our surroundings, store the information (memory), process the information to form our beliefs/patterns/links, and use the information to act based on the situational context and stimulus. Currently, we are at a very interesting juncture of evolution where the human race has found a way to store information in an electronic format. We are also trying to devise machines that imitate the human brain to be able to sense, store, and process information to make meaningful decisions and complement human abilities. Convergence of human intelligence and machine intelligence This chapter will set the context for the convergence of human intelligence and machine intelligence at the start of a data revolution. We have the ability to consume and process volumes of data that were never possible before. We will understand how our quality of life is the result of our decisive power and actions and how it translates to the machine world. We will understand the paradigm of Big Data along with its core attributes before diving into artificial intelligence (AI) and its basic fundamentals. Overview of the Big Data frameworks and how those can be leveraged for building intelligence into machines. The chapter will end with some of the exciting applications of Big Data and AI. Topics in the chapter Results pyramid Comparing the human and the electronic brain Overview of Big Data  Results pyramid The quality of human life is a factor of all the decisions we make: the results we get (positive, negative, good, or bad) are a result of our actions, our actions are a result of the beliefs we hold, and the beliefs we hold are a result of our experiences. This is represented as a results pyramid as follows Results Pyramid Theory At the core of the results pyramid theory is the fact that it is certain that we cannot achieve better or different results with the same actions. E.g. If the team continues to have same beliefs, which translate to similar actions, the company cannot see noticeable changes in its outcomes. Similarly, at the core of computing evolution, cannot evolve to be more effective and useful with the same outcomes (actions), models (beliefs), and data (experiences) that we have access to traditionally. We can evolve for the better if human intelligence and machine power start complementing each other. What the Human Brain does best While the machines are catching up fast for intelligence, nothing can come close to some of the capabilities that the human brain has. Sensory input The human brain has an incredible capability using all the senses in parallel. We can see, hear, touch, taste, and smell at the same time, and process the input in real time. In terms of computer terminology, these are various data sources that stream information, and the brain has the capacity to process the data and convert it into information and knowledge. What the Human Brain does best Storage The information collected from the sensory organs is stored consciously and subconsciously. The brain is very efficient at filtering out the information that is non-critical for survival. Storage capacity is similar to terabytes in computers. The brain's information retrieval mechanism is also highly sophisticated and efficient. The brain can retrieve relevant and related information based on context. Brain stores information in the form of linked lists, where the objects are linked to each other by a relationship, which is one of the reasons for the availability of data as information and knowledge, to be used as and when required. What the human brain does best Processing power The human brain can read sensory input, use previously stored information, and make decisions within a fraction of a millisecond. This is possible due to a network of neurons and their interconnections. The human brain possesses about 100 billion neurons with one quadrillion connections. It coordinates hundreds of thousands of the body's internal and external processes in response to contextual information. one quadrillion : 1,000,000,000,000,000 What the human brain does best Low energy consumption The human brain requires less energy for sensing, storing, and processing information. The power requirement in calories (or watts) is insignificant compared to the equivalent power requirements for electronic machines. With growing amounts of data, along with the increasing requirement of processing power for artificial machines, we need to consider modeling energy utilization on the human brain. The computational model needs to fundamentally change towards quantum computing and eventually to bio-computing. What the human brain does best Low energy consumption The computational model needs to fundamentally change towards quantum computing and eventually to bio-computing. Biocomputing uses molecular biology parts as the hardware to implement computational devices Quantum computing is an area of computing focused on developing computer technology based on the principles of quantum theory (which explains the behavior of energy and material on the atomic and subatomic levels). What the electronic brain does best As the processing power increases with computers, the electronic brain—or computers—are much better when compared to the human brain in some aspects: Speed information storage The electronic brain (computers) can read and store high volumes of information at massive speeds. Storage capacity is exponentially increasing. The information is easily replicated and transmitted from one place to another. Processing: The more information we have, the more accurate our predictions will be, and the machines will be much more intelligent. Information storage speed is consistent across machines when all factors are constant. in the case of the human brain, storage and processing capacities vary based on individuals.  What the electronic brain does best Processing by brute force The electronic brain can process information using brute force. A distributed computing system can scan/sort/calculate and run various types of compute on very large volumes of data within milliseconds. The human brain cannot match the brute force of computers. Computers are very easy to network and collaborate with in order to increase collective storage and processing power. The collective storage can collaborate in real time to produce intended outcomes. While human brains can collaborate, they cannot match the electronic brain in this aspect. Best of both worlds AI is finding and taking advantage of the best of both worlds in order to augment human capabilities. The sophistication and efficiency of the human brain and the brute force of computers combined together can result in intelligent machines that can solve most challenging problems faced by human beings. AI will complement human capabilities and will be a step closer to social inclusion by facilitating collective intelligence. Examples include epidemic predictions, disease prevention based on DNA sampling and analysis, self driving cars, robots that work in hazardous conditions, and machine assistants for differently able people. The availability of Big Data has accelerated the growth and evolution of AI and machine learning applications. AI & Big Data The primary Goal of AI To implement human-like intelligence in machines and to create systems that: Gather data, process Create models (hypothesis), Predict or influence outcomes, and * Ultimately improve human life.  The primary Goal of AI With Big Data at the core of the pyramid, we have the availability of massive datasets from heterogeneous sources in real time. This promises to be a great foundation for an AI that really augments human existence  Data is defined as facts and statistics collected together for reference or analysis. Storage mechanisms have greatly evolved with human evolution: handwritten texts, punch cards, magnetic tapes, Big Data hard drives, floppy disks, CDs, DVDs, human DNA, and more. With the advent of the internet and the Internet of Things (IoT), data volumes have been growing exponentially. Why we call it big data ? Big Data Characteristics – 4 Vs Volume: ever increasing and exponentially growing amount of data. We are now collecting data through more and more interfaces between man-made and natural objects. For example, a patient's routine visit to a clinic now generates electronic data in megabytes. Velocity: This represents the amount of data generated with respect to time and a need to analyze that data in near-real time for some mission critical operations. There are sensors that collect data from natural phenomenon, and the data is then processed to predict hurricanes/earthquakes. Healthcare is a great example of the velocity of the data generation; analysis and action is mission critical Big Data Characteristics Variety: variety in data formats. Historically, most electronic datasets were structured and fit into database tables (columns and rows). However, more than 80% of the electronic data are now unstructured format, for example, images, video files, and voice data files. With Big Data, we are in a position to analyze the vast majority of structured/unstructured and semi- structured datasets. Value: This is the most important aspect of Big Data. The data is only as valuable as its utilization in the generation of actionable insight. Remember the results pyramid where actions lead to results. Data quality, comprehensive data holds the key to actionable insight; systems need to evolve quickly to be able to analyze the data, understand the patterns within the data, and, provide solutions that ultimately create value. Big Data Characteristics Evolution from dumb to intelligent machines The basic elements of a computer are the CPU (Central Processing Unit), the RAM (temporary memory), and the disk (persistent storage). One of the core components of a CPU is an ALU (Arithmetic and Logic Unit). This is the component that is capable of performing the basic steps of mathematical calculations along with logical operations. Traditional computers evolved with greater and higher processing power. However, they were still dumb machines without any inherent intelligence. Extremely good at following predefined instructions. ?! Only answer specific questions they were meant to solve. process lots of data and perform computationally heavy jobs, they would be always limited to what they were programmed to do This limitation of traditional computers to respond to unknown or non-programmed situations leads to the question: Can a machine be developed to think and evolve as humans do History of AI In the year 1956, the term artificial intelligence was raised. The last decade of the 20th century marked remarkable advancements in AI techniques. In 1990, there were significant demonstrations of machine learning algorithms supported by case-based reasoning and natural language understanding and translations. Machine intelligence reached a major milestone when then World Chess Champion, Gary Kasparov, was beaten by Deep Blue in 1997. AI systems have greatly evolved to the extent that some experts have predicted that AI will beat humans at everything eventually. One of the greatest revolutions in human history. Intelligence Human intelligence, is a constantly evolving phenomenon. It evolves through four Ps when applied to sensory input or data assets: Perceive, Process, Persist, and Perform To develop artificial intelligence, we need to also model our machines with the same cyclical approach Types of intelligence Here are some of the broad categories of human intelligence: Linguistic intelligence: Ability to associate words to objects and use language (vocabulary and grammar) to express meaning Logical intelligence: Ability to calculate, quantify, and perform mathematical operations Interpersonal and emotional intelligence: Ability to interact with other human beings and understand feelings and emotions  Intelligence tasks classification Basic tasks: Intermediate Expert tasks: Perception tasks: Mathematics Financial analysis Common sense Games Engineering Reasoning Scientific analysis Natural language processing Medical analysis Intelligence tasks classification For human intelligence, basic tasks are easy to master and they are hardwired at birth. However, for machine intelligence, perception, reasoning, and natural language processing are some of the most computationally challenging and complex tasks. Big data frameworks In order to derive value from data that is high in volume, varies in its form and structure, and is generated with ever increasing velocity, There are two primary categories of framework that have emerged over a period of time: Batch processing Real-time processing Big data frameworks Batch processing: Data is collected from various sources in the staging areas and loaded and transformed with defined frequencies and schedules. In most use cases there is no critical need to process the data in real time or in near real time. As an example, the monthly report on a student's attendance: Apache Hadoop.  Big data frameworks Real-time processing: Processing the data and generating actionable insight as soon as the data is available. For example, in a credit card fraud detection system, the alert should be generated as soon as the first instance of logged malicious activity. Apache Spark: This is a distributed execution engine that relies on in-memory processing based on fault tolerant data abstractions named RDDs (Resilient Distributed Datasets). Intelligent Applications with Big Data We need to use an algorithmic approach with the massive data and computers. Leveraging a combination of human intelligence, large volumes of data, and distributed computing power, we can create expert systems which can be used as an advantage to lead the human race to a better future. Areas of AI While we are in the infancy of developments in AI, here are some of the basic areas in which significant research and breakthroughs are happening: Natural language processing: Facilitates interactions between computers and human languages. Fuzzy logic systems: These are based on the degrees of truth instead of programming for all situations with IF/ELSE logic. These systems can control machines and consumer products based on acceptable reasoning. Intelligent robotics: These are mechanical devices that can perform repetitive tasks. Expert systems: These are systems or applications that solve complex problems in a specific domain. They are capable of advising, diagnosing, and predicting results based on the knowledge base and models. Review Q: What is a results pyramid? A:. Review Q: What is a results pyramid? A: The results we get (man or machine) are an outcome of our experiences (data), beliefs (models), and actions. If we need to change the results: (better) sets of data, models, and actions.  Q and A Q: How is this paradigm applicable to AI and Big Data? A  Q and A Q: How is this paradigm applicable to AI and Big Data? A: In order to improve our lives, we need intelligent systems. With the advent of Big Data, there has been a boost to the theory of machine learning and AI due to the availability of huge volumes of data and increasing processing power. We are getting better results for humanity as a result of the convergence of machine intelligence and Big Data. Review Q: What are the basic categories of Big Data frameworks? A: Review Q: What are the basic categories of Big Data frameworks? A: Based on the differentials between the event time and processing time, there are two types of framework: Batch processing and Real-time processing. Review Q: What is the goal of AI? Review Q: What is the goal of AI? A: The fundamental goal of AI is to augment and complement human life.  not to replace human Review Q: What is the difference between machine learning and AI? A: Review Q: What is the difference between machine learning and AI? A: Machine learning is a core concept which is integral to AI. In ML, the models are trained based on data and the models can predict outcomes for the new datasets. AI systems try to emulate human cognitive abilities and are context sensitive. Depending on the context, AI systems can change their behaviors and outcomes to best suit the decisions and actions the human brain would take