23-24 - M2AI - DL4CV - 1 - Deep Learning 1-56.pdf

DL4CV Deep Learning for Computer Vision Master in Applied Artificial Intelligence...

DL4CV Deep Learning for Computer Vision Master in Applied Artificial Intelligence José Henrique Brito 1 Agenda Artificial Intelligence and Computer Vision Application Domains Artificial Intelligence and Computer Vision tasks Machine Learning and Deep Learning Neural Networks Neural Networks for Classification in Computer Vision Evaluation and Metrics Training Neural Networks Implementation challenges Neural Networks for other Computer Vision tasks More Neural Networks José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 2 2 1  Agenda Artificial Intelligence and Computer Vision Application Domains Artificial Intelligence and Computer Vision tasks Machine Learning and Deep Learning Neural Networks Neural Networks for Classification in Computer Vision Evaluation and Metrics Training Neural Networks Implementation challenges Neural Networks for other Computer Vision tasks More Neural Networks José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 3 3 AI and CV Application Domains Automotive - Self-driving cars, Driver Assistance Manufacturing - Industrial Inspection, Quality Assurance Security and Defense - Surveillance Agriculture - Crop monitoring, Precision Agriculture, Pest control Retail - Customer tracking, Theft detection, Automatic checkout Healthcare - Medical image analysis, Computer Aided Diagnosis Entertainment - Cinema and Digital Games https://www.forbes.com/sites/robtoews/2021/02/28/a-wave-of-billion-dollar-computer-vision-startups-is-coming/ José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 4 4 2  Automotive Tesla.com Advanced Driver Assistance Systems (ADAS) Autonomous Driving … Nauto.com José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 5 5 Manufacturing Automated visual inspection Automated manufacturing Autonomous vision guided vehicles … industryeurope.com Medium.com Seegrid.com José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 6 6 3  Security and Defense Hikvision Surveillance systems Traffic monitoring Access control Crowd monitoring, Violence Detection Facial Recognition Blacklist, VIP list, stranger detection Airport security, office buildings, hotels Automatic Speech Recognition Speaker Identification Terrorism Prevention Radicalization Prediction w/ Social Network Analysis Patrol Robots (UGVs) G-NIUS Guardium … José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 7 7 Agriculture Disease detection Precision agriculture Automated harvesting Drones/ satellite images … FlyGuys.com root-ai.com José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI FarmManagement.pro 8 8 4  Retail People Counting Theft detection Queue Detection Customer tracking, favorite areas Automated Checkout … Hikvision Amazon Go José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 9 9 Healthcare Computer Aided Diagnosis aidoc.com Precision Medicine Computational biochemistry/Drug discovery … Elix, Inc arterys.com José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 10 10 5  Media Hololens Cinema Visual Effects Digital Game Interaction Digital Content Generation … Dawn of the planet of the apes The Matrix José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 11 11 Agenda Artificial Intelligence and Computer Vision Application Domains Artificial Intelligence and Computer Vision tasks AI, ML, Deep Learning Computer Vision tasks Traditional Approach vs Deep Learning Approach Machine Learning and Deep Learning Neural Networks Neural Networks for Classification in Computer Vision Evaluation and Metrics Training Neural Networks Implementation challenges Neural Networks for other Computer Vision tasks More Neural Networks José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 12 12 6  Artificial Intelligence Natural Language Planning Machine Learning Processing Artificial Intelligence Fuzzy Logic Expert Systems … José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 13 13 Artificial Intelligence José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 14 14 7  Deep Learning Most major recent advances in AI are based on Deep Learning Deep Learning has had the biggest impacts on: Computer Vision Audio/Sound Processing Natural Language Processing (NLP) I will mostly cover Computer Vision José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 15 15 Image Acquisition Cameras have a human eye model https://www.researchgate.net/publication/318500799_Image_Acquisition_and_Techniques_to_Perform_Image_Acquisition/ José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 16 16 8  Image Acquisition f Pinhole camera c f = focal length c = center of the camera José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 17 Figure from Forsyth 17 Image Acquisition Camera Sensor http://www.photoaxe.com/a-guide-for-understanding-the-camera-sensor/ José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 18 18 9  Image Acquisition Bayer filter José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 19 19 Image Acquisition Three sensor cameras https://www.vision-doctor.com/en/area-scan-cameras/three-chip-colour-cameras.html José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 20 20 10  Computer Vision Tasks https://www.esri.com/about/newsroom/arcwatch/where-deep-learning-meets-gis/ José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 21 21 Image Classification José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 22 22 11  Object Detection José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 23 23 Semantic Segmentation José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 24 24 12  Instance Segmentation José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 25 25 Tracking José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 26 26 13  Many other CV tasks https://neurohive.io/en/news/mit-researchers-show-how-single-classifier-can-solve-multiple-computer-vision-tasks/ José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 27 27 Traditional Approach José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 28 28 14  Traditional Approach José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 29 29 Deep Learning Approach José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 30 30 15  Agenda Artificial Intelligence and Computer Vision Application Domains Artificial Intelligence and Computer Vision tasks Machine Learning and Deep Learning Supervised Learning Evaluation and Metrics Overview Features and Classifiers Neural Networks Neural Networks for Classification in Computer Vision Evaluation and Metrics Training Neural Networks Implementation challenges Neural Networks for other Computer Vision tasks More Neural Networks José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 31 31 Machine Learning https://www.javatpoint.com/subsets-of-ai José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 32 32 16  Machine Learning Slide: James Hays José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 33 33 Sci-kit Learn Cheat Sheet José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 34 34 17  Machine Learning Slide: James Hays José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 35 35 Machine Learning Slide: James Hays José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 36 36 18  Supervised Learning Training Training Labels Training Images Image Learned Training Features model Testing Image Features Learned model Prediction Test Image José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI Slide credit: D. Hoiem and L. Lazebnik 37 37 Evaluation Metrics Classification – binary (2 class) problems True Positives (TP) – correctly identified positives (class 1) instances True Negatives (TN) – correctly identified negatives (class 0) instances False Positives (FP) – incorrectly classified as positives (class 1) that are really negatives (class 0) False Negatives (FN) – incorrectly classified as negatives (class 0) that are really positives (class 1) José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 38 38 19  Evaluation Metrics Confusion Matrix https://blog.revolutionanalytics.com/2016/03/classification-models.html José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 39 39 Evaluation Metrics 𝑇𝑃 Precision = 𝑇𝑃+𝐹𝑃 𝑇𝑃 Recall = 𝑇𝑃+𝐹𝑁 𝑇𝑃+𝑇𝑁 Accuracy = 𝑇𝑃+𝑇𝑁+𝐹𝑃+𝐹𝑁 𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛∗𝑅𝑒𝑐𝑎𝑙𝑙 F1score = 2 ∗ 𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛+𝑅𝑒𝑐𝑎𝑙𝑙 https://en.wikipedia.org/wiki/Precision_and_recall José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 40 40 20  Features Raw pixels Histograms GIST descriptors SIFT descriptor … José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 41 Slide credit: L. Lazebnik 41 Classifiers Support Vector Machines (SVM) Naïve Bayes Bayesian networks Logistic regression Randomized Forests Boosted Decision Trees K-nearest neighbor (KNN) … Neural networks Deep Learning José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 42 42 21  Learning a classifier Given some set of features with corresponding labels, learn a function to predict the labels from the features x x x x x x o x x o o o o x2 Slide: James Hays José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI x1 43 43 Learning a classifier SVM – Calculate Max Margin Hyperplane x x x x x x o x x o o o o x2 Slide: James Hays José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI x1 44 44 22  Learning a classifier KNN – look at K neighbors and use their class x x x x x x ? x o x o o o o x2 Slide: James Hays José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI x1 45 45 ML Algorithms https://noeliagorod.com/2020/03/16/a-tour-of-machine-learning-algorithms/ José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 46 46 23  Deep Learning Vocabulary A.K.A. (or related to): Recurrent Neural Networks Deep Neural Networks RNN DNN LSTM Deep Structural Learning GRU Deep Belief Networks GAN Convolutional Neural Networks Transformers CNN Stable Diffusion ConvNets … Neural Networks “on Red Bull” Don’t be afraid of the buzzwords José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 47 47 Agenda Artificial Intelligence and Computer Vision Application Domains Artificial Intelligence and Computer Vision tasks Machine Learning and Deep Learning Neural Networks Neurons, Neural Networks, Deep Neural Networks Activations and Label Encoding Convolutional Neural Networks Neural Networks for Classification in Computer Vision Evaluation and Metrics Training Neural Networks Implementation challenges Neural Networks for other Computer Vision tasks More Neural Networks José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 49 49 24  Neurons Neuron = Linear function + (optional) non-linear activation José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 50 50 Neural Network https://www.bmc.com/blogs/deep-neural-network/ José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 51 51 25  Neural Network It’s actually a linear function in the form yi = Σ xj*wij + bi https://medium.com/analytics-vidhya/build-our-first-deep-learning-basic-model-using-keras-python-and-tensorflow-step-by-step-approach-d61c41b8a866 José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 52 52 Images and Neural Networks https://ml4a.github.io/ml4a/neural_networks/ José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 53 53 26  Deep Neural Network José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 54 54 Deeper Neural Network José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 55 55 27  Activations Activations for intermediate neurons José Henrique Brito | 2Ai - EST - IPCA, PT | DL4CV @ M2AI 56 56 28