Lecture1 - 2023.pdf

Duration: 13 Sessions Graduate Program Fall 2023 HW/SW Co-Design Real-Time AI Instructor: Dr. Amin Safaei Fall 2023 Xilinx 2021 Department: Computer Science Lesson overview • This course discusses following subjects: • Basics understanding of machine learning, deep learning and computer visio...

Duration: 13 Sessions Graduate Program Fall 2023 HW/SW Co-Design Real-Time AI Instructor: Dr. Amin Safaei Fall 2023 Xilinx 2021 Department: Computer Science Lesson overview • This course discusses following subjects: • Basics understanding of machine learning, deep learning and computer vision • Ability to understand the principles hardware/software co-design • The Jupyter Notebook framework deployed in the Xilinx Pynq context • Hardware/software codesign based on Xilinx Pynq solution • Basic knowledge of computer vision algorithms and OpenCV • Basic Knowledge of Deep learning paradigm including CNNs and RNNs models (with a focus on the real-time inference) • Understanding of FPGA overlay design and Xilinx deep learning and computer vision APIs Course Resources • Primary text • Slides and instructor study materials. • There is no required textbook for this course. Office Hours: • Instructor: Monday and Wednesday at 12:30 pm–1:30 pm (by email appointment), [email protected] • TA: Take-Home Assignments • There will be three assignments this term. • Late assignment delivery will be penalized by a 5% deduction per working day and will not be accepted after three days. Software environment • Jupyter Notebook • PYNQ (Python) Evaluations • • • • • Assignment: (10+10+10) Mid-term exam (Closed book) Weekly Quizzes Final exam (Closed book) Project 30% 20% 10% 30% 10% What’s in it for you ? 1. Course Introduction 2. Introduction to AI big bang, and basics of machine learning and deep learning 3. Real-time edge intelligence and FPGA-based SoCs 4. Overview of PYNQ [VITIS] platform 5. Basics of Hardware/Software co-design and FPGA-based SoCs. 6. CJupyter Arraysnotebooks (Notebook) 6. Getting started with 6. 7. C Arrays Pointers 7. Overlay design methodology and PYNQ overlay design C Pointers 8. Basics of computer7.vision and OpenCV What’s in it for you ? 9. Basics of deep learning inference (Convolutional and Recurrent Neural Networks) 10. Quantized Neural Network (QNN) on PYNQ [VITIS] 11. Binarized Neural Network (BNN) on PYNQ [VITIS] 12. Quantized LSTM on PYNQ [VITIS] 8. C Characters and Strings Duration: 180 min Graduate Program Fall 2023 Course Introduction Instructor: Dr. Amin Safaei Fall 2023 Xilinx 2021 Department: Computer Science 1.1 Defining the Problem Jon Cory, ML Live Detroit 1.2 Inference Compute and Memory • Across a Spectrum of Neural Networks Quenton Hall, Accelerating AI Camera Development with Xilinx VITIS Michaela Blott, Hotchips 2018 1.2 Inference Compute and Memory • Across a Spectrum of Neural Networks Quenton Hall, Accelerating AI Camera Development with Xilinx VITIS Michaela Blott, Hotchips 2018 1.2 Inference Compute and Memory • Industry Trend • Performance Scaling: Application optimized architecture • Cloud and edge application • Deep Learning Xilinx 2021 • AI is Evolving Rapidly • GoogleNet → ResNet • How to adapt to your workloads ? Adaptable Hardware Xilinx 2021 Xilinx 2021 1.2 Inference Compute and Memory GoogLeNet’s architecture https://www.quora.com/What-is-GoogLeNet-s-architecture ResNet’s architecture https://d2l.ai/chapter_convolutional-modern/resnet.html 1.2 Inference Compute and Memory • Lowest Latency AI Inference Xilinx 2021 1.2 Inference Compute and Memory • AI Compute Compared to CPUs and GPUs Quenton Hall, Accelerating AI Camera Development with Xilinx VITIS 1.2 Inference Compute and Memory • ZYNQ 7000 • ZYNQ UltraSCALE Xilinx 2021 Xilinx 2021 1.2 Inference Compute and Memory • ZYNQ Applications Xilinx 2021 1.2 Inference Compute and Memory • How to use the advantages of hardware (FPGA/ACAP) without hardware background? • PYNQ • VITIS • Make Zynq so easy-to-use that programmers can access the benefits without learning advanced digital design skills 1.3 Vitis AI Xilinx 2021 1.4 PYNQ • PYNQ is a Framework Xilinx 2021 Python Productivity for Zynq 1.4 PYNQ • Why Python? • Python is increasingly the Language of Choice Xilinx 2021 Python Productivity for Zynq 1.4 PYNQ • Why Python? • Ecosystem advantage: there’s a Python library for that… Xilinx 2021 Python Productivity for Zynq 1.4 PYNQ • Why Python? • Jupyter Notebooks … the engine of data science Xilinx 2021 Python Productivity for Zynq 1.4 PYNQ • Why Python? • JupyterLab: web-based IDE incl. Notebooks Xilinx 2021 Python Productivity for Zynq 1.4 PYNQ • PYNQ provides Linux Drivers for PS-PL Interfaces wrapped in Python Libraries Xilinx 2021 Python Productivity for Zynq 1.4 PYNQ Xilinx 2021 Python Productivity for Zynq 1.4 PYNQ • Ubuntu-based Linux versus embedded Linux Xilinx 2021 Python Productivity for Zynq 1.4 PYNQ • PYNQ’s Ubuntu-based Linux Xilinx 2021 Python Productivity for Zynq 1.4 PYNQ • PYNQ enabled boards Xilinx 2021 Python Productivity for Zynq 1.4 PYNQ • PYNQ-Z2 Board Xilinx 2021 Python Productivity for Zynq Next Section • • • • • Basics of machine learning and deep learning Deployment Model to implementation Team Members Setup the Boards