AI for Creative Industries - ITCT101 Lecture - Mahidol University - PDF

AI for Creative Industries ITCT101 Computer Technologies Module2: AI, ML, and Data Science Kanrawi Kitkhachonkunlaphat ([email protected]) The goal of this section Introduce Introduce current Apply generative Introduce current multimedia and use of AI for the AI to cre...

AI for Creative Industries ITCT101 Computer Technologies Module2: AI, ML, and Data Science Kanrawi Kitkhachonkunlaphat ([email protected]) The goal of this section Introduce Introduce current Apply generative Introduce current multimedia and use of AI for the AI to create media issue with creative content creative industry generative AI What is Multimedia? Content that uses a combination of different forms of media to convey information or provide entertainment. What is Media? Media (the plural of Medium) Various means of communication that are used to deliver information, entertainment, and other forms of content to a wide audience. Components of Multimedia  Components of Multimedia Text Audio Images Video Interactive Media Written content Sound elements Static visual Moving visual Elements that E.g. Articles, E.g. Podcasts, representations media E.g. Movies, require user captions, subtitles, music tracks, E.g. Photos, video clips, interaction to descriptions. voiceovers. diagrams, icons, tutorials, function E.g. infographics. animations. Websites, video games, virtual reality experiences, interactive presentations. Current use of AI in the creative industry  Supervised Learning The model is trained on a labeled dataset The goal is for the model to learn the mapping from inputs to outputs so that it can accurately predict the labels for new, unseen data. Unsupervised Learning Training a model on data that does not have labeled Machine The model tries to learn the underlying structure or patterns in the data without any explicit instructions Learning in Deep Learning Focuses on neural networks with many layers to model complex patterns in Multimedia large datasets Generative Models The models that aim to generate new data points that are similar to the existing data They are all AI ! Enabling them to create new samples that resemble the original data Multimodal Learning Integrating and processing information from multiple types of data (modalities) simultaneously 3D rendering Rendering with AI Upscaling Video rendering AI for 3D Animation Speech Animation Synchronizing lip movements and facial expressions with speech, often used in character animation and virtual avatars. Automated facial animation from audio. Speech is analyzed and translated into virtual muscle activations within vocal tract simulation. Image processing with AI Use AI to enhance image or video Generative AI  Supervised Learning The model is trained on a labeled dataset The goal is for the model to learn the mapping from inputs to outputs so that it can accurately predict the labels for new, unseen data. Unsupervised Learning Training a model on data that does not have labeled Machine The model tries to learn the underlying structure or patterns in the data without any explicit instructions Learning in Deep Learning Focuses on neural networks with many layers to model complex patterns in Multimedia large datasets Generative Models The models that aim to generate new data points that are similar to the existing data They are all AI ! Enabling them to create new samples that resemble the original data Multimodal Learning Integrating and processing information from multiple types of data (modalities) simultaneously  AI that apply generative model and can generate new data points that are similar to the existing data AI generated story AI generated image AI generated audio Generative AI generated video AI AI generated 3D model Generative AI Artist have a design in Artist create the artwork mind and create prompt to with the assist of ask AI to generate content. generative AI. Editing techniques might Use AI as a software tool. be applied to enhance the result. AI tools assisting artwork creation AI generated content with image editing AI generated content with image editing AI generated content with image editing Video editing using AI generated video, audio, and voice over Video editing using AI generated video, audio, and voice over  Technical side: How generative AI work?  LEARNING PROCESS ORIGINAL AI FIXED ADD NOISE Content by REALTIFY_AI  IMAGE GENERATION DIFFUSION MODEL Image 1 Image 2 Image 3 Image 4 Output B Input A Input A Output B Image 2 Image 1 Image 3 Image 2 Slightly less Noisy Image Noisy Image Image 4 Image 3  FIX FORWARD DIFFUSION PROCESS GENERATIVE REVERSE DENOISE PROCESS  IMAGE GENERATION Image 1 Input A + Prompts ( cat ) Output B  IMAGE GENERATION Image 1 Image 2 Input A + Prompts ( cat ) Output B  IMAGE GENERATION Image 1 Image 2 Input A + Prompts ( cat ) Output B  IMAGE GENERATION Image 1 Image 2 Input A + Prompts ( cat ) Output B  IMAGE GENERATION Image 1 Image 2 TYPICAL STEPS FOR DIFFUSION MODEL Prompt engineering Prompt engineering Practice of designing inputs for AI tools that will produce optimal outputs  How to write a prompt to create the expected result?  First step Before writing a prompt Let’s have the clear image of the target picture in your head first!  1. Type of expected image E.g. Realistic, Semi-Realistic, Illustrator, etc. Good prompt 2. Subject, action, object structure 3. Light and camera setting  PROMPTS Only 2. Subject, action, object 1 GIRL,WORKING LOOK, BLACK SUIT, FORMAL SUIT, HOLDING A RABBIT DOLL SIMPLE BACKGROUND, WHITE BACKGROUND,  PROMPTS 1 GIRL,WORKING LOOK, BLACK SUIT, FORMAL SUIT, HOLDING A RABBIT DOLL garden, morning light, rim light, view from above Add 3. Light and camera setting  PROMPTS 1 GIRL, 25 YO,LONG HAIR, BROWN HAIR,BROWN EYE,WORKING LOOK, BLACK SUIT, FORMAL SUIT, SMILE, HOLDING A RABBIT DOLL garden, flowers, morning light, rim light, view from above, look at viewer, Olympus em5,25 mm f 1.4,bokeh, Add details of 2. Subject, action, object 3. Light and camera setting Practice time! Dreamina account registration Goto https://dreamina.capcut.com/ai-tool/login Sign up to create an account. Follow the steps to finish account creation. 2. 1. Dreamina Interface You will have up to 450 free credits per day. Each generation cost different amount of credits. Try your first prompt Download your result Select the variation that you satisfy with. Download the resulting image.  Practice 1: AI Image generation Generate image of your favorite pet inside your dreamed home Do not forget to follow good prompt structure 1. Type of expected image E.g. Realistic, Semi-Realistic, Illustrator 2. Subject, action, object 3. Light and camera setting Submit your work to the practice1 submission on ICT elearning.  Practice 2: AI Image generation Generate an image for a product advertisement Requirements 1. Select one type of product to sell 2. Need to have a presenter and the presenter should look like yourself. 3. The presenter must hold the product in his/her hand(s). Submit your work to the practice2 submission on ICT elearning. Current common flaw in AI generated image Text presentation Current common flaw in AI generated image Number and representation of fingers, legs, tails  Some generative AI can detect objects inside an image AI Inpainting They might allow you to make change of parts of an image Download source image Select the provided image from the section Resource for Practice3 in ICT Elearning. Download the selected image Go to Canvas in Dreamina and Upload the selected image Upload the selected image onto the canvas Enter image editing mode Select the image on the canvas to edit the image Select the inpainting tool 1. 3. 2. Apply inpainting Use brush to select the 1. area that you would like to edit. Write a prompt to edit the selected part. 2. 3. 4. Apply inpainting (Cont.) 3. Select the variation that you need or do further editing. Confirm the result by tapping the Done button after you satisfy with the result Download your result 1. 2.  Practice 3: AI Image Inpainting 1. Select one out of three provided image on ICT eLearning 2. Use AI inpainting to decorate the selected image in Valentine Theme Submit your work to the practice2 submission on ICT elearning. Generative AI for Audio Audio generation Can use text prompt to generate lyric Text prompt to generate melody Select from the provided singer sound and let the singer sing Limitation Limited length and style for free version Generative AI for Video Text to Video Prompt “A pope cat with a cross wand in his hand standing on two legs and praying in front of his followers in a church decorated with decorative color glass windows” Limitation Limited length of the video Limited movements MU Software download Free for Mahidol University students and staff Adobe creative Cloud with MU license Free for MU students and staff Can be installed on personal machine Need to use MU Wi-Fi to download Software download While on MU Wi-fi (not MUIC Wifi) Goto https://muit.mahidol.ac.th/ Software download & Manual > Software Download Login using Mahidol account Select applications Download Adobe creative cloud Installation of MU license version is not the same as the Installation manual Software download commercial version. Please follow the installation manual.  Download photoshop In Adobe creative cloud, you can download multiple software including Photoshop. Generative AI and Copyright Issue Copyright protects original works of authorship Copyright A legal concept that grants the creator of an original work exclusive rights to its use and distribution, typically for a limited time. Intended to protect the creator's intellectual property and provide them with control over how their work is used Grant for a tangible medium of expression Copyright does not protect facts, ideas, systems, or methods of operation Cover intellectual works including literary, dramatic, musical, and artistic works, such as poetry, novels, movies, songs, computer software, and architecture. Control the right to: Make copies Distribute Prepare Perform the Display the copies derivatives work publicly work publicly based on the original work (Edit the work) Public domain (PD) Creative works that are not protected by copyright They are free for anyone to use without seeking permission from the original creator or paying any royalties. Anyone can freely copy, modify, distribute, or used for any purpose. Types Expiration of copyright Works created by the government Voluntary release, such as the CC0 (No Rights Reserved) license No copyright protection, such as ideas and facts Fair Use A legal doctrine that allows limited use of copyrighted material without requiring permission from the rights holders. A particular use qualifies as fair use Transformative Use If the use adds new expression, meaning, or message to the original work, making it transformative. E.g. a parody that comments on or critiques the original work, news reporting Non-commercial uses E.g., educational, research, or personal use Terms of Use Or Terms and Conditions or Terms of Service Legal agreements between a service provider and the user that outline the rules, responsibilities, and restrictions related to the use of a service, product, website, or application These terms set the expectations for both the service provider and the user, specifying what is allowed and what is not Usually cover usage rights, restrictions, intellectual property, user responsibilities, privacy, etc. Copyright and generative AI Is generative AI stealing from creators? As of April 2024 several law suits have been brought against AI image and text generation platforms that have used visual and text content created or owned by others as training material. These law suits claim that the use of artists’ or writers' content, without permissions, to train generative AI is an infringement of copyright. While these cases are ongoing, we have no definitive answer on whether https://blogs.gwu.edu/law-eti/ai-litigation-database/ the training of AI models is considered an infringement of copyright. Copyright and generative AI Generated artworks might infringe the copyright Depending on the data set that the generative AI used to train the model Recheck both original data set and the terms of use before use the generated images in commercial  Last Updated: Aug 12, 2024 Copyright and generative AI Material created by generative AI tools do not currently receive copyright protections in US. If the terms of use do not address further uses, then you are free to use the material. Currently, copyright protection is not granted to works created by Artificial Intelligence. Copyright and generative AI The U.S. Copyright Office has issued guidance “While AI-assisted inventions are not categorically unpatentable, the inventorship analysis should focus on human contributions, as patents function to incentivize and reward human ingenuity.” https://copyright.gov/ai/ai_policy_guidance.pdf