Untitled

Common AI Workloads

• Features of common AI workloads are identifiable.

Content Moderation and Personalization Workloads

• Detects and filters harmful or inappropriate content like images, text and video.
• Tailors content and recommendations based on user preferences and behavior.
• Common services include Azure Content Moderator, Azure Personalizer.

Computer Vision Workloads

• Analyzes and extracts information from visual content such as images and videos.
• Identifies objects, faces, text, and scenes.
• Tracks objects and motion.
• Common services include Azure Computer Vision, Azure Video Analyzer.

Natural Language Processing (NLP) Workloads

• Understands and processes human language.
• Extracts meaning, sentiment, entities, and relationships from text.
• Translates languages.
• Generates text summaries, answers questions, and powers chatbots.
• Common services include Azure Text Analytics, Azure Translator, and Azure Language Understanding (LUIS).

Knowledge Mining Workloads

• Extracts knowledge from unstructured text sources like documents, emails, and web pages.
• Discovers patterns, relationships, and insights.
• Builds knowledge graphs and ontologies.
• Common services include Azure Cognitive Search and Azure Machine Learning.

Document Intelligence Workloads

• Analyzes and processes structured and semi-structured documents like invoices, forms, and contracts.
• Extracts key information such as text, tables, and fields
• Automates document processing tasks.
• Common services include Azure Form Recognizer, Azure Document Translation.

Generative AI Workloads

• Creates new content in the form of text, images, audio, and video, based on learned patterns from existing data.
• Generates realistic and creative outputs.
• Common services include Azure OpenAI Service which is powered by OpenAI's models.

Guiding Principles for Responsible AI

• Fairness, reliability and safety, privacy and security, inclusiveness, transparency, and accountability are guiding principles for responsible AI.

Fairness Considerations

• Identify and mitigate potential biases in training data and algorithms. Ensure the solution doesn't unfairly advantage or disadvantage any particular group.

Reliability and Safety Considerations

• Data used for training and testing should be accurate, complete, and relevant, and poor data leads to unreliable results.
• Test the solution for errors and unexpected input to ensure failures can be handled gracefully while risks are minimized.
• Continuously monitor the AI solution for performance and potential negative impacts. Protocols should be in place to address issues.

Privacy and Security Considerations

• Implement robust security measures to protect user data from unauthorized access, theft, or misuse and ensure compliance with data privacy regulations.
• Inform users about what data is collected, how it is used, and their rights concerning their data.

Inclusiveness Considerations

• Give users control over their data and how it is used in the AI solution and allow users to opt-in/out and access/correct their data.
• Ensure the solution is accessible to people with diverse abilities and backgrounds and consider language, cultural differences, and accessibility needs.
• Include diverse perspectives in the design and development process to avoid or reinforce existing biases or exclude certain groups.
• Ensure the AI solution benefits everyone equally and does not disproportionately harm any particular group.

Transparency Considerations

• Explain how the AI solution works and reaches its decisions to help users understand the reasoning behind outputs and build trust.
• Reveal the algorithms used in the solution, or at least offer high-level explanations of their function.
• Clearly communicate the capabilities and limitations of the AI solution to users and avoid overhyping or making misleading claims.

Accountability Considerations

• Clearly identify who is responsible for the development, deployment, and use of the AI solution to ensure accountability for potential harms.
• Regularly audit the AI solution for ethical lapses and compliance with regulations. Mechanisms for reporting and addressing concerns is required.
• Have clear protocols for addressing harmful outcomes or unintended consequences of the AI solution, including compensation or fixing errors.

Machine Learning Techniques

Machine Learning Scenarios

• Machine learning involves training algorithms to identify patterns and make predictions from data.
• Choosing the right type of algorithm depends on the kind of problem you're trying to solve.

Regression Machine Learning

• You want to predict a continuous value based on other features.
• Predict future temperatures, estimate house prices, and predict stock market trends.
• Models: Linear regression, polynomial regression, support vector regression, random forest regression.

Classification Machine Learning

• You want to predict a discrete category based on other features.
• Models classify emails as spam/not spam, identify fraudulent transactions, and recognize handwritten digits.
• Models: Logistic regression, K-nearest neighbors, decision trees, support vector machines, random forests.

Clustering Machine Learning

• You want to group unlabeled data points based on their similarities.
• Examples include identifying customer segments, find anomalies in sensor data, and grouping genes with similar functions.
• Models: K-means clustering, hierarchical clustering, density-based spatial clustering (DBSCAN).

Features of Deep Learning Techniques

• Deep learning is a specific type of machine learning that uses artificial neural networks inspired by the structure and function of the brain.
• Deep learning models can learn complex, non-linear relationships between features, unlike simpler models like linear regression; this is non-linearity.
• Deep neural networks can automatically extract relevant features from raw data, reducing the need for manual feature engineering; this is high feature extraction.
• They handle data with a large number of features effectively; this is high dimensionality.
• They learn internal representations of the data that are more informative than the raw features; this is representation learning.
• They can be trained on large datasets, leading to better performance with more data; this is scalability. Choose regression for predicting continuous values, classification for discrete categories, and clustering for grouping unlabeled data.

Core Machine Learning Concepts

Features and Labels

• Features are the individual input variables that represent the characteristics or attributes of the data points in a dataset. They are used by machine learning models to make predictions or decisions.
• Labels, are the known outcomes or values that correspond to each data point. They are used to train supervised machine learning models which learn to map features to labels.
• i.e For images of cats and dogs, the features might include pixel values, color information, and texture details. The labels would be "cat" or "dog," indicating its correct classification.

Training and Validation Datasets:

• The training dataset is used to train a machine learning model, where the model identifies patterns and relationships between features and labels.
• The validation dataset is used to evaluate the model's performance on unseen data and to fine-tune its hyperparameters. Testing with unseen data ensures that the model is not overfitting to training data.

Azure Machine Learning Capabilities

Automated Machine Learning (AutoML)

• Automates various tasks in the machine learning process
• Cleans, transforms, and features engineering of data automatically.
• Model training: Selects and trains various models and save you time and effort.
• Compares and ranks trained models based on their performance, helping you choose the best one.
• Hyperparameter tuning: Optimizes model hyperparameters for better accuracy and efficiency.
• Explainability: Generates insights into why a model makes certain predictions, improving transparency.

Data and Compute Services

• Azure offers a range of services for various data science and machine learning needs,

Data Storage

• Data can be stored in different formats (structured, unstructured) with services like Azure Blob Storage, Azure Data Lake Storage, and Azure SQL Database.

Data Management

• Azure Synapse Analytics and Azure Data Factory provide tools for data ingestion, cleansing, transformation, and orchestration.

Compute

• Access powerful VMs, container instances, and serverless options like Azure Machine Learning compute or Azure Functions for training and deploying models.

Model Management and Deployment in Azure Machine Learning

• Azure Machine Learning simplifies the model lifecycle management.
• Stores and tracks different versions of trained models through model registration.
• Prepares models for deployment in various environments (cloud, on-premises, edge devices) through model packaging.
• Deploys models to Azure web services, containers, or other platforms for real-world use via model deployment.
• Tracks model performance metrics and ensure continuous improvement through monitoring and evaluation.
• Easily switches between model versions and roll back if needed through versioning and rollback.
• Benefits include faster time to insights, lower costs, improved model performance, increased productivity, and scalability and flexibility.

Computer Vision Workloads

• Build and deploy efficient, data-driven solutions across various industries and applications

Common Types of Computer Vision Solutions

Image Classification

• Model performance measures how accurately the model classifies images into predefined categories with metrics including accuracy, precision, recall, and F1 score.
• Speed and latency measures how quickly the process classifies an image
• Scalability and adaptability measures the ability to handle large datasets and adapt to new categories or changes in existing ones.
• Customization refers to whether the model can be tailored to specific requirements and datasets for improved performance.
• Interpretability refers to the ability to understand the model's reasoning behind its classification.

Object Detection

• Multi-object detection is the ability to identify and localize multiple objects within an image.
• Bounding box accuracy defines how precisely the model can delineate the location and size of each detected object.
• Background suppression refers to effectively differentiating objects from the background for accurate detection.
• Class-specific features refers to recognizing specific features and attributes of objects based on their category.
• Real-time tracking is the ability to track identified objects continuously across video frames.

Optical Character Recognition (OCR)

• Accuracy and precision recognizes characters (including special symbols) with minimal errors.
• Font and style recognition adapts to different font types, sizes, and styles.
• Language support recognizes characters in multiple languages.
• Complex document handling recognizes text within images of documents with layouts, tables, and other formatting elements.
• Integration with other systems allows seamless integration with OCR results into downstream workflows.

Facial Detection and Facial Analysis

• Robustness accurately detects faces under various lighting conditions, poses, and occlusions.
• Multi-face detection identifies and locate multiple faces within an image.
• Facial landmark detection recognizes key points on the face for further analysis.
• Emotion recognition infers emotions from facial expressions.
• Biometric identification matches faces to known individuals for identification or verification.

Azure Tools and Services for Computer Vision Tasks

Azure AI Vision

• Identifies objects, scenes, and actions in images, along with emotions and facial expressions. Extracts text from images using optical character recognition (OCR) through Image Analysis.
• Categorizes images into pre-defined categories or custom categories through Image Classification.
• Generates descriptive tags for images based on their content through Tagging.
• Train custom classification models with Custom Vision.
• Locates and identifies objects within images and provides bounding boxes around them through Object Detection.

Azure AI Face Detection

• Face Detection locates faces with high accuracy, even in challenging conditions.
• Facial Recognition identifies known individuals based on a pre-trained face database.
• Facial Analysis extracts emotional attributes, age, gender, and other facial features.
• Person Identification tracks identified individuals across multiple images and videos.
• Large Gallery Search identifies known individuals in a large gallery of faces with millions of entries.

Azure AI Video Indexer

• Automatic Speech Recognition (ASR) transcribes spoken audio in videos to text.
• Optical Character Recognition (OCR) extracts text from images within videos.
• Scene Detection identifies and segments different scenes within a video
• Topic Modeling analyzes video content and identifies key topics discussed.
• Sentiment Analysis understands the sentiment of spoken dialogue and overall video content.
• Speaker Identification recognizes and distinguishes different speakers based on voices.
• Content Moderation detects potentially inappropriate content.

Natural Language Processing (NLP) Workloads on Azure

• Identify features of common NLP Workload Scenarios

Key Phrase Extraction

• Identifies the most important and relevant concepts, keywords, and topics within a text.
• Uses statistical methods, linguistic rules, and machine learning algorithms for analysis.
• Can be customized to target specific domains or tasks.
• Common uses are summarization of documents and news articles, information extraction and retrieval from text, clustering and tagging documents based on content, keyword research for marketing and SEO.

Entity Recognition

• Identifies and classifies named entities like people, organizations, locations, dates, and quantities within text.
• Employs gazetteers, dictionaries, and machine learning models for classification.
• Can be extended to identify custom entity types relevant to specific domains.
• Common uses are question answering systems and conversational bots, information extraction and knowledge base construction, financial analysis and risk assessment, sentiment analysis based on entity context.

Sentiment Analysis

• Analyzes the emotional tone and opinion expressed in text data.
• Classifies sentiment as positive, negative, neutral, or mixed.
• Utilizes lexical analysis, machine learning, and rule-based approaches.
• Can be fine-tuned to analyze specific emotions.
• Common uses are customer feedback analysis and social media monitoring, market research and product development, political opinion analysis and risk assessment, personalized recommendations and targeted advertising.

Language Modeling

• Predicts the word in a sequence based on the preceding context.
• Enables generation of coherent and grammatically correct text.
• Employs statistical models like n-grams or neural networks for prediction.
• Can be trained on various text formats and genres.
• Common uses are machine translation and text summarization, dialogue systems and chatbot development, automated writing and content creation, text prediction and autocompletion in software applications.

Speech Recognition

• Converts spoken language into text format.
• Utilizes acoustic models and language models for accurate transcription.
• Adapts to different accents and speaking styles.
• Common uses are voice assistants and smart home devices, dictation and voice-to-text applications, automated call centers and speech analytics.

Speech Synthesis

• Converts text into spoken language.
• Generates natural-sounding audio with different voices and emotions.
• Common uses are text-to-speech applications for accessibility and convenience, interactive audiobooks and voiceovers, chatbots and virtual assistants with more engaging interactions.

Translation

• Converts text from one language to another while preserving meaning.
• Utilizes statistical machine translation and neural networks for accuracy.
• Adapts to different domains and contexts.
• Website and document localization for global audiences is a common use.
• Real-time communication and interpretation tools is a common use.
• Content creation and dissemination across languages is a common use.
• Breaking down language barriers in information access and communication is a common use.

Azure Tools and Services for NLP Workloads

Azure AI Language Service

• Text Analytics analyzes text for sentiment, key phrases, entities, language and extracts information and gain insights from unstructured text data.
• Text Mining uncovers hidden relationships and patterns, and performs topic modeling, clustering, and anomaly detection.
• Question Answering builds chatbots and virtual assistants.
• Optical Character Recognition (OCR) extracts text from images and documents.
• Custom Vision trains image classification models with custom datasets to recognize objects, scenes, or actions.
• Machine Translation translates text between languages with high-quality output.

Azure AI Speech Service

• Speech-to-Text converts spoken audio to text with real-time or batch processing to transcribe audio recordings, podcasts, and conversations are ideally suited.
• Text-to-Speech synthesizes human-like speech to generate audio for videos, or e-learning modules.
• Speaker Recognition identifies individual speakers in audio recordings.
• Sentiment Analysis analyzes the emotion of spoken language.
• Custom Speech to train speech recognition models to recognize specialized vocabulary.

Azure AI Translator Service

• Machine Translation translates text and speech with high accuracy and fluency.
• Custom Translator trains translation models based on specific terminology.
• Speech Translation translates conversations in real-time, enabling multilingual communication and collaboration.
• Document Translation translates entire documents while preserving formatting and layout.
• Neural Machine Translation (NMT) utilizes the latest NMT technology for state-of-the-art translation quality, delivering natural-sounding and accurate results.

Generative AI Workloads on Azure

Generative AI

• Creates new content, data, or code.

Features of Generative AI Models

• Can generate novel content that was not explicitly taught in the training data, mimicking human creativity; this is creativity.
• The same model can produce different outputs from the same input; this is variability.
• They can generate large amounts of content quickly and efficiently; this is scalability.
• Can be readily fine-tuned and customized; this is adaptability.

Common Scenarios

• Text Generation creates dialogue for chatbots, writing marketing copy, and creative fiction.
• Image and Video Synthesis creates images and videos from text descriptions, editing existing visuals, and generating deepfakes.
• Music Composition generates new music in different styles, producing sound effects, and remixing existing songs.
• Software and Code Generation to automatically write code for specific tasks and generating test data for software development.

Responsible considerations

• Generative models can inherit biases from training data, which can lead to discriminatory output; this requires careful data selection and mitigation strategies.
• Difficult to understand how models reach their outputs making it challenging to address bias or accuracy issues; addressed with transparency and explainability.
• Generative AI can create and spread false information with robust detection and verification mechanisms; misinformation and disinformation.
• Questions arise about who owns the copyright requiring clear legal frameworks and ethical consideration; ownership and copyright.

Key Features of Azure OpenAI

• Text Generation – Create diverse text formats like emails, letters, blog posts, articles, product descriptions, etc.
• Conversational AI – Build chatbots and virtual assistants that engage in natural, fluid conversations.
• Translation – Translate text between languages with fluency and accuracy.
• Content Creation - Generate content for marketing, education, and entertainment.
• Code Generation Capabilities generate functions, methods, classes, or even complete programs in various programming languages.
• Code Generation can translate natural language to code. Convert descriptions of desired code functionality into actual code.
• Code Generation can write unit tests: Automate test generation for code quality assurance.
• Code Generation assists with code completion by suggesting code completions and autocomplete options.
• Code Generation can refactor code improving readability and maintainability.
• Image Generation Capabilities creates images from text descriptions, edits and manipulates existing images, combines text and images, and/or generate images for various purposes.