Natural Language Processing (NLP)

Chapter 6: Natural Language Processing

• Natural Language Processing (NLP) is a field of algorithms focused on processing unstructured data.
• NLP is used to process large amounts of unstructured text data in various formats like word documents, PDFs, emails, and web pages.
• Organizations often have extensive corpora of unstructured text.

Needs for Text Processing - NLP

• Advancements in technology have led organizations to rely on large volumes of text data, such as legal agreements, court orders, and documents within legal firms.
• NLP translates valuable textual assets into actionable knowledge using intelligent machines.
• NLP for big data leverages text from numerous sources to identify relationships and trends.

Types of NLP

• NLP approaches are categorized into two types:
  • Supervised NLP
  • Unsupervised NLP

Types of NLP - Supervised

• This approach employs supervised learning algorithms, like Naive Bayes and Random Forests
• Models in this process are trained based on predicted output given for a set of inputs.
• Supervised models are not self learning.
• Models are fine-tuned based on the provided target output.

Types of NLP - Unsupervised

• Unsupervised learning algorithms do not rely on a target output for model training.
• Models deduce insights from input data through multiple iterations, fine-tuning parameters, and improving results.
• Recurrent Neural Networks (RNNs) are a common unsupervised learning technique used in NLP.

Topics

• NLP basics
• Text preprocessing
• Feature extraction
• NLP techniques
• Sentiment analysis

Natural Language Processing basics - Definition

• NLP combines processes, algorithms, and tools to interpret text data for actionable knowledge from human language inputs
• NLP aims to interpret unstructured data.

Natural Language Processing basics

• NLP organizes unstructured text data using advanced methods to solve diverse problems
• Common problems solved include sentiment analysis, document classification, and text summarization.

Natural Language Processing Hierarchy

• This topic shows a diagram outlining the hierarchical stages involved in NLP processing.
  • It includes components like Text Preprocessing, Feature Extraction, Supervised Learning, Unsupervised Learning, Model Training, Model Verification, Model Deployment, and Model APIs followed by subsequent stages involving Web Applications and Analytics Engines.

Steps involved in NLP - Type of machine learning

• NLP can be either supervised or unsupervised.
• Supervised techniques utilize labeled data, while unsupervised models make predictions without labeled data.
• The steps for preprocessing and feature extraction are the same for both types of NLP.

Steps involved in NLP - Text Preprocessing

• Raw text data is not usable in NLP, so preprocessing is essential.
• Techniques include removing stop words, replacing capital words, and eliminating special characters.
• Part of speech tagging (annotation) and normalisation (stemming and lemmatization) are also used.

Steps involved in NLP - Feature Extraction

• Any ML algorithm requiring text input needs numerical data.
• The process transforms text into a numerical form, often using vectors.

Steps involved in NLP - Model Training

• Model training establishes a mathematical function that predicts outcomes based on the given input.
• Iterations and parameter tuning are vital components of this process.

Steps involved in NLP - Model Verification

• Verifying models ensures accuracy.
• Data is typically split into 80% training and 20% validation sets for verification.

Model deployment and APIs

• Verified models are deployed for real-world applications to predict outcomes in the context.
• They are saved to storage locations for usage in various applications ranging from in-memory applications to Hadoop batch processes and web applications.
• Commonly used methods such as pickle files are often used to store the models used in production.
• APIs are used to handle requests from different applications, enabling access to the models.

Text preprocessing - preprocessing

• Data cleaning and preparation for meaning analysis and classification are core tasks in preprocessing.
• Removing noise, such as HTML tags and irrelevant words, leading to a better overall semantic context for analysis, is essential.

Text preprocessing steps

• Reading the corpus, tokenization, stop-word removal are some of the stages in text preprocessing.
• Stemming, lemmatization, and conversion into numerical format are other steps.

Corpus

• A corpus comprises a complete collection of text documents that need to be processed and analysed.
• A collection of emails is an example of a corpus.

Tokenize

• Tokenization involves breaking down a sentence into individual words and removing unnecessary symbols such as punctuation from the sentence and also normalizing the input by converting it into lowercase if needed.

Text preprocessing

• Lowercasing the text, removing numbers, removing punctuation, and removing spaces are basic text preprocessing techniques.

Removing stop words

• Removal of common words found in sentences that may not have much analytical importance from sentences.
• Example words are "the", "a," "is" and other words.
• These words do not significantly impact and have little contextual importance.
• Removal of these words is part of preprocessing for better model accuracy.

Bag of Words- BOW

• Converting text data into numerical format suitable for machine learning is the key concept in Bag of Words (BOW).
• BOW does not recognize the ordering of words but focuses only on the occurrence.

Bag of Words, example

• Demonstrates the creation of a vocabulary from documents and transforming each document into a numerical vector.
• Demonstrates an example using the numerical representation from document in a fixed size.
• Provides a basic numerical representation for each document based on the frequency of the words in the document, this representation makes the sentence easier to work with as a tool in NLP problems.

BOW limitations

• BOW does not take sentence order or semantic meanings into consideration
• The method primarily focuses on the presence or absence of words but does not consider the order or intent.

Count Vectorizer

• A crucial element to understand the Count Vectorizer method
• Explains how the Count Vectorizer differs from Bag of Words by considering the frequency of each word.

Stemming

• Techniques (Porter, Snowball, Lancaster) for reducing words to their root form.

Stemming - Porter

• Explains the Porter Stemming algorithm and its associated rules for simplifying words to their base form.

Stemming - Snowball

• Explains the Snowball Stemming algorithm and its comparison to other stemming algorithms in terms of accuracy.

Stemming - Lancaster stemming

• Explains how the Lancaster algorithm works as it is faster than other algorithms like Porter and Snowball.
• Shows that the algorithm will considerably reduce the working data set of words.
• Explains the algorithm and its approach on normalization.

Lemmatization

• Finds the dictionary form of the words from a word and simplifies the words.

N-grams

• Methods to identify continuous sequences in sentences or text.
• N-grams can be unigrams, bigrams, or trigrams.

Uni-Gram, Bi-Gram, and Tri-Gram example

• Provides examples of unigrams, bigrams, and trigrams extracted from a sample sentence in a table format.

Feature extraction

• Converting text data into numerical features for use in machine learning algorithms.
• Explains different methods used in the process.

TF-IDF

• A weighting method that considers both term frequency (how often a word appears in a document) and inverse document frequency (how often a word appears across all documents in a corpus).
• Used to identify more important words in a document relative to the corpus.

TF-IDF mathematical formula

• Shows the mathematical formulas for calculating term frequency (TF) and inverse document frequency (IDF).

TF-IDF - example

• Shows an example using a table to explain how TF-IDF is calculated.

TF-IDF example- calculation TF

• Shows a sample example of using a table to extract the TF values.

TF-IDF - example, Step 2 Find TF for all docs

• This step shows calculation of the TF of each words in the document using the given formula. Shows the table calculated from document 1 to 3.

TF-IDF , Step 3 Find IDF

• This step shows calculation of the IDF of the words in the documents using the given formula and excel formula for calculation.

TF-IDF , Step 4: Build model: stack all words next to each other

• This step is how we stack the words next to each other based on values from the earlier steps.

TF-IDF , Step 5: Compare results and use table to ask questions

• How to compare the TF-IDF results and asking questions example.

Example, continue- Analysis and outcomes

• This section outlines how to analyze results from the previous table to draw conclusions about the similarities and differences among documents concerning the use case.

Applying NLP techniques

• Describes the general sequence of applying NLP techniques to address various NLP problems. Shows that text classification is one example.

Applying NLP techniques: Text classification

• Text classification is a common NLP use case.
• Examples include email spam detection, retail product hierarchy identification, and sentiment analysis.
• The process involves categorizing or classifying text into meaningful groups.

Applying NLP techniques: Text classification

• Describes how text classification requires computing resources.
• Shows the use cases when processing a large amount of text data (like legal documents on the internet) that a distributed computing framework is suitable for this purpose.

Applying NLP techniques: Text Classification

• Shows the diagram for the text classification process.

Thank you

• Presents the conclusion.

References

• Lists the references used in the presentation.