Web and Text Analytics Week 5

Web and Text Analytics 2024-25, Week 5

• The course covers web and text analytics.
• The presenter's name and website are provided.
• The information systems lab website and the university are also referenced.
• The material focuses on supervised machine learning, sentiment analysis, and feature extraction.

Supervised Machine Learning

• Supervised machine learning uses input (X) to predict output (Y^).
• The prediction is compared to the true value (Y).
• The difference (cost) is used to update parameters (θ).
• Features (X) are inputs to the system which then uses a prediction function to produce an Output (Y). This output is then compared to the actual value of the Output (Y) to calculate the error which is then used to update the parameters of the system
• Parameters are variables that control the prediction function.

Sentiment Analysis

• Sentiment analysis classifies text (e.g., tweets) as positive or negative (1 or 0) using an algorithm.
• A tweet like "I am happy because I am learning NLP" is classified as positive (1).
• Logistic regression is one classification algorithm used for sentiment analysis.

Supervised ML & Sentiment Analysis

• Representing text as features is the first step in sentiment analysis.
• Logistic regression is an example of a classifier.
• Sentiment is classified as either 1 (positive) or 0 (negative).

Vocabulary & Feature Extraction

• A vocabulary is a collection of words.
• A tweet can be represented as a vector based on the vocabulary.
• Vocabulary size (V) determines the vector dimension.
• The vocabulary is generated from all tweets in a corpus.

Feature Extraction

• In feature extraction, a 1 is placed in the index corresponding to a word in the tweet; otherwise, 0 is used.
• Tweets are encoded as vectors representing words.
• The vocabulary is created using all tweets within a corpus; this vocabulary might consist of potentially millions of words.

Sparse Vectors

• As the vocabulary size (V) increases, the feature vectors become sparser.
• The training and prediction time of the system increase as the number of features increase.

Feature Extraction with Frequencies

• Given a corpus of positive and negative tweets, a vocabulary is constructed.
• The frequency of each word in each sentiment class is recorded.

Positive and Negative Tweets

• The corpus contains positive and negative tweets.
• Positive tweets are specifically identified.
• Negative tweets are specifically identified.

Feature Extraction with Frequencies (Implementation)

• To encode each tweet, a dictionary mapping words to their frequency in each sentiment class (positive or negative) is created.
• The frequency of a word within each class determines the features for a given tweet.

Positive Counts

• Positive count dictionary is determined by positive tweets.

Negative Counts

• Negative count dictionary is determined by negative tweets.

Frequency Dictionary

• Words like "happy" and "sad" are characteristically associated with positive and negative sentiments respectively — their frequencies are significantly greater in either the positive or negative sets.
• Words like "I" and "am" are frequently present in both positive and negative tweets, and are accordingly less characteristic of either sentiment.

Feature Extraction

• Based on the frequency dictionary, features for each tweet are calculated.

Preprocessing: Stop Words and Punctuation

• Stop words and punctuation are removed from tweets before calculating feature vectors.
• Stop words (e.g., "and", "is") and punctuation (e.g., ".", "!") are eliminated.

Preprocessing: Handles and URLs

• Handles (@mentions) and URLs are removed from tweets.

Preprocessing: Stemming and Lowercasing

• Words are stemmed (reduced to their root form) and converted to lowercase.
• Stemming converts "tuning" to "tun".
• Lowercasing converts "GREAT" to "great".

Preprocessing

• The initial tweet is processed to generate a list of tokens.
• The tweet is preprocessed according to the mentioned steps before feature extraction and the associated values are calculated.

Preprocessing and Feature Extraction (Initial Case)

• Preprocessing steps are applied to the initial tweet.
• Preprocessed words and associated frequency values are stored.

Features Extraction

• Features are extracted for each tweet.
• Features are listed as a vector with a bias term.

Logistic Regression

• The sigmoid function maps 'z' to a probability value (between 0 and 1).
• The implementation details of logistic regression are explained in Part 1 of the notebook provided.

Create the Feature Matrix (X)

• A matrix (X) is constructed from tweet features.
• The extract_features() function is used to obtain features from tweets.

Implement the extract_features Function

• The extract_features function processes single tweets to extract feature vectors.

Create the feature Matrix (x) and train the model

• A feature matrix X is generated from the training set.
• The LogisticRegression model is trained from the data in the feature matrix X with corresponding Y label data.

Check Performance Using the Test Set

• The model's performance is evaluated using a held-out test set.
• Accuracy, precision, recall, and F1-score are calculated.