Cyber Bullying Detection on Social Media Using Machine Learning PDF

ITM Web of Conferences 40, 03038 (2021) https://doi.org/10.1051/itmconf/20214003038 ICACC-2021 Cyber Bullying Detection on Social Media using Machine Learning Aditya Desai1, Shashank Kalaskar2, Omkar Kumbhar3, and Rashmi...

ITM Web of Conferences 40, 03038 (2021) https://doi.org/10.1051/itmconf/20214003038 ICACC-2021 Cyber Bullying Detection on Social Media using Machine Learning Aditya Desai1, Shashank Kalaskar2, Omkar Kumbhar3, and Rashmi Dhumal4 1,2,3 Student, Department of Computer Engineering, Ramrao Adik Institute of Technology, Nerul 4 Assistant Professor, Department of Computer Engineering, Ramrao Adik Institute of Technology, Nerul Abstract. Usage of internet and social media backgrounds tends in the use of sending, receiving and posting of negative, harmful, false or mean content about another individual which thus means Cyberbullying. Bullying over social media also works the same as threatening, calumny, and chastising the individual. Cyberbullying has led to a severe increase in mental health problems, especially among the young generation. It has resulted in lower self-esteem, increased suicidal ideation. Unless some measure against cyberbullying is taken, self-esteem and mental health issues will affect an entire generation of young adults. Many of the traditional machine learning models have been implemented in the past for the automatic detection of cyberbullying on social media. But these models have not considered all the necessary features that can be used to identify or classify a statement or post as bullying. In this paper, we proposed a model based on various features that should be considered while detecting cyberbullying and implement a few features with the help of a bidirectional deep learning model called BERT. As the social lifestyle exceeds the physical barrier of human interaction and contains unregulated Keywords: Cyberbullying, Social Media, BERT, contact with strangers, it is necessary to analyze and NLP, Semi-supervised learning , Twitter API. study the context of cyberbullying. Cyberbullying makes the victim feel that he is being attacked everywhere as the internet is just a click away. It can have mental, physical, and emotional effects on the victim. Cyberbullying mainly takes place in the form of text or 1 Introduction images on social media. If bullying text can be Millions of young people spend their time on social distinguished from non-bullying text, then a system can networking, and the sharing of information is online. act accordingly. An efficient cyberbullying detection Social networks have the ability to communicate and to system can be useful for social media websites and other share information with anyone, at any time, and in the messaging applications to counter such attacks and number of people at the same time. There are over 3 reduce the number of cyberbullying cases. The objective billion social media users around the world. According of the cyberbullying detection system is to identify the to the National Crime Security Council (NCPC), cyberbullying text and also take its meaning into cyberbullying is available online where mobile phones, consideration. One first analyzes the various aspects of a video game apps, or any other way to send or send text, particular text and then applies the previous information photos, or videos deliberately injure or embarrass or visuals to find the context of the text. There is a need another person. Cyberbullying can happen at any time all to create a personalized system that can access such a day, week and you can reach anyone anywhere via the text effectively and efficiently. internet. Text, photos, or videos of cyberbullying may be posted in an undisclosed manner. It can be difficult, and sometimes impossible, to track down the source of this post. It was also impossible to get rid of these messages 2 Literature Survey later. Several social media platforms such as Twitter, Instagram, Facebook, YouTube, Snapchat, Skype, and M. Di Capua, et al. proposes an unsupervised Wikipedia are the most common bullying sites on the approach to develop a cyberbullying model based on an internet. Some of the social networking sites, such as amalgam of features, based on traditional textual features Facebook, and the provision of guidance on the as well as some “social features”. The features were prevention of bullying. It has a special section that separated into 4 categories as Syntactic features, explains how to report cyber-bullying and to prevent any Semantic features, Sentiment features, and Social blocking of the user. On Instagram, when someone features. The author used a Growing Hierarchical Self shares photos and videos made by the user to be Organizing Map (GHSOM) network, with a grid of 50 x uncomfortable, so the user can monitor or block them. 50 neurons and 20 features as the input layer. M. Di Users can also report a violation of our Community and Capua, et al have applied the clustering algorithm make Recommendations to the app. k-means to classify the input dataset along with GHSOM on the Formspring dataset. The results of this hybrid unsupervised methodology surpassed the previous * Corresponding author: [email protected] © The Authors, published by EDP Sciences. This is an open access article distributed under the terms of the Creative Commons Attribution License 4.0 (http://creativecommons.org/licenses/by/4.0/). ITM Web of Conferences 40, 03038 (2021) https://doi.org/10.1051/itmconf/20214003038 ICACC-2021 results. The author then tested the youtube dataset with 3 paper, various practical steps required for the different Machine Learning Models: a Naive Bayes development of an effective and efficient application for Classifier, Decision Tree Classifier(C4.5), and a Support cyberbullying detection are described thoroughly. The Vector Machine(SVM) with a Linear Kernel. It was trends involved in the categorization and labeling of data observed that clustering results for the hate posts turned platforms, machine learning models and feature types, out to have a lower precision in the youtube dataset and case studies that made use of such tools are when compared to the FormSpring tests, as textual explained. This paper will serve as an initial step for the analysis and syntactical features perform differently on project in Cyberbullying Detection using Machine both sides. When this hybrid approach was applied to the learning. Twitter dataset, it resulted in a weak recall and F1 Score. G. A. León-Paredes et al. have explained the The model proposed by the authors can be improved and development of a cyberbullying detection model using used in building constructive applications to mitigate Natural Language Processing (NLP) and Machine cyberbullying issues. Learning (ML). A Spanish cyberbullying Prevention J. Yadav, et al. proposes a new approach to System (SPC) was developed by applying machine cyberbullying detection in social media platforms by learning techniques Naïve Bayes, Support Vector using the BERT model with a single linear neural Machine, and Logistic Regression. The dataset used for network layer on top as a classifier. The model is trained this research was extracted from Twitter. The maximum and evaluated on the Formspring forum and Wikipedia accuracy of 93% was achieved with the help of three dataset. The proposed model gave a performance techniques used. The cases of cyberbullying detected accuracy of 98% for the Form spring dataset and of 96% with the help of this system presented an accuracy of for the Wikipedia dataset which is relatively high from 80% to 91% on average. Stemming and lemmatization the previously used models. The proposed model gave techniques in NLP can be implemented to further better results for the Wikipedia dataset due to its large increase the accuracy of the system. Such a model can size g without the need for oversampling whereas the also be implemented for detection in English and local Form spring dataset needed oversampling. languages if possible. R. R. Dalvi, et al. suggests a method to P. K. Roy, et al. detail about creating a detect and prevent Internet exploitation on Twitter using request for the discovery of hate speech on Twitter with Supervised classification Machine Learning algorithms. the help of a deep convolutional neural network. In this research, the live Twitter API is used to collect Machine learning algorithms such as Logistic Regression tweets and form datasets. The proposed model tests both (LR), Random Forest (RF), Naïve Bayes (NB), Support Support Vector Machine and Naive Bayes on the Vector Machine (SVM), Decision Tree (DT), Gradient collected datasets. To extract the feature, they have used Boosting (GB), and K-nearby Neighbors (KNN) has the TFIDF vectorizer. The results show that the accuracy been used to identify tweets related to hate speech on of the cyberbullying model based on the Support Vector Twitter and features have been removed using the tf-idf Machine is almost 71.25% that is better than the Naive process. The best ML model was SVM but it managed to Bayes which was almost 52.75%. predict 53% hate speech tweets in a 3: 1 dataset to test Trana R.E., et al. goal was to design a the train. The reason behind the low prediction scale was machine learning model to minimize special events the unequal data. The model is based on the prediction of involving text extracted from image memes. The author hate speech tweets. Advanced forms of learning based has compiled a database containing approximately on the Convolutional Neural Network (CNN), 19,000 text views published on YouTube. This study Long-Term Memory (LSTM), and their Contextual discusses the effectiveness of the three machine learning LSTM (CLSTM) combinations have the same effects as machines, the Uninformed Bayes, the Support Vector a separate distributed database. 10-fold cross-validation Machine, and the convolutional neural network used on was used along with the proposed DCNN model and the YouTube database, and compares the results with the obtained a very good recall rate. It was 0.88 for hate existing Form databases. The authors further speech and 0.99 for non-hate speech. Test results investigated algorithms for Internet cyberbullying in confirmed that the k-fold cross-validation process is a sub-categories within the YouTube database. Naive better decision with unequal data. In the future, the Bayes surpassed SVM and CNN in the following four current database can be expanded to achieve better categories: race, ethnicity, politics, and generalism. SVM accuracy. has passed well with the inexperienced Naïve Bayes and S. M. Kargutkar, et al. had proposed a CNN in the same gender group, and all three algorithms system to give a double characterization for have shown equal performance with central body group cyberbullying. The system uses Convolutional Neural accuracy. The results of this study provided data that can Network (CNN) and Keras for content examination as be used to distinguish between incidents of abuse and the relevant strategies at that time provided a guideless non-violence. Future work could focus on the creation of view with less precision. This research involved data a two-part segregation scheme used to test the text from Twitter and YouTube. CNN accuracy was 87%. extracted from images to see if the YouTube database In-depth learning-based models have found their way to provides a better context for aggression-related clusters. identify digital harassment episodes, they can overcome N. Tsapatsoulis, et al. a detailed review of the imprisonment of traditional models, and improve cyberbullying on Twitter is presented. The importance of adoption. identifying different abusers on Twitter is given. In the 2 ITM Web of Conferences 40, 03038 (2021) https://doi.org/10.1051/itmconf/20214003038 ICACC-2021 Jamil H. et al. have described the crucial stage for the effectiveness of the algorithms in implementation of a new social network model and its pattern recognition and classification problems. query language called GreenShip. They showed that with In sentimental features, we try to evaluate the the support devices, GreenShip users can be more sentiment( positive, negative ) of a given text document. effective in the fight against online bullying, and loss of The research shows that human analysts tend to agree traditional, online, and social networks. The reputation around 80-85% of the time and that is the baseline we of a management model that has been introduced to have tried to consider while training our sentiment restrict access to harmful information, which focuses on scoring system. the denial of the criminal code, the means for the In sarcastic features, we try to consider the dissemination of information to the users associated with context incongruity. Incongruity occurs when a the target. GreenShip has a reputation as a model that nonverbal behavior contradicts a person's word. A text provides safe, “green” friends, due to the recognition of may contain half of the objects in a congruent context the different types of friendships on Facebook. The which can be considered as expected context, whereas damage is as a result of bad friends and that was very for the other half, objects were embedded in incongruent limited, and the more complex, that there are many contexts. This can be a major factor in cyberbullying forms of friendship, and the communication lines are put detection because the hidden nature of the sarcastic aside for the sake of the benefits of privacy and control. comment won’t be detected in sentiment analysis because of the context incongruity. We also consider Rasel, Risul Islam, et al. focuses on the removal of pragmatic features like emojis, mentions, etc. while the comments made on social networks, and the analysis detecting the sarcastic nature of the source material. of the question as to whether these observations provide While considering the syntactic features we an offensive meaning. The reactions can be divided into have identified in the lists of insults, we also monitor and three categories: offensive, hate speech, and neither of take into consideration the number of such bad words or the two. The proposed model classifies the notes on the insults present in a single sentence and accordingly map species), with an accuracy of more than 93%. Latent a density to it. We have also validated the badness of the Semantic Analysis (LSA) has been used as a feature entire sentence based on certain parameters like density selection method to reduce the size of the input data. In range. The emphasis of uppercase characters while addition to standard feature extraction methods such as making hate statements is also taken into consideration tokenization, N-gram, TF-IDF was applied to detect the while generating syntactic features because it can be important notes. We made three different machine referred to as an act of shouting or attacking over social learning models, Random Forest, Logistic Regression, media platforms. Similarly, the use of special characters and Support Vector Machines (SVMs) to perform the or patterns formed by them is also brought into calculation, analysis, forecasting, and a teasing consideration while deriving syntactic features. comment. Semantic Features can be used to determine the lexical relation which exists between two words in a language. The meaning of the word can be represented by Semantic features. Here we have tried to identify the trigram and the bigrams that occur while referencing 3 Proposed Methodology something in the text format. Here usually the negation In this paper, a method to detect cyberbullying on social of the sentence is considered along with the mapping of media is proposed that is not just based on the different pronouns that can be implicitly or explicitly sentimental analysis but also considers the syntactic, used to refer to another individual while harassing semantic, and sarcastic nature of the sentence before someone over social media. classifying it as hate speech. To achieve our goal we start Social features refer to the social behavior of with the traditional sentiment analysis where we perform the victim or the bully itself. The post itself won't be contextual mining of text to identify and extract the sufficient to detect the nature of the text. We have subjective information in the source material to considered patterns in the behaviors of the bullies and understand the opinion, emotion, or attitude towards the identified a few features. We have considered the direct topic. Later we introduce a group of “social” features tagging of the victim while using hate speech. We also that can highly affect and guide the cyberbullying try to gain information regarding the context of the post detection process. We have divided all the features we based on the previous interactions between the bully and have extracted into five categories: the victim. Profiling of the author can be done to Sentimental Features discover its past interactions and involvement in similar Sarcastic Features malicious activities over social media platforms. Syntactic Features We proposed a cyberbullying detection model Semantic Features based on transformers. Similar to RNN, transformers can Social Features also be used to solve a wide variety of NLP(Natural All these features have been categorized based Language Processing) problems like translation and text on the literature survey of the existing systems and each summarization as they can take sequential data as input. feature uniquely identifies the text. Choosing A recent improvement on the natural language task informative, descriptive, and independent features is a introduced the BERT. The BERT is a recent paper published by researchers at Google AI Language. BERT 3 ITM Web of Conferences 40, 03038 (2021) https://doi.org/10.1051/itmconf/20214003038 ICACC-2021 stands for Bidirectional Encoder from Transformers. It is a bidirectional model that is pre-trained on unlabeled texts from both left and right directions to understand the meaning of both contexts. BERT is a powerful model for NLP tasks because of the use of semi-supervised Fig.2. BERT model flow chart based on sentiment analysis learning. We can use this model to create a state-of-the-art machine learning model for a specific task by applying an additional task-specific layer on top of the BERT architecture. BERT is a Bidirectional model which means it aims to understand the meaning of the 4 Result Analysis word from both the left and the right context to derive a The Fig.3 represents the input processing and prediction better meaning during the training phase. result that we performed during our testing. We used a tweet from Twitter with the trace of bullying and applied We saw a bat. it to our model. Fig 4 shows the classification report This bat was given to me by my father. based on our testing data. Here labels 0 and 1 represent Bullying and Non-Bullying respectively. The Fig.5 Here in the first sentence if we focus on the represents the confusion matrix based on the result of context of the underlined word “bat” from the left till the our testing data. Table 1 represents the accuracy of the word, it refers to the nocturnal animal. Whereas if we SVM and Naive Bayes that is 71.25% and 52.70% focus on the context of the word “bat” in the second respectively, when applied on the same dataset from. sentence from right till the word it refers to the bat from Table 2 represents the accuracy of the BERT model on the game of cricket. Thus a machine can face problems the same dataset. The result shows better accuracy when predicting the actual meaning of the word without using the BERT model for sentiment analysis on the considering both the context. This problem is solved by Twitter dataset. Our proposed model gave a better BERT as it is a bidirectional model. accuracy of 91.90% when applied to the Twitter dataset BERT model requires its input in a for the sentimental analysis which can be considered as a preprocessed form as per the rules made by its developer. greater result when compared to the traditional machine These rules have helped the model to achieve better learning models used on similar datasets. performances. All inputs are embedded as a combination of the other 3 embeddings and given as an input to the model: Position embedding: BERT reads and uses existing embedding to express word order in a sentence. Segment Embedding: BERT can also take more than one sentence as input functions. It uses this embedding to understand the difference between two different sentences. Token Embeddings: This is the embedded text token from Word Piece token vocabulary. Fig.3. Implementation Result Fig.4. Classification report Fig.1. BERT model based on sentiment analysis The Fig.1 depicts the BERT model we developed for sentimental analysis. The three separate embeddings are summarized together to give input to the internal layers of the model. The Fig.2 demonstrates the flow of the sentimental analysis process. To begin with, the final CLS token will output a matrix of hidden size. Furthermore it will be passed to a classifier layer. In conclusion the classifier layer will determine the sentiment of the input text. Fig. 5. Confusion Matrix 4 ITM Web of Conferences 40, 03038 (2021) https://doi.org/10.1051/itmconf/20214003038 ICACC-2021 Found on YouTube. In: Ahram T. (eds) Table 1. Accuracy of SVM and Naive Bayes from Advances in Artificial Intelligence, Software Classifier Accuracy in percentage and Systems Engineering. AHFE 2020. Advances in Intelligent Systems and Naïve Bayes Classifier 52.70 Computing, vol 1213. Springer, Cham. https://doi.org/10.1007/978-3-030-51328-3_2. (2020) Support Vector Machine 71.25 5. N. Tsapatsoulis and V. Anastasopoulou, Cyberbullies in Twitter: A focused review, SMAP, pp. 1-6, doi: Table 2. Accuracy of BERT Model 10.1109/SMAP.2019.8864918. (2019) Classifier Accuracy in 6. G. A. León-Paredes et al., Presumptive percentage Detection of Cyberbullying on Twitter through Natural Language Processing and Machine Pre-Trained BERT (testing) 70.89 Learning in the Spanish Language, CHILECON pp. 1-7, doi: Pre-Trained BERT 91.90 10.1109/CHILECON47746.2019.8987684. (training) (2019) 7. P. K. Roy, A. K. Tripathy, T. K. Das and X. -Z. Gao, A Framework for Hate Speech Detection Using Deep Convolutional Neural Network, in IEEE Access, vol. 8, pp. 204951-204962,, doi: 5 Conclusion 10.1109/ACCESS.2020.3037073. (2020) We proposed a semi-supervised approach in detecting 8. S. M. Kargutkar and V. Chitre, A Study of cyberbullying based on the five features that can be used Cyberbullying Detection Using Machine to define a cyberbullying post or message using the Learning Techniques, ICCMC, pp. 734-739, BERT model. While considering just one of the features doi:10.1109/ICCMC48092.2020.ICCMC-00013 which was sentimental features the BERT model 7. (2020) achieved 91.90% accuracy when trained over dual cycles 9. Jamil, H. and R. Breckenridge. Greenship: a which outperformed the traditional machine learning social networking system for combating models. The BERT model can achieve more accurate cyber-bullying and defending personal results if provided with a large dataset. We can try to reputation., ACM : n. pag. (2018) achieve even better results in the cyberbullying detection 10. Rasel, Risul Islam & Sultana, Nasrin & Akhter, process if we consider all the features that we have Sharna & Meesad, Phayung, Detection of proposed in this research paper. Based on all the features Cyber-Aggressive Comments on Social Media an application can be created to detect the bullying traces Networks: A Machine Learning and Text mining and thus help in detecting and reporting such posts. A approach. 37-41. 10.1145/3278293.3278303. combination of other models on top of the BERT model (2018) can also be used in the future to create a state-of-the-art model for the specific NLP tasks in detecting cyberbullying. References 1. M. Di Capua, E. Di Nardo and A. Petrosino, Unsupervised cyberbullying detection in social networks, ICPR, pp. 432-437, doi: 10.1109/ICPR.2016.7899672. (2016) 2. J. Yadav, D. Kumar and D. Chauhan, Cyberbullying Detection using Pre-Trained BERT Model, ICESC, pp. 1096-1100, doi: 10.1109/ICESC48915.2020.9155700. (2020) 3. R. R. Dalvi, S. Baliram Chavan and A. Halbe, Detecting A Twitter Cyberbullying Using Machine Learning, ICICCS, pp. 297-301, doi: 10.1109/ICICCS48265.2020.9120893. (2020) 4. Trana R.E., Gomez C.E., Adler R.F. (2021) Fighting Cyberbullying: An Analysis of Algorithms Used to Detect Harassing Text 5