Disclaimer: The content and data used for the article contains text that may be considered profane, vulgar, or offensive. The content and data do not purport to reflect the opinions or views of the author or the company.
The 80s and 90s were rife with the high pitched scream of dial up internet and exasperated sighs of frustrated users awaiting hours long downloads of websites and dropped connections during email deliveries.
Fast forward 30 years. With the advent of high speed internet, came the explosion of Web 2.0 and Social Media. The rise of Google, Facebook, Twitter, Instagram and LinkedIn has changed the way we consume information, interact and communicate. The evolution of the Internet also inevitably gave rise to Phishing Websites, Fake news and Internet Trolling, bringing along malicious acts of theft, spreading misinformation and bullying/harassment. This article is related to some machine learning exploratory work carried out for these three topics.
Summary
The article contains the Context, Objective, Data Description and Model Results, following :
Phishing Websites
Fake News
Internet Trolling
Below is an summary of the machine learning model performance for each use case:
Phishing Websites
Context
A data set containing real and fake websites, available at PhishStorm.
Objective
The objective was to build a machine learning model to detect real/fake websites.
Data Description
A pretty straight forward data set that contained urls labeled "Real" or "Fake".
Training data: 47956, Test data: 47954
The data used for the experiment can be found here.
Model Results
The results can be found here.
The model was 97.8% accurate overall.
Fake News
Context
Data containing short political statements in various contexts from PolitiFact.com.
Objective
The objective was to build a machine learning model to detect fake political statements.
Data Description
The original data set contained political statements made in various contexts. The statements had the following associated categories/labels. The original categories were converted into True or False statement labels:
Below is a data breakdown:
The original data set also had subject, speaker, speaker's job title, party affiliation, and venue columns.
Narratives were derived combining the statement and the above additional columns to train and test the model . Below is an example:
Narrative: Peter Kinder, a Lieutenant Governor of the Republican party, spoke about diversity, economy, and jobs at a gubernatorial debate in Missouri. Youth unemployment in minority communities is about 40 to 45 percent.
The data can be found here.
Model Results
Overall, the model was 65% accurate when evaluated on the test data. Below is a breakdown for the accuracy of true and fake statements. The results can be found here:
Internet Trolling
Context
Data set originating from Kaggle containing social media comments, classified by toxic comment category types and toxic comments target by demographic type.
Objective
The objective was to build machine learning models to toxic comments target by category and demographic type.
Toxic Category Types
Data Description
A dataset containing social comments labeled by toxic category type. The toxic category types are toxic, severe toxicity, obscene, identity attack, insult and threat. The social comments can be made up of a combination of any of the categories (i.e. a social comment could be toxic, obscene and an insult). For example:
The data can be found here. Below is the training data (121,875 records) and test data (121,873 records) breakdown:
Model Results
Results can be found here. Overall, the model was 79.5% accurate when evaluated on the test data. Below is a breakdown of the accuracy by category:
Toxic Comments By Target Demographic Type
Data Description
A dataset containing social comments labeled by target demographic type. The target demographic types are black, christian, female, homosexual, male and muslim. The social comments can be made up of a combination of any of the types (i.e. a social comment could be classified as toxic, male and female). For example:
The data can be found here. Below is the training data (25,647 records) and test data (25,650 records) breakdown:
Model Results
Results can be found here. The model was 93% accurate overall. Below is a breakdown of the accuracy by category:
Conclusion
It has been demonstrated that machine learning can play a part in monitoring the negative elements of social media. It has the potential to prevent the significant impacts like influence on a country's elections, or cyber bullying.
How
The NoamAi platform is a fully automated NLP system, embodying the principles of DevOps and CI/CD (MLOps) in machine learning (ML) and AI. It simplifies and automates the end to end ML model build process (data preparation -> model training -> model deployment) by ways of standardisation, consistency, speed and scale. All users need to do is to provide the data and define the problem.
The platform has the ability to build models for the following use cases:
Classification - Classify sentences or text narratives (i.e. sentiment analysis)
Comprehension - Answer questions based on paragraphs of text
Summarisation - Summarising of text
Named entity recognition - Text information extraction into predefined categories such as person names, organizations, locations, medical codes, time expressions, quantities, monetary values, percentages