Evaluation of a Social Media Bot Detector for integration into MITRE SP!CE™

Mulhern, Sullivan

Student Work

Evaluation of a Social Media Bot Detector for integration into MITRE SP!CE™

Public

Social media bots have become an inevitable nuisance for online communities. These bots are used to heavily influence these communities by either drowning out actual conversation or control the narrative. Due to their potential, social media companies and governments are intensely interested in identifying and minimizing the impact of bot actors. Bot detectors can detect some bot activity, but different methods vary in their ability to correctly classify users. This study aims to evaluate a method of bot detection called the Rest-Sleep-Comment (RSC) method for integration into MITRE SP!CE™. This detector should be able to be to detect bots at the same rate as proposed in the original RSC paper. The data that was used for evaluating the detector is Reddit data from September 2015, October 2015, June 2020, and June 2021. This data was stripped down to only the data needed, which were the author and created_utc keys. Originally, there were no bot accounts in the dataset. Without any bots, the detector could not be properly trained. It was then decided to fabricate bot accounts using noised timestamps. These synthetic bot accounts were then added to the datasets. These new datasets were then used to test the detector again. Once the detector was made and the dataset contained bot accounts, the detector was run and the results were not promising. The detector was unable to classify any bot accounts correctly. Therefore, the detector should not be integrated into MITRE SP!CE™.

This report represents the work of one or more WPI undergraduate students submitted to the faculty as evidence of completion of a degree requirement. WPI routinely publishes these reports on its website without editorial or peer review.

Creator