Detection of Illicit Drug Trafficking Events on Instagram: A Deep Multimodal Multilabel Learning Approach

by   Chuanbo Hu, et al.

Social media such as Instagram and Twitter have become important platforms for marketing and selling illicit drugs. Detection of online illicit drug trafficking has become critical to combat the online trade of illicit drugs. However, the legal status often varies spatially and temporally; even for the same drug, federal and state legislation can have different regulations about its legality. Meanwhile, more drug trafficking events are disguised as a novel form of advertising commenting leading to information heterogeneity. Accordingly, accurate detection of illicit drug trafficking events (IDTEs) from social media has become even more challenging. In this work, we conduct the first systematic study on fine-grained detection of IDTEs on Instagram. We propose to take a deep multimodal multilabel learning (DMML) approach to detect IDTEs and demonstrate its effectiveness on a newly constructed dataset called multimodal IDTE(MM-IDTE). Specifically, our model takes text and image data as the input and combines multimodal information to predict multiple labels of illicit drugs. Inspired by the success of BERT, we have developed a self-supervised multimodal bidirectional transformer by jointly fine-tuning pretrained text and image encoders. We have constructed a large-scale dataset MM-IDTE with manually annotated multiple drug labels to support fine-grained detection of illicit drugs. Extensive experimental results on the MM-IDTE dataset show that the proposed DMML methodology can accurately detect IDTEs even in the presence of special characters and style changes attempting to evade detection.


page 2

page 6

page 7


Identifying Illicit Drug Dealers on Instagram with Large-scale Multimodal Data Fusion

Illicit drug trafficking via social media sites such as Instagram has be...

Unveiling the Potential of Knowledge-Prompted ChatGPT for Enhancing Drug Trafficking Detection on Social Media

Social media platforms such as Instagram and Twitter have emerged as cri...

Image Matters: A New Dataset and Empirical Study for Multimodal Hyperbole Detection

Hyperbole, or exaggeration, is a common linguistic phenomenon. The detec...

MM-Claims: A Dataset for Multimodal Claim Detection in Social Media

In recent years, the problem of misinformation on the web has become wid...

MM-AU:Towards Multimodal Understanding of Advertisement Videos

Advertisement videos (ads) play an integral part in the domain of Intern...

VERITE: A Robust Benchmark for Multimodal Misinformation Detection Accounting for Unimodal Bias

Multimedia content has become ubiquitous on social media platforms, lead...

Protest Activity Detection and Perceived Violence Estimation from Social Media Images

We develop a novel visual model which can recognize protesters, describe...

Please sign up or login with your details

Forgot password? Click here to reset