MalBoT-DRL: Malware Botnet Detection Using Deep Reinforcement Learning

This technical report presents MalBoT-DRL, a novel approach for detecting malware botnets in Internet of Things (IoT) networks using deep reinforcement learning (RL). The report outlines the increasing threat posed by multistage malware botnets that exploit IoT devices for various cyberattacks, including phishing and Distributed Denial-of-Service (DDoS) attacks. It discusses the limitations of existing machine learning solutions, particularly their lack of generalizability and challenges associated with model drift. MalBoT-DRL aims to address these issues by providing a robust intrusion detection system (IDS) that adapts to evolving malware patterns. The report details the integration of damped incremental statistics with an attention reward mechanism, enhancing the model's adaptability. Performance validation through trace-driven experiments on datasets MedBIoT and N-BaIoT demonstrates high detection rates. The report emphasizes the significance of early-stage detection in the malware lifecycle, contributing to the field of cybersecurity by exploring RL's efficacy in improving IDS generalizability.