Behavioural correlation for malicious bot detection

Al-Hammadi, Yousof Ali Abdulla (2010) Behavioural correlation for malicious bot detection. PhD thesis, University of Nottingham.

[thumbnail of thesis_final.pdf]
Preview
PDF - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
Download (2MB) | Preview

Abstract

Over the past few years, IRC bots, malicious programs which are remotely controlled by the attacker, have become a major threat to the Internet and its users. These bots can be used in different malicious ways such as to launch distributed denial of service (DDoS) attacks to shutdown other networks and services. New bots are implemented with extended features such as keystrokes logging, spamming, traffic sniffing, which cause serious disruption to targeted networks and users. In response to these threats, there is a growing demand for effective techniques to detect the presence of bots/botnets. Currently existing approaches detect botnets rather than individual bots. In our work we present a host-based behavioural approach for detecting bots/botnets based on correlating different activities generated by bots by monitoring function calls within a specified time window. Different correlation algorithms have been used in this work to achieve the required task. We start our work by detecting IRC bots' behaviours using a simple correlation algorithm. A more intelligent approach to understand correlating activities is also used as a major part of this work. Our intelligent algorithm is inspired by the immune system. Although the intelligent approach produces an anomaly value for the classification of processes, it generates false positive alarms if not enough data is provided. In order to solve this problem, we introduce a modified anomaly value which reduces the amount of false positives generated by the original anomaly value.

We also extend our work to detect peer to peer (P2P) bots which are the upcoming threat to Internet security due to the fact that P2P bots do not have a centralized point to shutdown or traceback, thus making the detection of P2P bots a real challenge. Our evaluation shows that correlating different activities generated by IRC/P2P bots within a specified time period achieves high detection accuracy. In addition, using an intelligent correlation algorithm not only states if an anomaly is present, but it also names the culprit responsible for the anomaly.

Item Type: Thesis (University of Nottingham only) (PhD)
Supervisors: Aickelin, U.
Keywords: web, internet, malicious bot detection, bots, ddos, distributed denial of service, irc bots, p2p bots, correlation
Subjects: Q Science > QA Mathematics > QA 75 Electronic computers. Computer science
Faculties/Schools: UK Campuses > Faculty of Science > School of Computer Science
Item ID: 11359
Depositing User: EP, Services
Date Deposited: 20 Oct 2010 09:29
Last Modified: 14 Oct 2017 06:59
URI: https://eprints.nottingham.ac.uk/id/eprint/11359

Actions (Archive Staff Only)

Edit View Edit View