Browsing by Author "Xu, Kui"

Now showing 1 - 3 of 3
  • Thumbnail Image
    Anomaly Detection Through System and Program Behavior Modeling
    Xu, Kui (Virginia Tech, 2014-12-15)
    Various vulnerabilities in software applications become easy targets for attackers. The trend constantly being observed in the evolution of advanced modern exploits is their growing sophistication in stealthy attacks. Code-reuse attacks such as return-oriented programming allow intruders to execute mal-intended instruction sequences on a victim machine without injecting external code. Successful exploitation leads to hijacked applications or the download of malicious software (drive-by download attack), which usually happens without the notice or permission from users. In this dissertation, we address the problem of host-based system anomaly detection, specifically by predicting expected behaviors of programs and detecting run-time deviations and anomalies. We first introduce an approach for detecting the drive-by download attack, which is one of the major vectors for malware infection. Our tool enforces the dependencies between user actions and system events, such as file-system access and process execution. It can be used to provide real time protection of a personal computer, as well as for diagnosing and evaluating untrusted websites for forensic purposes. We perform extensive experimental evaluation, including a user study with 21 participants, thousands of legitimate websites (for testing false alarms), 84 malicious websites in the wild, as well as lab reproduced exploits. Our solution demonstrates a usable host-based framework for controlling and enforcing the access of system resources. Secondly, we present a new anomaly-based detection technique that probabilistically models and learns a program's control flows for high-precision behavioral reasoning and monitoring. Existing solutions suffer from either incomplete behavioral modeling (for dynamic models) or overestimating the likelihood of call occurrences (for static models). We introduce a new probabilistic anomaly detection method for modeling program behaviors. Its uniqueness is the ability to quantify the static control flow in programs and to integrate the control flow information in probabilistic machine learning algorithms. The advantage of our technique is the significantly improved detection accuracy. We observed 11 up to 28-fold of improvement in detection accuracy compared to the state-of-the-art HMM-based anomaly models. We further integrate context information into our detection model, which achieves both strong flow-sensitivity and context-sensitivity. Our context-sensitive approach gives on average over 10 times of improvement for system call monitoring, and 3 orders of magnitude for library call monitoring, over existing regular HMM methods. Evaluated with a large amount of program traces and real-world exploits, our findings confirm that the probabilistic modeling of program dependences provides a significant source of behavior information for building high-precision models for real-time system monitoring. Abnormal traces (obtained through reproducing exploits and synthesized abnormal traces) can be well distinguished from normal traces by our model.
  • Thumbnail Image
    Epidemiology Network
    Sundar, Naren; Xu, Kui (2014-05-11)
    This project aims at developing an RDF graph building service for Cyber Infrastructure for Network Science (CINET). The purpose of this service is to do web crawling and find digital contents related to user requests. More specifically, the type of contents to be collected should be related to epidemiology. Eventually the service should deliver an RDF network of digital contents that can be stored on CINET for analysis. Simply using a search engine such as Google, or a web crawler in an undirected way, won't be able to satisfy the requirements of this problem, due to the lack of organization of the results and the ambiguity of the information. Our service should present to users networks of interconnected digital objects, which are organized based on their topics. In the results, all digital objects are connected as a network of related contents based on a user's request. In addition to that, those who are closer to a topic will be more strongly connected in a sub-network. The developed topic modeling approach emulates human behavior when searching relevant research papers. It automatically crawls the DBLP bibliography website and constructs a network of papers based on a user query.
  • Thumbnail Image
    User-Behavior Based Detection of Infection Onset
    Xu, Kui; Yao, Danfeng (Daphne); Ma, Qiang; Crowell, Alexander (Department of Computer Science, Virginia Polytechnic Institute & State University, 2010)
    A major vector of computer infection is through exploiting software or design flaws in networked applications such as the browser. Malicious code can be fetched and executed on a victim’s machine without the user’s permission, as in drive-by download (DBD) attacks. In this paper, we describe a new tool called DeWare for detecting the onset of infection delivered through vulnerable applications. DeWare explores and enforces causal relationships between computer-related human behaviors and system properties, such as file-system access and process execution. Our tool can be used to provide real time protection of a personal computer, as well as for diagnosing and evaluating untrusted websites for forensic purposes. Besides the concrete DBD detection solution, we also formally define causal relationships between user actions and system events on a host. Identifying and enforcing correct causal relationships have important applications in realizing advanced and secure operating systems. We perform extensive experimental evaluation, including a user study with 21 participants, thousands of legitimate websites (for testing false alarms), as well as 84 malicious websites in the wild. Our results show that DeWare is able to correctly distinguish legitimate download events from unauthorized system events with a low false positive rate (< 1%).