The IEEE 802.11 based Wireless LAN (WLAN) has become increasingly ubiquitous in recent years. However, due to the broadcast nature of wireless communication, attackers can exploit the existing vulnerabilities in IEEE 802.11 to launch various types of attacks in wireless and wired networks.

This thesis presents a statistical based hybrid Intrusion Detection System (IDS) for Rogue Access Point (RAP) detection, which employs distributed monitoring devices to monitor on 802.11 link layer activities and a centralized detection module at a gateway router to achieve higher accuracy in detection of rogue devices. This detection approach is scalable, non-intrusive and does not require any specialized hardware. It is designed to utilize the existing wireless LAN infrastructure and is independent of 802.11a/b/g/n. It works on passive monitoring of wired and wireless traffic, and hence is easy to manage and maintain. In addition, this approach requires monitoring a smaller number of packets for detection as compared to other detection approaches in a heterogeneous network comprised of wireless and wired subnets.

Centralized detection is done at a gateway router by differentiating wired and wireless TCP traffic using Weighted Sequential Hypothesis Testing on inter-arrival time of TCP ACK-pairs. A decentralized module takes care of detection of MAC spoofing and totally relies on 802.11 beacon frames. Detection is done through analysis of the clock skew and the Received Signal Strength (RSS) as fingerprints using a naïve Bayes classifier to detect presence of rogue APs.

Analysis of the system and extensive experiments in various scenarios on a real system have proven the efficiency and accuracy of the approach with few false positives/negatives and low computational and storage overhead.