Yeoh, SengMing2021-02-022021-02-022021-02-01vt_gsexam:28919http://hdl.handle.net/10919/102158Control flow hijacking attacks such as Return Oriented Programming (ROP) and data oriented attacks like Data Oriented Programming (DOP) are problems still plaguing modern software today. While there have been many attempts at hardening software and protecting against these attacks, the heavy performance cost of running these defenses and intrusive modifications required has proven to be a barrier to adoption. In this work, we present Monguard, a high-performance hardware assisted in-process monitor protection system utilizing Intel Memory Protection Keys (MPK) to enforce execute-only memory, combined with code randomization and runtime binary patching to effectively protect and hide in-process monitors. Next, we introduce L-MVX, a flexible lightweight Multi-Variant Execution (MVX) system running in the in-process monitor system that aims to solve some of the performance problems of recent MVX defenses through selective program call graph protection and in-process monitoring, maintaining security guarantees either by breaking attacker assumptions or creating a scenario where a particular attack only works on a single variant.ETDIn CopyrightMulti-Variant ExecutionIn-Process MonitorMemory IsolationThesis