The performance of strained silicon (Si) as the channel material for today’s metal-oxide-semiconductor field-effect transistors may be reaching a plateau. New channel materials with high carrier mobility are being investigated as alternatives and have the potential to unlock an era of ultra-low-power and high-speed microelectronic devices. Chief among these new materials is germanium (Ge). This work reviews the two major remaining challenges that Ge based devices must overcome if they are to replace Si as the channel material, namely, heterogeneous integration of Ge on Si substrates, and developing a suitable gate stack. Next, Ge is compared to compound III-V materials in terms of p-channel device performance to review how it became the first choice for PMOS devices. Different Ge device architectures, including surface channel and quantum well configurations, are reviewed. Finally, state-of-the-art Ge device results and future prospects are also discussed.