Our understanding of how magma‐poor rifts accommodate strain remains limited largely due to sparse geophysical observations from these rift systems. To better understand the magma‐poor rifting processes, we investigate the lithospheric structure of the Malawi Rift, a segment of the magma‐poor western branch of the East African Rift System. We analyze Bouguer gravity anomalies from the World Gravity Model 2012 using the two‐dimensional (2‐D) radially averaged power‐density spectrum technique and 2‐D forward modeling to estimate the crustal and lithospheric thickness beneath the rift. We find: (1) relatively thin crust (38–40 km) beneath the northern Malawi Rift segment and relatively thick crust (41–45 km) beneath the central and southern segments; (2) thinner lithosphere beneath the surface expression of the entire rift with the thinnest lithosphere (115–125 km) occurring beneath its northern segment; and (3) an approximately E‐Wtrending belt of thicker lithosphere (180–210 km) beneath the rift's central segment. We then use the lithospheric structure to constrain three‐dimensional numerical models of lithosphereasthenosphere interactions, which indicate ~3‐cm/year asthenospheric upwelling beneath the thinner lithosphere. We interpret that magma‐poor rifting is characterized by coupling of crust‐lithospheric mantle extension beneath the rift's isolated magmatic zones and decoupling in the rift's magma‐poor segments. We propose that coupled extension beneath rift's isolated magmatic zones is assisted by lithospheric weakening due to melts from asthenospheric upwelling whereas decoupled extension beneath rift's magma‐poor segments is assisted by concentration of fluids possibly fed from deeper asthenospheric melt that is yet to breach the surface.