The upcoming transition to self-driving cars could lead to a seismic shift in society, one that affects industry practices, regulation landscapes, as well as personal decision-making and social norms around privacy. Major tech companies and traditional auto manufacturers have started working together to conceive an optimal regulatory environment for autonomous vehicles. However, the issue of privacy in the process of collecting, managing, and using data generated from self-driving and connected cars remain one of the biggest challenges yet to be solved. In this position paper, I highlight key privacy challenges and issues in the context of connected and self-driving cars.