Cancer is a disease of cellular evolution. For this cellular evolution to take place, a population of cells must contain functional heterogeneity and an assessment of this heterogeneity in the form of natural selection. Cancer cells from advanced malignancies are genomically and functionally very different compared to the healthy cells from which they evolved. Genomic alterations include aneuploidy (numerical and structural changes in chromosome content) and polyploidy (e.g., whole genome doubling), which can have considerable effects on cell physiology and phenotype. Likewise, conditions in the tumor microenvironment are spatially heterogeneous and vastly different than in healthy tissues, resulting in a number of environmental niches that play important roles in driving the evolution of tumor cells. While a number of studies have documented abnormal conditions of the tumor microenvironment and the cellular consequences of aneuploidy and polyploidy, a thorough overview of the interplay between karyotypically abnormal cells and the tissue and tumor microenvironments is not available. Here, we examine the evidence for how this interaction may unfold during tumor evolution. We describe a bidirectional interplay in which aneuploid and polyploid cells alter and shape the microenvironment in which they and their progeny reside; in turn, this microenvironment modulates the rate of genesis for new karyotype aberrations and selects for cells that are most fit under a given condition. We conclude by discussing the importance of this interaction for tumor evolution and the possibility of leveraging our understanding of this interplay for cancer therapy.