Evaluating the Evaluation Matrices: Integrating Spatial Assessment in Geospatial AI Model Training and Evaluation

Abstract

This paper examines the limitations of current evaluation metrics in GeoAI. Through two case studies on deep learning models—a building detection classification problem and a remote sensing image fusion regression problem—this paper demonstrates how traditional statistical evaluation matrices alone can be misleading in geospatial problems. The findings indicate that traditional metrics (e.g., RMSE, MAE) used in current GeoAI models can have difficulty capturing the spatial dimensions inherent to geospatial problems. This paper suggests that the model evaluation process in GeoAI should move beyond traditional evaluation matrices by integrating spatial thinking throughout the modeling pipeline—not only incorporating spatial accuracy in model evaluation but also embedding it within optimization functions in model structure and model training.