ImageSI: Interactive Deep Learning for Image Semantic Interaction

Interactive deep learning frameworks are crucial for effectively exploring and analyzing complex image datasets in visual analytics. However, existing approaches often face challenges related to inference accuracy and adaptability. To address these issues, we propose ImageSI, a framework integrating deep learning models with semantic interaction techniques for interactive image data analysis. Unlike traditional methods, ImageSI directly incorporates user feedback into the image model, updating underlying embeddings through customized loss functions, thereby enhancing the performance of dimension reduction tasks. We introduce three variations of ImageSI, ImageSI, prioritizing explicit pairwise relationships from user interaction, and ImageSI and ImageSI, emphasizing clustering by defining groups of images based on user input. Through usage scenarios and quantitative analyses centered on algorithms, we demonstrate the superior performance of ImageSI and ImageSI in terms of inference accuracy and interaction efficiency. Moreover, ImageSI shows competitive results. The baseline model, WMDS$\mathrm{<mrow\; data-mjx-texclass="ORD">-1</mrow>}$, generally exhibits lower performance metrics.