Visual Motifs and Artificial Intelligence: Developing Machine Learning Models Based on Comparative Iconography

Abstract

Can new AI datasets go beyond the hierarchical logics of imitation and replication, relating images from different media, comparatively? This article tries to answer this question, and poses new ones, by sharing the methodological foundations and the preliminary results of a research project we are currently developing at Pompeu Fabra University, entitled Visual Motifs Identification and Comparative Image Learning. The basis of this project is to combine the machine learning and computer vision background of mathematicians and engineers with the humanistic expertise of art and film historians, in order to foster a radically different approach to AI datasets and models. Instead of developing algorithms capable of imitating or replicating styles and artists, we propose a new working methodology rooted in the concept of visual motif. What visual motifs offer, compared with existing models that use computer vision strategies, is a more nuanced and refined interpretation of images, based not only on standard recognition of geometrical or semantic data but on the meaningful aesthetic and ideological choices of previous creators through art and media history. Instead of isolating images in a given medium, genre or period, the aim of visual motifs is to juxtapose, compare and discriminate across multiple artforms (painting, sculpture, cinema, photography, video games, comic books, etc.). Therefore, we are first curating a dataset using this comparative methodology, and then training a machine learning model capable of recognizing different visual motifs in a previously unseen image, with an aesthetic and critical background.