(КПІ ім. Ігоря Сікорського, 2022) Martynenko, A. A.; Tevyashev, A. D.; Kulishova, N. Ye.; Moroz, B. I.
In order to prevent the illegal export of paintings abroad, a museum
examination using various methods for studying a work of art is carried out. At the
same time, an analysis is also made of historical, art history, financial and other information
and documents confirming the painting’s authenticity — provenance.
Automation of such examination is hampered by the need to take into account numerical
values of visual features, quality indicators, and verbal descriptions from
provenance. In this paper, we consider the problem of automatic multi-task classification
of paintings for museum expertise. A system architecture is proposed that
checks provenance, implements a fine-grained image analysis (FGIA) of visual
image features, and automatically classifies a painting by authorship, genre, and
time of creation. Provenance is contained in a knowledge graph; for its vectorization,
it is proposed to use a graph2vec type encoder with an attention mechanism.
Fine-grained image analysis is proposed to be performed using searching discriminative
regions (SDR) and learning discriminative regions (LDR) allocated by convolutional
neural networks. To train the classifier, a generalized loss function is proposed.
A data set is also proposed, including provenance and images of paintings by European
and Ukrainian artists.