The problem of automatic classification of pictures using an intelligent decision-making system based on the knowledge graph and fine-grained image analysis
Вантажиться...
Дата
2022
Науковий керівник
Назва журналу
Номер ISSN
Назва тому
Видавець
КПІ ім. Ігоря Сікорського
Анотація
In order to prevent the illegal export of paintings abroad, a museum
examination using various methods for studying a work of art is carried out. At the
same time, an analysis is also made of historical, art history, financial and other information
and documents confirming the painting’s authenticity — provenance.
Automation of such examination is hampered by the need to take into account numerical
values of visual features, quality indicators, and verbal descriptions from
provenance. In this paper, we consider the problem of automatic multi-task classification
of paintings for museum expertise. A system architecture is proposed that
checks provenance, implements a fine-grained image analysis (FGIA) of visual
image features, and automatically classifies a painting by authorship, genre, and
time of creation. Provenance is contained in a knowledge graph; for its vectorization,
it is proposed to use a graph2vec type encoder with an attention mechanism.
Fine-grained image analysis is proposed to be performed using searching discriminative
regions (SDR) and learning discriminative regions (LDR) allocated by convolutional
neural networks. To train the classifier, a generalized loss function is proposed.
A data set is also proposed, including provenance and images of paintings by European
and Ukrainian artists.
Опис
Ключові слова
automatic multi-task classification, knowledge graph, attention mechanism, fine-grained image analysis, museum expertise, paintings, convolutional neural networks, автоматична багатозадачна класифікація, граф знань, механізм уваги, дрібнодетальний аналіз зображень, музейна експертиза, твори живопису, згорткові нейронні мережі
Бібліографічний опис
The problem of automatic classification of pictures using an intelligent decision-making system based on the knowledge graph and fine-grained image analysis / A. A. Martynenko, A. D. Tevyashev, N. Ye. Kulishova, B. I. Moroz // Системні дослідження та інформаційні технології : міжнародний науково-технічний журнал. – 2022. – № 4. – С. 58-67. – Бібліогр.: 30 назв.