Low-resource text classification using cross-lingual models for bullying detection in the ukrainian language
Вантажиться...
Дата
2023
Автори
Науковий керівник
Назва журналу
Номер ISSN
Назва тому
Видавець
КПІ ім. Ігоря Сікорського
Анотація
This paper aims on building bullying detection model for Ukrainian language. Considering absence of labeled datasets for bullying detection and classification in Ukrainian, small Ukrainian dataset (4k samples) was gathered and used for testing models in this research. Taking into account very small number of Ukrainian datasets in general this dataset is publicly available for testing and benchmarking other text classification models. Modern approaches to text class classification in low-resource languages are studied in the paper. We apply zero-shot technique and evaluate performance of modern multilingual, cross-lingual state-of-the-art models and embeddings for text classification in Ukrainian language, including mBERT, XLM-R, LASER and MUSE. Experimental results shows that zero-shot approaches for classification task allow to achieve F1 score of 67-69% for multilingual models trained on English dataset only, having 88-91% test accuracy on English data. We also show that machine translation of English data can be used for estimating model performance in other languages, i.e. only 0-2% difference in test accuracy compared to natural data was received for best models XLM-R and LASER. Zero-shot approach for binary detection task showed even better results 81% compared to 91,59% on original English data. We then enhance the best XLM-R model by training it on our natural Ukrainian dataset and confirm benefits of augmenting low-resource language dataset with machine transla tions from resource-rich English data. Finally, the model for bullying detection in the Ukrainian language is built achieving F1 score of 91,59% with only 12k samples dataset in different languages.
Опис
Ключові слова
multilingual models, zero-shot classification, bullying detection, XLM-RoBERTa, mBERT, LASER, MUSE
Бібліографічний опис
Oliinyk, V. Low-resource text classification using cross-lingual models for bullying detection in the ukrainian language / V. Oliinyk, І. Matviichuk // Адаптивні системи автоматичного управління : міжвідомчий науково-технічний збірник. – 2023. – № 1 (42). – С. 87-100. – Бібліогр.: 24 назви.