Abstract:

Extreme Multi-Label Classification (XMLC) plays a pivotal role in organizing and retrieving information in largescale textual collections, by considering a very high number of potential labels for the documents. In this paper, we conduct an empirical evaluation of several XMLC approaches encompassing both dedicated techniques (AttentionXML and XR Transformer) and the use of Large Language Models (LLaMA2 7b Chat, LLaMA3 8b Instruct, and two Mistral models). We introduce both a new dataset based on OpenAlex as well as several new metrics to conduct our evaluations. Our results suggest that fine-tuning the LLMs using Low-Rank Adaptation significantly improves the performance of the models, bringing their results close to the ones of dedicated techniques. In the end, none of the method emerges as a clear winner, as picking the optimal XMLC technique heavily depends on the requirements of the use-case at hand.

Research Paper:

article

Source: 2024 IEEE International Conference on Big Data (BigData)

BibTex:

@INPROCEEDINGS{10825837,
author={Solanki, Bhargav and Ostapuk, Natalia and Dolamic, Ljiljana and Mermoud, Alain and Cudré-Mauroux, Philippe},
booktitle={2024 IEEE International Conference on Big Data (BigData)},
title={AttentionXML VS LLMs: An Empirical Evaluation of Extreme Multi-Label Classification Techniques},
year={2024},
volume={},
number={},
pages={6151-6160},
keywords={Training;Measurement;Hands;Adaptation models;Costs;Large language models;Multi label classification;Big Data;Transformers;Energy efficiency;Extreme Multi-label Classification;LLM;Empirical Evaluation},
doi={10.1109/BigData62323.2024.10825837}}