Scaling transformer-based abstractive multi-document summarization to Arabic language

Abstract

 The abstractive multi-document summarization (AMDS) of the Arabic language is a daunting task because of the morphological richness of its language, the syntactic diversity, and the lack of task-specific benchmarks. We present Ara25, a Modern Standard Arabic multi-document dataset, built upon more than a hundred news sources, and divided into 700+ real-world events, in this work, wherein each event contains several related articles, each with a single human-written abstractive description. Based on this resource, we construct and comparatively analyze five transformer-based summarizers, mT5, Longformer EncoderDecoders (LED), AraT5, AarBERT and multilingual BERT, within a common long-context fine-tuning protocol that encodes and decodes multi-article inputs with explicit article-boundary markers. We include Arabic conscious preprocessing and a full evaluation suite, consisting of ROUGE, BLEU, METEOR, chrF, BERTScore, MoverScore and human judgments of fluency, coherence, faithfulness, and coverage. According to experimental results, long-context abstractive models, especially those initialized with an XLSum-tuned checkpoint, significantly outperform encoder-decoder models on short contexts with respect to syntactic consistency and semantic adequacy, making them the best current baselines to date on Arabic AMDS on Ara25. The combination of the dataset, modeling recipe and evaluation protocol gives a realistic benchmark and a reproducible reference point to future study on the abstractive multi-document summarization in low-resource languages.