Tommaso Bonomo

sapienzanlp/bookcoref · Datasets at Hugging Face We’re on a journey to advance and democratize artificial intelligence through open source and open science.

We hope our contribution can become a useful benchmark for the challenging setting of Coreference Resolution on books! We release our code and dataset:

GitHub: github.com/sapienzaNLP/bookcoref
HuggingFace: huggingface.co/datasets/sapienzanlp/bookcoref

Thank you for your attention! (6/6)

Longdoc is the best performing model in both off-the-shelf and fine-tuned settings. Although BOOKCOREF Silver enables fine-tuned models to achieve a better score, 67 CoNLL F1 points are still far from current SOTA scores on small- or medium-sized datasets. (5/6)

Our pipeline achieves 80.5 CoNLL F1 on our manual annotated split, BOOKCOREF Gold, warranting the use of BOOKCOREF Silver as a fine-tuning dataset.

We test current SOTA Coreference Resolution systems on our benchmark, both off-the-shelf and after fine-tuning. (4/6)

We manually annotate 3 books for our test set and develop a 3-step pipeline to auto-annotate long books from text and character lists. We initialize coreferential clusters via explicit character mentions, refine them with an LLM, and expand using a local coreference model. (3/6)

Coreference Resolution datasets currently focus on small- to medium-sized text, preventing the development of robust long-document Coreference Resolution systems. We introduce BOOKCOREF, a large-scale dataset obtained through our BOOKCOREF Pipeline to fill this gap. (2/6)

🚨 Our paper "BOOKCOREF: Coreference Resolution at Book Scale" was accepted at #ACL2025 main conference!

Kudos to all my co-authors: @giulianomartinelli.bsky.social, @perelluis13.bsky.social and Roberto Navigli, within the Sapienza NLP group.

Paper: arxiv.org/abs/2507.12075

🧵A brief thread: (1/6)

Hi, would also love to be added if there's still space! Thank you!