Maik Fröbe

SCAI 2025 Online Event on Search-Oriented Conversational AI.

Check out the slides from our SCAI'2025 #convsearch workshop collocated with @ijcai.org #IJCAI2025 on LLMs, retrieval & QA, recommendations, negotiations, evaluation and transparency

scai.info/scai-2025

@patuchen.bsky.social @maik-froebe.bsky.social @tuetschek.bsky.social @mila-quebec.bsky.social

🌟Really excited to share the fourth Strategic Workshop on Information Retrieval (SWIRL) report published in SIGIR Forum!

Paper 👉🏻 www.johannetrippas.com/papers/tripp...

More info 👉🏻 sites.google.com/view/swirl20...

#SWIRL2025 #SIGIR2026 #IR #GenAI #Research #CHIIR2026

an aerial view of tokyo at night with lots of lights ALT: an aerial view of tokyo at night with lots of lights

Some exciting news! 🤗 After 3 amazing years at TREC, the Tip-of-the-Tongue (ToT) shared task will be a core task at NTCIR-19 in 2026. The new track will focus on tip-of-the-tongue information needs in English and East Asian languages.

More details coming soon. See you all in Tokyo next year!

Hello TREC-ToTers! 👋🏽

📆 Good news! We are extending the run submission deadline by 2 weeks. Please submit your runs by **September 10 (Wednesday)** and spread the word.

More info: trec-tot.github.io/guidelines
#TREC2025 #TRECToT #TREC2025ToT

Gentle reminder 📢
All run submissions for the Tip-of-the-Tongue (ToT) Track are due next week Wednesday (Aug 27).

More info: trec-tot.github.io/guidelines
#TREC2025 #TRECToT #TREC2025ToT

Here are some impressions from our ReNeuIR workshop on "Reaching Efficiency in Neural IR" that we had yesterday at #SIGIR2025.

Happy to share that our paper "The Viability of Crowdsourcing for RAG Evaluation" received the Best Paper Honourable Mention at #SIGIR2025! Very grateful to the community for recognizing our work on improving RAG evaluation.

 📄 webis.de/publications...

Now @fschlatt.bsky.social presents "TITE: Token-Independent Text Encoder for Information Retrieval" at #SIGIR2025

Paper: webis.de/publications...

Want to know how to make bi-encoders more than 3x faster with a new backbone encoder model? Check out our talk on the Token-Independent Text Encoder (TITE) #SIGIR2025 in the efficiency track. It pools vectors within the model to improve efficiency dl.acm.org/doi/10.1145/...

To Eun Kim just presented the work on "Tip of the Tongue Query Elicitation for Simulated Evaluation" at #SIGIR2025. The approach will be used in the #TREC2025 Tip-of-the-Tongue track, and we had some sweets at the poster :)

The paper is available online: dl.acm.org/doi/10.1145/...

Lukas Gienapp presents "The Viability of Crowdsourcing for RAG Evaluation" at #SIGIR2025

The paper is available at: webis.de/publications...

Variations in Relevance Judgments and the Shelf Life of Test Collections The fundamental property of Cranfield-style evaluations, that system rankings are stable even when assessors disagree on individual relevance decisions, was validated on traditional test collections. ...

@mrparryparry.bsky.social presenting our work on reproducing TREC DL 2019 judgements and the implications for evaluating modern ranking models on modern collections. Paper: arxiv.org/abs/2502.20937

Here are some of the statistics that I found very interesting from the #SIGIR2025 opening session. (Over 1000 attendees!)

Hello TREC-ToTers!

We have released the test queries for the TREC 2025 Tip-of-the-Tongue (TREC-ToT) Track. Please see the guidelines for more information: trec-tot.github.io/guidelines. Run submission deadline will tentatively be in August. #TREC2025 #TRECToT #TREC2025ToT

Please spread the word!

Thank you Carlos for the shout-out of Lightning IR in the LSR tutorial at #SIGIR2025

If you want to fine your own LSR models, check out our framework at github.com/webis-de/lig...

Djoerd Hiemstra proudly holding a copy of the first issue

Never seen our editor in chief, Djoerd Hiemstra, more happy than today, holding a copy of the first issue of #IRRJ

Dory from finding nemo with the quote: "I remember it like it was yesterday. Of course, I dont remember yesterday."

Do not forget to participate in the #TREC2025 Tip-of-the-Tongue (ToT) Track :)

The corpus and baselines (with run files) are now available and easily accessible via the ir_datasets API and the HuggingFace Datasets API.

More details are available at: trec-tot.github.io/guidelines

Cool, congratulations! 🥳🥳

The deadline for submissions to the ReNeuIR workshop at #SIGIR2025 is extended to June 10 😸

Details: reneuir.org

#ReNeuIr2025 #SIGIR25

Hello TREC-ToTers! 👋🏽

Excited to announce the release of TREC 2025 Tip-of-the-Tongue (TREC-ToT) Track guidelines: trec-tot.github.io/guidelines. We will release test queries in July and run submission deadline will be in August. #TREC2025 #TRECToT #TREC2025ToT

Please register to participate:

Today I had the pleasure to talk about child-safe search at #ECIR2025. We created an cranfield-style evaluation dataset to contrast relevance with harm in web search scenarios.

Details: webis.de/publications...

The Workshop on Open Web Search just finished #WOWS2025 #ECIR2025.

It was a very cool experience with many interesting talks. Lets hope we can do it again next year at #ECIR2026 in Delft :)

The Workshop on Open Web Search at #ECIR2025 just starts with a keynote by @claclarke.bsky.social on Annotative Indexing. #WOWS25 #WOWS2025 #ECIR25

Honored to receive the best short paper award and best paper honourable mention award at #ECIR2025. Thank you to all co-authors @maik-froebe.bsky.social, @hscells.bsky.social, Shengyao Zhuang, @bevankoopman.bsky.social, Guido Zuccon, Benno Stein, @martin-potthast.com, @matthias-hagen.bsky.social 🥳

Now we have @fschlatt.bsky.social on the #ECIR2025 stage predenting the research on the Set-Encoder.

The paper is online at: webis.de/publications...

An cool example of the Circle of Research at #ECIR2025 (or was it the Circle of Life?):

Yesterday, I saw Ronak presenting research to Ferdi, today it was the other way around :)

I was very happy to talk about corpus subsampling at #ECIR2025 today.

Please find the paper at webis.de/publications...

And lat bur not least, here are some of my favorite impressions of the first day of ECIR :)