FBK - NLP Research Group

🚀 Can Vision-Language Models plan effectively?

We introduce ViPlan, a benchmark comparing:
🔹 VLM-as-planner
🔹 VLM-as-grounder

🏠 Home robotics
🧱 Visual Blocksworld

Spoiler alert:❌ Visual-Reasoning

📍 Pre-print here: arxiv.org/abs/2505.13180

#VLM #AI #NLProc

21.05.2025 08:44 — 👍 0    🔁 0    💬 0    📌 0

Evalita Llm Leaderboard - a Hugging Face Space by evalitahf Duplicate this leaderboard to initialize your own!

🚀 Exciting News! The Evalita-LLM Leaderboard is now live on Hugging Face! Explore the performance of over 40 Large Language Models on native Italian tasks. Dive in here: huggingface.co/spaces/evali...

@FondazioneBrunoKessler
@igenius
@diunito.bsky.social

#NLProc #LLM #AI #Italian #Benchmarking

08.04.2025 11:08 — 👍 0    🔁 2    💬 0    📌 0

Don't miss out ‼️ Join us at Wired Health 2025 #WH25💭 Our Head Unit, Bernardo Magnini will take the stage to discuss "Artificial Intelligence and Clinical Data: The Future of Emergency Medicine." 🔥
Full program here: lnkd.in/eKi44biT
@wired.com

#AI #healthcare #Innovation

13.03.2025 10:29 — 👍 0    🔁 0    💬 0    📌 0

Collab with Cimec (UniTrento)- @raffagbernardi.bsky.social , ItalianNLP_Lab - @alessiomiaschi.bsky.social and UniPisa (Alessandro Lenci, Lucia Passaro and Alessandro Bondielli)

27.02.2025 11:28 — 👍 3    🔁 0    💬 0    📌 1

All-in-one: Understanding and Generation in Multimodal Reasoning with the MAIA Benchmark We introduce MAIA (Multimodal AI Assessment), a native-Italian benchmark designed for fine-grained investigation of the reasoning abilities of visual language models on videos. MAIA differs from other...

Welcome to MAIA! 🚀 Our new benchmark for evaluating multimodal reasoning in Vision-LMs on videos fully in Italian! 🇮🇹 MAIA tests understanding & generation with fine-grained reasoning categories and a brand-new evaluation metric! 🔎🔥Discover MAIA here: arxiv.org/abs/2502.16989
#NLProc #evaluation #AI

27.02.2025 09:34 — 👍 6    🔁 2    💬 1    📌 1

Thank you for supporting this project! 💪🏻

25.02.2025 15:18 — 👍 0    🔁 0    💬 0    📌 0

Evalita-LLM: Benchmarking Large Language Models on Italian We describe Evalita-LLM, a new benchmark designed to evaluate Large Language Models (LLMs) on Italian tasks. The distinguishing and innovative features of Evalita-LLM are the following: (i) all tasks ...

🚀 **Exciting News!** 🎉 Evalita-LLM is here! 🇮🇹 A new benchmark for evaluating LLMs—offering native Italian tasks, generative challenges, and fair multi-prompt evaluations. Now also available in lm-evaluation harness by @eleutherai.bsky.social !
ArXiv: arxiv.org/abs/2502.02289
#NLProc #LLM #Evaluation

24.02.2025 17:07 — 👍 9    🔁 3    💬 0    📌 1

Our group leader took the stage at the FBK plenary session to showcase our research interests, ongoing projects, challenges and future plans. An exciting moment to share our vision and push the boundaries of NLP even further!
Here’s a glimpse of the event! 📸✨
#NLProc #AI #Research #FBK #Innovation

07.02.2025 14:35 — 👍 2    🔁 1    💬 0    📌 0

A big congratulations to Sofia Lugli, student of our Carlo Strapparava, for receiving a special mention for her thesis at
CLiC-it conference 2 weeks ago! 🎉👏
We are incredibly proud of her achievement. Can’t wait to see what she accomplishes next. Well done Sofia!🌟
#NLP #AI @ailc-nlp.bsky.social

20.12.2024 09:44 — 👍 1    🔁 1    💬 0    📌 0

🚀 #CALAMITA has officially kicked off - and we’re on board!
Fondazione Bruno Kessler proudly participated in this 1st #evaluation #campaign for #LLMs with our Andrea Zaninello together with @fbk-mt.bsky.social group!🤖
Here some pictures of our calamitici!🧲
See u next year 4 other challenges! 🔥🔍

19.12.2024 13:42 — 👍 6    🔁 3    💬 0    📌 0

Data-LLM-Tutorial You Are what You Eat Processing Data for Training and Evaluating LLMs Giovanni Bonetta and Bernardo Magnini Fondazione Bruno Kessler, Trento, Italy {gbonetta|magnini}@fbk.eu Tutorial at CLiC-it 2024, ...

Exciting news! 🎉 If you’re curious about the opening #tutorial at CLiC-it 2024 conference made by our group on processing #data for #training and #evaluating #LLMs , here’s your chance! 📊 Explore the slides and get inspired!!! 🚀🔍✨
docs.google.com/presentation...
@ailc-nlp.bsky.social

17.12.2024 16:15 — 👍 3    🔁 2    💬 0    📌 1

too sweet darling!!! ❤️

17.12.2024 12:08 — 👍 0    🔁 0    💬 0    📌 0

Here are our group leader Bernardo Magnini and postdoc
Giovanni Bonetta kicking off CLiC-it conference in #Pisa 2 weeks ago as invited speakers with an insightful #tutorial on #data: how to collect and use it for various purposes in #AI and #NLP !💡🔍
Thanks to @ailc-nlp.bsky.social for inviting us!✨

17.12.2024 10:37 — 👍 4    🔁 0    💬 0    📌 0

#NLP Research group in action! Our group leader Bernardo Magnini, alongside @tizaino.bsky.social, Sofia Brenna, and Giovanni Bonetta, presenting their #paper "Are you a Good Assistant? Assessing LLM Trustability in Task-oriented Dialogues" at CLiC-it '24 #Pisa 🔍✨
@ailc-nlp.bsky.social
#NLProc #AI

16.12.2024 09:05 — 👍 7    🔁 2    💬 0    📌 1

What a conference! 🎉 CLiC-it in #Pisa was truly special! Returning to where it all began a decade ago! ❤️
Our group was honored to participate, share ideas and connect with such an inspiring community.
Here are some highlights from the conference!!!📸✨
#CLiCit2024 #NLP #AI @ailc-nlp.bsky.social

13.12.2024 08:26 — 👍 6    🔁 2    💬 0    📌 0

🎄✨ Happy Holidays from our Research Center! ✨🎄

Our director @ferruccioresta shares a warm holiday message with all @FBK_research: gratitude for a year of outstanding achievements and best wishes for a joyful festive season and a 2025 filled with groundbreaking discoveries!

11.12.2024 23:25 — 👍 6    🔁 0    💬 0    📌 0