GITT 2025

And that's a wrap on a very hot 🥵 yet very rewarding #GITT2025
💭 We need more languages, better quality estimation, and intersectionality
😍 Thanks to everyone who was there, presented, contributed to the discussion, and helped with practicalities.
🌞 Enjoy the rest of @mtsummit2025.bsky.social!

Results show GPT4-o and Qwen 72B outperforming the baseline classifier and improved accuracy for intermediate reasoning steps. More details in the paper - have a read! 👀 #GITT2025

Experiments using mGeNTE as reference arxiv.org/abs/2501.09409

RQs: can LLMs identify neutral versus gendered translation? How do we improve the accuracy? Tests on 3 language pairs (English into Italian, Spanish, and German) on sentence and phrase level in monolingual and crosslingual scenarios

Work on LLM-as-judge suggests this could be useful for GNT evaluation as well

As already raised by our keynote, quality estimation is a great industry need as there is often no time for human evaluation in production, this is particularly tricky for GNT

Not one single solution for gender neutral translation (GNT) (tradeoff adequacy/fluency?) making GNT a complex evaluation task

Last but definitely not least: @bsavoldi.bsky.social presenting joint work with @apierg.bsky.social @matteo-negri.bsky.social @luisabentivogli.bsky.social on scalable gender neutral translation evaluation using LLM-as-a-judge at #GITT2025

Poster session now happening at #GITT2025 Some really exciting research and potluck discussions happening! 🔥

Summary for inclusive AI: representation, transparency, community input, iteration, respect #GITT2025

What should the industry do to be more inclusive? Rely on NB and inclusive language experts, connect with the community, stay updated, train the people (this can be you!), test and iterate

Gender tags can be used in training! Postprocessing can help in time sensitive situations, but can be dangerous too, not always useful

Some attempted strategies: style indication alone not enough, few shot and style guide additions helped

What about prompting? Industry requires everything to work within the pipeline, within the system. You can't have prompts leading to different outputs, it cannot be wrong, it cannot contain hallucinations

Microsoft custom translator was finetuned with +/- 5k-10k segments. Because of the specificity of the problem, limited data is enough for improvements. "overfitting in my favor"

Perfect data doesn't exist, but data was created with support of experts and generation. Opposite strategies from what you'd want for non-inclusive datasets! Multiple solutions suddenly useful and even necessary

Great crowd participation! 👌🔥 Training, teaching, finetuning, data, prompting and more as potential solutions

Share your thoughts! #GITT2025 how do we shape AI for inclusive language?

Issues for MT: lack of training data (historical data was not inclusive, no representation), association bias, post-editing always necessary, publishing MT output as is can be problematic, tricky for gendered languages, evaluation metrics used in the industry, but might not work for inclusivity

Important to consult expert matters and the community. Some specific examples: having the inclusive schwa character on the keyboard for Italian, explicit representation of nonbinary characters, e.g. Dragon Age the Veilguard

In big corporations, teams had to be hired specifically for inclusivity, output had to be tested, guidelines needed to be written and constantly updated with rapidly changing language and character representation evolved (games = community, people need to find themselves)

Inclusivity has become increasingly important in the industry, especially for younger generations. The path towards inclusive language has a few different steps and inclusivity goes beyond just gender > echoing the idea of intersectionality from the opening notes!

Good keynotes use inspirations quotes from others. Great keynotes use inspirational quotes from themselves 👌

Already working on the industry when SMT was the default, not the easiest to use for localisation back then, all the way to NMT and LLM today. "video games today are like a work of art" > not necessarily the easiest match for MT #GITT2025

🔑🎵 Our keynote speaker Cristina Anselmi sharing localisation industry perspectives on optimising AI for inclusivity 🤩

Evaluation clearly still shows issues and how challenging this is in real world scenarios. Hillary and colleagues are open to collaborate with anyone willing to help tackle this issue in future work 🤝

Some response to actively ambiguous sentences, but no consistency across languages and systems.

Baseline: neutral translation when gender is known shows variance within languages across systems. Ambiguous scenarios show strong default masculine translation, no increase in neutral translations

Subset of this test suite specifically selected for gender neutral / ambiguous scenarios, 3 languages (Spanish, Czech, Icelandic) with different strategies