Alberto Parola

Associate/Assistant Professor: Language and Speech Technology | Radboud University Do you want to work as a Associate/Assistant Professor: Language and Speech Technology at the Faculty of Arts? Check our vacancy!

Come work with us! Our department is hiring an associate/assistant prof in language and speech technology www.ru.nl/en/working-a...
#interspeech #speech #SpeechTech #SpeechScience

Are there cross-linguistic vocal characteristics of schizophrenia? We test current machine learning approaches and show that they do generalize across languages *not even when being trained cross-linguistically*. Excellent thread by @alpar.bsky.social w some ways forward.

Vocal markers of schizophrenia: assessing the generalizability of machine learning models and their clinical applicability Background and Hypothesis Machine Learning (ML) models have been argued to reliably predict diagnosis and symptoms of schizophrenia based on voice data only. However, it is unclear to what extent such...

New pre-print with @fusaroli.bsky.social on voice markers of schizophrenia out:
www.medrxiv.org/content/10.1...
Thread below 📖
TL;DR: Cross-linguistic generalizability of vocal markers of SCZ is challenging, we need more collaborative efforts and large multi-center and cross-linguistic projects

8/8 🛠️ How can we improve generalization?
•Larger, open datasets capturing linguistic, clinical, and demogr. variability in SCZ to test generalization and modern ML architectures, e.g., LLMs, multimodal models.
•Focusing on fine-grained clinically relevant features to enhance clinical applicability.

7/8 🌍 Why does generalization fail?
• Linguistic differences affect how SCZ symptoms relate to acoustic features
• Clinical heterogeneity limits robustness of ML models trained on small, homogenous samples
• Models biased toward general features, not capturing diagnosis- or symptom-specific markers

6/8 📢 Key Finding #3:
We tested two alternative approach:
1️) Mixture of Experts models (combining predictions from models trained on different languages, Plot 3).
2) Multi-language training set (combin. training data from multiple languages, Plot 4).
❌ Results: Still near chance level (F1 ~ 0.50).

5/8 🚨 Key Finding
✔️#1: ML models perform when trained/tested on the same language (F1 ~ 0.75) (Plot1)
❌#2: But when trained/tested on different languages (e.g., Danish → Chinese), performance drops significantly (F1 ~ 0.50) (Plot 2).
Cross-linguistic generalizability remains a key challenge!

4/8💡What’s the goal?
In this study we build a large cross-linguistic speech corpus (Danish, German, Chinese) of patients with schizophrenia and controls to systematically test whether voice-based ML models predicting schizophrenia generalize across different languages, samples and context: 🧵

Voice Patterns as Markers of Schizophrenia: Building a Cumulative Generalizable Approach Via a Cross-Linguistic and Meta-analysis Based Investigation AbstractBackground and Hypothesis. Voice atypicalities are potential markers of clinical features of schizophrenia (eg, negative symptoms). A recent meta-a

3/8 In prior meta-analysis and experim. work (below), we showed that speech marker generalizability might be challenging. The assumption that SCZ speech markers manifest uniformly across heterogeneous samples and contexts must be systematically tested: doi.org/10.1093/schb... doi.org/10.1016/j.sc...

2/8💡Key question ❓
But how well do voice-based machine-learning models generalize across languages and cultural contexts? How well do they generalize across samples with heterogenous clinical features? Are they robust enough to biases for clinical applicability?

1/8 Schizophrenia and machine-learning-based speech markers
🎙️ Schizophrenia is associated with atypical voice patterns, making voice a promising candidate biomarker. Voice-based ML models can indeed predict diagnosis, symptoms and track socio-cognitive and motor features of SCZ with high accuracy.

Vocal markers of schizophrenia: assessing the generalizability of machine learning models and their clinical applicability Background and Hypothesis Machine Learning (ML) models have been argued to reliably predict diagnosis and symptoms of schizophrenia based on voice data only. However, it is unclear to what extent such...

New pre-print with @fusaroli.bsky.social on voice markers of schizophrenia out:
www.medrxiv.org/content/10.1...
Thread below 📖
TL;DR: Cross-linguistic generalizability of vocal markers of SCZ is challenging, we need more collaborative efforts and large multi-center and cross-linguistic projects

For more work in this line of research:
- do markers of schizophrenia and its symptoms generalize across languages? (voice: doi.org/10.1093/schb... text: doi.org/10.1016/j.sc...; led by
A. Parola) 1/