π’Thrilled to introduce ATLAS πΊοΈ: the largest multilingual scaling study to-dateβwe ran 774 exps (10M-8B params, 400+ languages) to answer:
π Is scaling diff by lang?
π§ββοΈ Can we model the curse of multilinguality?
βοΈ Pretrain vs finetune from checkpoint?
π X-lingual transfer scores across langs?
1/π§΅
28.10.2025 14:01 β π 17 π 1 π¬ 1 π 1
Matformer introduces nested structure into the Transformer's FFN block & jointly trains all the submodels, enabling free extraction of hundred of accurate submodels for elastic inference
I will be at poster #2507 w/ my co-authors in East Exhibit Hall A-C at #NeurIPS2024 chatting about MatFormer and elastic models today at 4.30pm!
Come by, or reach out if you want to chat about pretraining, scaling laws or conditional computation!
arxiv.org/abs/2310.07707
11.12.2024 21:42 β π 8 π 0 π¬ 0 π 0
Would love to be added!
11.12.2024 17:23 β π 3 π 0 π¬ 1 π 0
TUSL discord link: https://discord.gg/z3ya9EUS2U
PhD Candidate, University of Washington
https://salonidash.com/
Anti-cynic. Towards a weirder future. Reinforcement Learning, Autonomous Vehicles, transportation systems, the works. Asst. Prof at NYU
https://emerge-lab.github.io
https://www.admonymous.co/eugenevinitsky
compling phd student @ boulder
rare languages, morphology, finite state automata
michaelginn.com
First-year NLP PhD @ USC | Intern @ TogetherAI | Prev. UW, AWS
https://nanami18.github.io/
Research Scientist at @ibmresearch #NLProc, #RL.
Opinions are my own.
PhDing @AIM_Harvard @MassGenBrighamο½PhD Fellow @Google | Previously @Bos_CHIP @BrandeisU
More robustness and explainabilities π§ for Health AI.
shanchen.dev
Assistant Professor in CS + AI at USC. Previously at Stanford, CMU. Machine Learning, Decision Making, AI-for-Science, Generative AI, ML Systems, LLMs.
https://willieneis.github.io
Assoc. Prof. in Linguistics at CU Boulder (@bouldernlp.bsky.social). My group researches computational methods for endangered and low-resource languages, plus computational discourse and semantics: @lecslab.bsky.social. I'm a musician and have cats!
NLP PhD from CU Boulder.
prev: Apple, ETS, Pearson, Army Research Lab.
Next: Kensho
https://adamits.github.io
NLP/Computational Linguistics PhD @lecslab.bsky.social and @bouldernlp.bsky.social
typologically robust multilingual NLP
technology for language documentation
https://covetedfish.github.io/
π₯ LLMs together (co-created model merging, BabyLM, textArena.ai)
π₯ Spreading science over hype in #ML & #NLP
Proud shareLM㪠Donor
@IBMResearch & @MIT_CSAIL
Professor at Cardiff University (Cardiff NLP). Natural Language Processing researcher. Computational Social Science. Sometimes chess.
PhD student @CMU LTI
NLP | IR | Evaluation | RAG
https://kimdanny.github.io
Into creative ML/AI, NLP, data science and digital humanities, narrative, infovis, games, sf & f. Consultant, ds in Residence at Google Arts & Culture. (Lyon, FR) Newsletter arnicas.substack.com.
Assistant professor @ UIUC, studying personalized language and communication
Researching planning, reasoning, and RL in LLMs @ Reflection AI. Previously: Google DeepMind, UC Berkeley, MIT. I post about: AI π€, flowers π·, parenting πΆ, public transit π. She/her.
http://www.jesshamrick.com
NLP research - PhD student at UW
I (try to) do NLP research. Antipodean abroad.
currently doing PhD @uwcse,
prev @usyd @ai2
π¦πΊπ¨π¦π¬π§
ivison.id.au
