Our new work on continuous chain of thought.
10.12.2024 16:51 β π 4 π 0 π¬ 0 π 0
Analysis: AD picks high temp for creative & low for fact-seeking prompts, automatically via training.
Our methods AD & Latent Pref Optimization are general & can be applied to train other hyperparams or latent features.
Excited how people could *adapt* this research!
π§΅4/4
22.11.2024 13:06 β π 2 π 0 π¬ 0 π 0
We train on a mix of tasks:
GSM8K - requires factuality (low temp)
Stories - requires creativity (high temp)
UltraFeedback - general instruction following, requires mix
Results: Adaptive Decoding outperforms any fixed temperature, automatically choosing via the AD layer.
π§΅3/4
22.11.2024 13:06 β π 2 π 0 π¬ 2 π 0
Recipe π©βπ³:
Adaptive Decoder (AD) Layer:
- Assigns probability to each hyperparam choice (decoding temp) given hidden state. Given temp, sample a token.
Training (Latent PO):
- Train AD by sampling params+tokens & use reward model on rejected hyperparam preference pairs
π§΅2/4
22.11.2024 13:06 β π 1 π 0 π¬ 1 π 0
π¨ Adaptive Decoding via Latent Preference Optimization π¨
- New layer for Transformer, selects decoding params automatically *per token*
- Learnt via new method Latent Preference Optimization
- Outperforms any fixed temperature decoding, choosing creativity or factuality
arxiv.org/abs/2411.09661
π§΅1/4
22.11.2024 13:06 β π 43 π 6 π¬ 2 π 0
PhD at 19 |
Founder and CEO at @MedARC_AI |
Research Director at @StabilityAI |
@kaggle Notebooks GM |
Biomed. engineer @ 14 |
TEDx talkβ‘https://bit.ly/3tpAuan
a mediocre combination of a mediocre AI scientist, a mediocre physicist, a mediocre chemist, a mediocre manager and a mediocre professor.
see more at https://kyunghyuncho.me/
It's all about astrophotography.
Associate Professor @ Utrecht University, NLP & Computational Linguistics.
ELLIS Member. Utrecht Young Academy Board Member. CUCo Board Member.
Natural Language Processing @ NLTP nlp.sites.uu.nl π±πΊ
Ph.D. Student at UNC NLP | Prev: Apple, Amazon, Adobe (Intern) vaidehi99.github.io | Undergrad @IITBombay
I do research related to LLMs , their interaction with geospatial data and leveraging them for information extraction . PhD in computer Science at George Mason University.
PhD student @unc @unccs @uncnlp; Formerly Intern @AmazonScience @MSFTResearch @NlpWestlake. RT & like β endorsements. Views are my own. He/him
hannight.github.io
PhD student at KIT in Germany doing research on language models interacting with structured information.
Leader of Conversational Systems Team at the Center for Artificial Intelligence at Adam Mickiewicz University, PoznaΕ. Assistant Professor in the Department of Artificial Intelligence. https://marekkubis.com #AI #NLProc
βοΈ Assistant Professor of Computer Science at CU Boulder
π©βπ» NLP, cultural analytics
π https://maria-antoniak.github.io
Previously: Pioneer Centre for AI in Copenhagen, Ai2, Microsoft Research, Twitter, Facebook, Cornell, UW
Book: https://thecon.ai
Web: https://faculty.washington.edu/ebender
Data janitor and leftover linguist (retired). Tsundoku expert. Language & Cognition. NLP. Japanese literature. Anti-authoritarian. Pro-science.
Stanford Linguistics and Computer Science. Director, Stanford AI Lab. Founder of @stanfordnlp.bsky.social . #NLP https://nlp.stanford.edu/~manning/
Researcher trying to shape AI towards positive outcomes. ML & Ethics +birds. Generally trying to do the right thing. TIME 100 | TED speaker | Senate testimony provider | Navigating public life as a recluse.
Former: Google, Microsoft; Current: Hugging Face
Associate Professor, School of Information, UC Berkeley. NLP, computational social science, digital humanities.
Associate professor of computer science at Northeastern University. Natural language processing, digital humanities, OCR, computational bibliography, and computational social sciences. Artificial intelligence is an archival science.
Associate prof at @UMich in SI and CSE working in computational social science and natural language processing. PI of the Blablablab blablablab.si.umich.edu
He teaches information science at Cornell. http://mimno.infosci.cornell.edu
I like tokens! Lead for OLMo data at @ai2.bsky.social (Dolma π) w @kylelo.bsky.social. Open source is fun π€βοΈππ³οΈβπ Opinions are sampled from my own stochastic parrot
more at https://soldaini.net
#nlp #ml #hci research scientist @ai2.bsky.social, Co-lead of Data for OLMo w/ @soldaini.net, statistics @uw, open science, tabletop, seattle, he/him,π§ kyleclo.com
