Anirvan Sengupta

Asymptotic theory of in-context learning by linear attention | PNAS Transformers have a remarkable ability to learn and execute tasks based on examples provided within the input itself, without explicit prior traini...

Great to see this one finally out in PNAS! Asymptotic theory of in-context learning by linear attention www.pnas.org/doi/10.1073/... Many thanks to my amazing co-authors Yue Lu, Mary Letey, Jacob Zavatone-Veth @jzv.bsky.social and Anindita Maiti!

In-context denoising with one-layer transformers: connections between attention and associative memory retrieval We introduce in-context denoising, a task that refines the connection between attention-based architectures and dense associative memory (DAM) networks, also known as modern Hopfield networks. Using a...

At #ICML2025, presenting work done at @flatironinstitute.org w Matt Smart and @albertobietti.bsky.social on in-context denoising (arxiv.org/abs/2502.05164). Come to Matt’s oral, Thursday, 4:15-4:30 PM, West Ballroom A, and see us right after at poster #E-3207, 4:30-7:00 PM, East Exhibition Hall A-B.

Come hear Matt Smart's talk about in-context denoising with transformers at the Associative memory workshop #ICLR25, 2:15pm! This task refines the connection between transformers and associative memories. w/ M Smart and @albertobietti.bsky.social at @flatironinstitute.org

The official ad for this postdoc position is now (finally) live, see academicpositions.harvard.edu/postings/14486!

Excited to share our work on ‘Deep Learning Based Superconductivity Prediction and Experimental Tests’ at the #NeurIPS2024 workshop on Machine Learning and Physical Sciences

ml4physicalsciences.github.io/2024/files/N...