jgyou

In this technical note, we study the problem of inverse permutation learning in decoder-only transformers. Given a permutation and a string to which that permutation has been applied, the model is tasked with producing the original ("canonical") string. We argue that this task models a natural robustness property across a variety of reasoning tasks, including long-context retrieval, multiple choice QA and in-context learning. Our primary contribution is an impossibility result: we show that an arbitrary depth, decoder-only transformer cannot learn this task. This result concerns the expressive capacity of decoder-only transformer models and is agnostic to training dynamics or sample complexity. We give a pair of alternative constructions under which inverse permutation learning is feasible. The first of these highlights the fundamental role of the causal attention mask, and reveals a gap between the expressivity of encoder-decoder transformers and the more popular decoder-only architecture. The latter result is more surprising: we show that simply padding the input with "scratch tokens" yields a construction under which inverse permutation learning is possible. We conjecture that this may suggest an alternative mechanism by which chain-of-thought prompting or, more generally, intermediate "thinking" tokens can enable reasoning in large language models, even when these tokens encode no meaningful semantic information (e.g., the results of intermediate computations).

Implementing “reasoning” as “just keep talking to yourself” feels so weird & hacky.

But maybe not, according to this? They show that just-predicting-the-next-word is inherently unable to learn to solve some (permutation) problems unless you “pad” with extra thought tokens. arxiv.org/abs/2509.24125

Message passing for epidemiological interventions on networks with loops
https://arxiv.org/abs/2509.21596

Insights from Network Science can advance Deep Graph Learning Deep graph learning and network science both analyze graphs but approach similar problems from different perspectives. Whereas network science focuses on models and measures that reveal the organizati...

Had missed this paper!
It's great
arxiv.org/abs/2502.01177

Are you studying animal sociality? Confused about the "non-independence" of social network data: what on earth are they? are they threatening your results? should you attempt to get rid of them?

A thread summarising a part of our new paper (ecoevorxiv.org/repository/v...).

🎓 I've joined the PhD in Complex Systems & Data Science at @vcsi.bsky.social to work w/ @lhd.bsky.social & @juniperlov.bsky.social to explore social (complex) dynamics

🔬 Huge thanks to @joanagsa.bsky.social & Lília Perfeito for helping me get ready for this chapter

🚀 Excited for this next step!

🐒🕸️ New preprint! Confused about how to model animal social networks?

ASNA can be confusing—but also full of opportunity. We break down 5 common misunderstandings in animal social network analysis and share solutions from behavioural ecology, anthro, stats, & network science. Hope it helps!

A 🧵

All from an amazing interdisciplinary group 👥: @benkawam.bsky.social 🌟, @jgyou.bsky.social, Dan Franks, Connor Philson, Marijtje van Duijn, @jordanhart96.bsky.social, MB McElreath, @rmcelreath.bsky.social, @eapower.bsky.social, Cody Ross, @steglich.bsky.social, & @ljnbrent.bsky.social!

Leto Peel officially opens #NetSci2025, followed by Sonia Kefi's keynote

And the code is live here: github.com/DynamicaLab/...

With @allard.bsky.social and the never-online Simon Lizotte (but you can find him at siliz4.github.io)

Hear me discuss our new paper written with the inimitable Laurent Hébert-Dufresne and colleagues! @lhd.bsky.social #Complexity #Physiology #Governance www.nature.com/articles/s44...

On this day in 2023, the Vermont Complex Systems Institute (vermontcomplexsystems.org), TGIR Center (www.med.uvm.edu/tgircobre/home) & @sfiscience.bsky.social joined for an event on interacting contagions

Our work, "One pathogen does not an epidemic make," is now online: arxiv.org/abs/2504.15053

So glad to have @jgyou.bsky.social give an intriguing lecture on contagion modeling and network inference in our network science class!

I've been guest editing a focus issue for Journal of Physics: Complexity with @jgyou.bsky.social, @gcant.bsky.social, and Cate De Bacco on Statistical Inference and Machine Learning for Complex Networks. See iopscience.iop.org/collections/..., and feel free to get in touch with any questions!

You should find that haircut again.

Hate The Game, the book, is only one month away! Pre-order yours anywhere books are sold. 📖✨

Freakonomics coauthor Steven Levitt called it “Fabulous—unlike any economics book I’ve ever read!

We're excited to share our MAR10 day discounts!
- $1 off of Mario's Time Machine
- A free wallpaper

Downscaling mutualistic networks from species to individuals reveals consistent interaction niches and roles within plant populations Species-level networks emerge as the combination of interactions spanning multiple individuals, and their study has received considerable attention over the past 30 y. However, less is known about the structure of interaction configurations within species, even though individuals are the actual interacting units in nature. We compiled 46 empirical, individual-based, interaction networks on plant-animal seed dispersal mutualisms, comprising 1,037 plant individuals across 29 species from various regions. We compared the structure of individual-based networks to that of species-based networks and, by extending the niche concept to interaction assemblages, we explored individual plant specialization. Using a Bayesian framework to account for uncertainty derived from sampling, we examined how plant individuals “explore” the interaction niche of their populations. Both individual-based and species-based networks exhibited high variability in network properties, lacking remarkable structural and topological differences between them. Within populations, frugivores’ interaction allocation among plant individuals was highly heterogeneous, with one to three frugivore species dominating interactions. Regardless of species or bioregion, plant individuals displayed a variety of interaction profiles across populations, with a consistently-small percentage of individuals playing a central role and exhibiting high diversity in their interaction assemblage. Plant populations showed variable mid to low levels of niche specialization; and individuals’ interaction niche “breadth” accounted for 70% of the population interaction diversity, on average. Our results highlight how downscaling from species to individual-based networks helps understanding the structuring of interactions within ecological communities.

1/ New paper @pnas.org on the structure of mutualistic #networks between individuals plants and frugivore species.

Last chapter of @elequintero.bsky.social's PhD thesis

doi.org/10.1073/pnas... #ecopubs

Pandemic monitoring with global aircraft-based wastewater surveillance networks - Nature Medicine By simulating the implementation of airport-based wastewater surveillance sites at the global level, a modeling study shows how this early warning system would perform in identifying sources of pandem...

Wastewater from airplane toilets?
We introduce a global Aircraft-Based Wastewater Surveillance Network (WWSN) for pandemic monitoring in Nature Medicine 🔗 doi.org/10.1038/s415...

Aircraft-based wastewater surveillance allows for real-time, non-invasive monitoring of global pathogen spread
Short 🧵

On a similar note I got spoiled by Dr Watson in Julia. Is there a good Python equivalent?

We are reaching the time of year when I start hearing from undergrads who are looking for summer internships. Obviously, they should all apply to Pew, but what other research orgs have internship programs that I can tell them about? What other opportunities can I share with them?

24h left to apply to the REU experience at UVM!

Harnessing the Data Revolution for Vermont: The Science of Online Corpora, Knowledge, and Stories (SOCKS)

socks.w3.uvm.edu/socks/node/65

Comparison may be a thief of joy, but it is an undoubtably interesting statistical modeling problem. In my latent chapter I go deep into the general problem of modeling comparisons between pairs of abstract items.

HTML: betanalpha.github.io/assets/chapt...
PDF: betanalpha.github.io/assets/chapt...

ACE Lab: Applied Cultural Evolution Laboratory culture, evolution, cooperation, and sustainability

Fully-funded PhD position on the cultural evolution of environmental management!

Link evolution, ecology and behavioral science to study how people manage environmental challenges in the ACE Lab (timwaring.info).

Position Details: timwaring.info/wp-content/u...

I preemptively declare paper bankruptcy. Anything published on arXiv during the next 3 weeks will be permanently memory black holed.

Fall retreat in Vermont where the project was started. Panorama with a field and hills in the distance.

A project with friends from the Vermont Complex Systems Institute @juniperlov.bsky.social @jstonge.bsky.social @nwlandry.bsky.social @jgyou.bsky.social

Funded by an ambitious AUDACE project from friends and co-authors from Université de Sherbrooke.

Started in the Green Mountains two years ago.

Map shows the location of wastewater sampling locations around the US. Most of the sites indicate low levels and are colored blue. However, there are a number of sites in CA with high levels and few other scattered sites around the US with medium/high levels and are colored in orange/red

1/ Influenza wastewater activity levels are currently low across the US, aside from conspicuous areas in CA where #H5N1 is spreading in dairy cows.

A deeper dive into the Wastewater Scan data paints a very concerning picture and should add to our sense of urgency.

what idiot called it async/await instead of wait/what

Gaussian approximation of a target distribution: mean-field versus full-covariance! Below shows a simple gradient descent on KL(q||p)

Having a sides-only Thanksgiving this year. No turkey, just carbs and gravy. Modern problems require modern solutions.

👋