David Smith

Symmetry in language statistics shapes the geometry of model representations Although learned representations underlie neural networks' success, their fundamental properties remain poorly understood. A striking example is the emergence of simple geometric structures in LLM rep...

In our new preprint, we explain how some salient features of representational geometry in language modeling originate from a single principle - translation symmetry in the statistics of data.

arxiv.org/abs/2602.150...

With Dhruva Karkada, Daniel Korchinski, Andres Nava, & Matthieu Wyart.

YouTube video by Google TechTalks Privacy Ripple Effects from Adding or Removing Personal Information in Language Model Training

I gave a talk at the Google Privacy in ML Seminar last summer on privacy & memorization: "Privacy Ripple Effects from Adding or Removing Personal Information in Language Model Training".

It's up on YouTube now if you're interested :)
youtu.be/IzIsHFCqXGo?...

We also show that we are far from done, specifically for a complicated language like Old French.

But we
(1) defined the issue,
(2) propose a first solution that enables pre-annotation of larger dataset and
(3) offer an alternative to less trustable models that go beyond ATR.

Pre Editorial Normalization - a Hugging Face Space by comma-project Latin and Old French normalization of CATMuS output

We release:

📚 4.66M silver training samples
🧪 1.8k gold evaluation set huggingface.co/datasets/com...
🤖 ByT5-based model → 6.7% CER huggingface.co/comma-projec...

Try it here 👇
huggingface.co/spaces/comma...

👉 We propose Pre-Editorial Normalization (PEN):

An intermediate layer between:
📝 graphemic ATR output
📖 fully edited text

Goal: preserve palaeographic fidelity + enable usability.
Keep two layer, ATR output and normalization, with aligned token to go back to the source.

Recent ATR progress—especially with palaeographic datasets like CATMuS—has improved access to medieval sources.

But:
❌ Raw outputs are hard to use
❌ Fully normalized models over-normalize & hallucinate

There’s a methodological gap.

If I give you the text
📚 omnium peccatorum quia ex quo dyaconus quando esset in futurum, stultus esset

Can you find the ATR error without the manuscript ?

Probably not.

ATR models that predict text and normalize in one go generate trustable text, but prevent detecting issues.

📄 New paper:
Pre-Editorial Normalization for Automatically Transcribed Medieval Manuscripts in Old French and Latin

Thibault Clérice, @rachelbawden.bsky.social , Anthony Glaise, Ariane Pinche, @dasmiq.bsky.social (2026) arxiv.org/abs/2602.13905

We introduce Pre-Editorial Normalization (PEN).

🧵⬇️

Excited to be co-organizing the #CHI2026 workshop on augmented reading interfaces 📚✨ Submissions are open for one more week! We want to know what you're working on!

our open model proving out specialized rag LMs over scientific literature has been published in nature ✌🏻

congrats to our lead @akariasai.bsky.social & team of students and Ai2 researchers/engineers

www.nature.com/articles/s41...

Cool postdoc job opportunity! A chance to work with some great English & comp sci scholars at Carnegie Mellon. Appreciate this ad stresses: chance to do interesting technical work; work on an interesting humanities problem; chance to publish both in humanities & comp sci venues. Looks great, apply!

The bias/variance tradeoff in 2026: Claude Sonnet wrote a program to solve the problem as described; Claude Opus figured out a shortcut from the example data that won't generalize.

[Job 📣] Are you curious about #AI applications in the #humanities? My Print and Probability research group (@print-and-prob.bsky.social) is hiring a postdoc! Come help us develop computational methods for identifying clandestine early modern printers!

cmu.wd5.myworkdayjobs.com/CMU/job/Pitt...

Debate me bro Statistical inference is rhetoric.

Statistical inference is a rhetoric of counts.

I had the absolute pleasure to visit @craicexeter.bsky.social, where I laid out an argument for how critical & computational scholars should lead the conversation on AI. We need to expand research on harms, interrogate corporate hype, and support people’s critical understanding these technologies

I couldn't figure out eloquent language to describe digital agents that assist with web navigation tasks, so I just wrote "click click" and if I keep this up maybe I will start referring to language generation as "word word"

Hi Honey, I’m Homo Neuricus Six Ways I'm using AI to Become More Human

A very random view into how some people* outside of tech think about and use chatbots. It's not coding, that's for sure, and some of it might sound ridiculous, but I think this kind of perspective and usage is way more common than we might assume.

*LA people (sorry, I love LA, but this is very LA)

Let's think step by step. If you could reconstruct the original page with high probability using a language model given the bag of words, you could:
1. demonstrate that bag of words models are useful, and
2. destroy the legal arguments people used to allow them to share bags of words.

The second new class I'm teaching is a very experimental graduate level seminar in CSE: "Building Small Language Models". I taught the grad level NLP class last semester (so fun!) but students wanted more—which of these new ideas work, and which work for SLMs? jurgens.people.si.umich.edu/CSE598-004/

For social scientists interested in LLMs for text classification/coding, the process here is potentially very helpful (even if you don't use the product itself).

Their core technique: Contradictory Example Training
Their training method: Binocular Labeling

More details in the linked post below.

‘Written in the Style of’: ChatGPT and the Literary Canon

The first research paper from WashU's AI Humanities Lab, which I co-direct with Gabi Kirilloff, is available now in the Harvard Data Science Review! Read to learn more about how (badly) current LLMs are at replicating literary style: doi.org/10.1162/9960...

Article Preview

📢 New article in #JCLS 5(1)! 🎉
@axelpichler.fedihum.org.ap.brid.gy, Endres, M. & @nilsreiter.de (2026) “#Interpretation, Argument, #Evaluation. A Workflow for Assessing #LLM-Generated Interpretations of #Poetry” doi.org/10.48694/jcl...

#RollingIssue #NLG #CLS #LiteraryComputing

yes, this is a really great paper, showing how AI can enhance individual science but narrow the general scope.

But if this is the case, why are models acting so differently between languages?
Datasets like eclektic show that models know different things in different languages. A rare fact is usually only known in the language in which it was seen.
bsky.app/profile/lcho...

Is this Burton’s translation or did he get it from someone else?

Do you have ideas for the future of reading?

Submit a 2-4 page paper to the CHI workshop I am co-organising! (deadline Feb 12) “Science and Technology for Augmenting Reading"

chi-star-workshop.github.io

Reading environments for classical languages FTW

Excited about this Duke AI conference + stoked to present new work on cultural AI. Grateful this high profile conference will include humanistic perspectives. Meaning, history, aesthetics, narrative etc are a part of the society centered AI question. Glad the humanities will be a part of the convo.