Aldozzzzz

😂😂😂😭😭😭

😍😍😍

CLIP Is Shortsighted: Paying Attention Beyond the First Sentence CLIP models learn transferable multi-modal features via image-text contrastive learning on internet-scale data. They are widely used in zero-shot classification, multi-modal retrieval, text-to-image d...

Our freshly accepted CVPR 2026 paper is up on arxiv 😍. Project page, open code and more detailed post coming soon!

« CLIP is Shortsighted: Paying Attention Beyond the First Sentence ».

#CVPR

In this amazing multidisciplinary collaboration, we report our early experience with the @openclaw-x.bsky.social ->

Memorization vs. generalization in deep learning: implicit biases, benign overfitting, and more Or: how I learned to stop worrying and love the memorization

What is the relationship between memorization and generalization in AI? Is there a fundamental tradeoff? In infinitefaculty.substack.com/p/memorizati... I’ve reviewed some of the evolving perspectives on memorization & generalization in machine learning, from classic perspectives through LLMs.

Inspiring and on point quote for 2026 🥲.

Haven’t finished it yet. Played last winter on NSO 😅.

BOTW ~ TOTK ~ Skyward Sword 💚💚💚

He’s so freaking cringe 😬. Is this real life? 🤯🫠

Honestly a pretty simple solution: we force these models to pay attention to the whole text sequence 👀. The motivation of this work was that all previous models were mostly focusing on the first few tokens (the « summary » sentence). Details are in later parts of the caption typically 👀.

This was legit the first time I was feeling confident about a big AI conf. paper! Initial reviews were already 3/3 accepts, and we managed to get final scores of 5/5/6!

Teaser: we’re now the new SOTA (by a huge margin) for image-to-long-text CLIP-style retrieval 👀.

Accepted at CVPR! More info to come soon (we need to work on our project page and all 😅). 👀

a cartoon of a man sitting in front of a computer screen that says vertek ALT: a cartoon of a man sitting in front of a computer screen that says vertek

Me refreshing OpenReview every 10 seconds for CVPR decisions 👀.

😩😩😩

5 days to go until final CVPR decisions 👀👀👀. #cvpr

"Moderna's CEO announced the company will no longer invest in new Phase 3 vaccine trials for infectious diseases: 'You cannot make a return on investment if you don't have access to the U.S. market. Vaccines for Epstein-Barr virus, herpes, and shingles have been shelved.” 9:12 AM Feb 12, 2026 604.8K Views

In terms of what cures are being lost:

- Epstein-Barr virus is perhaps the major trigger for multiple sclerosis
- herpes simplex virus causes cold sores, genital herpes, infections in babies, deadly meningitis
- shingles virus causes an intensely painful disease

You still have time to apply to the CVPR (horror) art submission 😂.

It took me weeks, but finally it's there: an overlong blogpost on synthetic pretraining. vintagedata.org/blog/posts/s...

Very very cool and important blog!

😨😨😨

CONTRIBUTOR GREG BROCKMAN AMOUNT ELECTION TO DAT $25,000,000 $25,000,000

The largest Trump superPAC donor so far this cycle is the president of OpenAI

Geometry and symmetry obsession ft Berliner Dom 🇩🇪.

Coming soon #ECCV2026, Malmo 🇸🇪

Been pretty heads-down finishing Chapter 6 on implementing RLVR via GRPO. Just finished, and it might be my favorite chapter so far.

Code notebook: github.com/rasbt/reason...

(And it should be added to the early access soon.)

The next chapter adds stability and performance improvements to GRPO.

Or honestly just that they managed to get out of bed, did groceries, or baked cinnamon rolls 🤭😅

Nano Banana Pro surprisingly very good at meme generation with perfect text 🫢. #aimeme #airesearch

Hopefully we get some kind of remaster or remake in Switch 2.

The year of “reasoning” The year of agents The year of coding agents and Claude Code The year of LLMs on the command-line The year of YOLO and the Normalization of Deviance The year of $200/month subscriptions The year of top-ranked Chinese open weight models The year of long tasks The year of prompt-driven image editing The year models won gold in academic competitions The year that Llama lost its way The year that OpenAI lost their lead The year of Gemini The year of pelicans riding bicycles The year I built 110 tools The year of the snitch! The year of vibe coding The (only?) year of MCP The year of alarmingly AI-enabled browsers The year of the lethal trifecta The year of programming on my phone The year of conformance suites The year local models got good, but cloud models got even better The year of slop The year that data centers got extremely unpopular My own words of the year That’s a wrap for 2025

Here's my enormous round-up of everything we learned about LLMs in 2025 - the third in my annual series of reviews of the past twelve months
https://simonwillison.net/2025/Dec/31/the-year-in-llms/
This year it's divided into 26 sections! This is the table of contents