Kyle

Here's a version that lets me look at individual layers and heads (a layer just being the mean of all of its heads). Notice some heads really care about specific tokens that others don't.

You wrote “average attention score as the prompt progresses” so I was just clarifying for myself average over what

So the opacity is a running average score?

What do LLMs see?

I wrote a lil' tool that extracts the attention matrices out of open models and creates this typing visual, with each token's opacity changing according to its average attention score as the prompt progresses. Dimmer words are considered less important to the model.

Screenshot of the vancouver metro area rendered by the Stamen Design water color tileset, and it's gorgeous. it looks like a real water color painting

i just discovered Stamen's Watercolor tileset and it's just beautiful! I wish all maps where this beautiful.

Bring more whimsy and beauty to websites

We had this beautiful internet, and now it's filled with slop. Outside of my circles, I don't trust digital interactions with people, and I know they don't trust me either. (Some effects spill over into real life too.)

Was it worth it? Time will tell.

What happened to Science Goodreads and how do we rebuild it? A 65 million dollar question (at least) - CAIROS Blog The story of the rise and fall of Mendeley

1/ Finally wrote up “The Story of Mendeley”! Most people know the tool, few know about its rise and fall. The Mendeley story provides important clues for how to build self-sustaining AND non-extractive knowledge commons, which is why I think it deserves more attention 🧵

Highest quality content on here

On a lark, tested Opus 4.6 on a common academic task: take a 10k word article and shorten it to 6k words for submission to a new journal. I told it to use the command line tool texcount to count words given the known inability to count words. It failed in a fairly funny way…

And ML agrees! https://arxiv.org/abs/1803.03635 https://proceedings.neurips.cc/paper/2020/hash/322f62469c5e3c7dc3e58f5a4d1ea399-Abstract.html

And it’s not because of what scientists did. It is a political change that we did not ask for, nor control.

But now that is has happened, we can’t ignore it.

Scoop of ice cream in a shot of espresso, served in a wide-brimmed glass

I'm starting (a)ffogato fridays. it's legal. it's legal for me to have ice cream in a shot of espresso at 9am on a friday. it's legal for me to have a little treat.

Global warming discourse 10 years ago immunized me against predicting an end to long running trends on the basis of a few data points, because this little blip made people think global warming had stopped. Afterwards it wasn't even visible in the trend.

The Bluesky mute words panel showing “Waymo” muted for the next 24 hours

Network propagation on this site is fun. You can watch how misinformation spreads just scrolling through your feed. No sleuthing required.

Same for programming. One might say languages/communicating succinctly ≈ tool for thinking.

Elon's decision to shut down USAID is the direct cause of countless deaths. That he isn't persona non grata in many communities after that is a source of shame.

In almost all (all?) latent diffusion models, the VAE that sandwiches the diffusion model is not a real VAE: the KL loss has a weight of 10^-5 and there is an additional GAN loss that significantly boosts the visual quality. So it's more an adversarial auto-encoder than anything else.

Okay this JPL article has more details. It actually used image data as context which is pretty impressive!

https://www.nasa.gov/missions/mars-2020-perseverance/perseverance-rover/nasas-perseverance-rover-completes-first-ai-planned-drive-on-mars/

The output as markup makes sense, but what is the input context?

every epstein file drop underscores how elite power operates through shared socio-economic networks, regardless of people's ideological differences, populist posturing, or public feuds

How AI assistance impacts the formation of coding skills Anthropic is an AI safety and research company that's working to build reliable, interpretable, and steerable AI systems.

This study by people from Anthropic itself should raise huge alarm bells about the use of AI in teaching how to code (and later on in coding itself, but esp. in the learning stage).

And remember: this is by the people who make Claude!

tl;dr: not that long, read it
www.anthropic.com/research/AI-...

good afternoon

Ah I see it does have a built-in labeler. Maybe adjusting these settings will help: https://bsky.app/profile/moderation.bsky.app

Does bsky have a built-in spam label? I know you can subscribe to labelers (many exist for spam/bots) and when you do, you choose whether labeled accounts just have a label or are hidden. It would be nice to have a “show hidden” UX though.

The City’s Budget is our future. And you deserve to know how it works.

slack notification about AI searches

didn't really need this notification popping up at random in every slack I'm in

me: How should I invest my savings to avoid the AI bubble?

tech friends: What bubble?

non-tech friends: What savings?