Jeremy Jordan

one feature i really enjoy on twitter is muted keywords, would love to see this added to bluesky.

(ironically it’s because i want to mute the keyword “twitter” on here, among other keywords)

Understanding the Transformer architecture for neural networks The attention mechanism allows us to merge a variable-length sequence of vectors into a fixed-size context vector. What if we could use this mechanism to entirely replace recurrence for sequential modeling? This blog post covers the Transformer architecture which explores such an approach.

new blog post: all about transformers! covering the intuition behind an "attention only" network, building up an understanding of scaled dot product attention, and detailing the original transform architecture. let me know what you think! 🙏

hello and welcome! 👋🏼😄

it’s such a satisfying feeling to put in the effort to deeply understand something. poke and prod at different perspectives of the idea until it all clicks ❤️

we should also evaluate our tokens based on “how well does modeling {this set of tokens} improve performance on {other sets of tokens/downstream tasks}”. not all tokens created equal! i would argue yours have been pretty impactful 🙂 at least they have for me!

granted, humans hallucinate as an adaptive measure shaped by evolution (our brain tricks us into perceiving things that help us stay alive) so it’s a little different, but still

people criticize LLMs for hallucinating like we’re not all out here hallucinating reality

👋🏼 i’m interested in the open protocol approach behind bluesky, curious to see how it evolves. i’m optimistic!