Ross Taylor

Friend of the pod Ross Taylor (ex reasoning lead for Llama) launched a platform for open reasoning data, gr.inc, seems promising!

“And yet it reasons”

YouTube video by Ross Taylor The Hitchhiker's Guide to Reasoning

(🎥45m) The Hitchhiker's Guide to Reasoning

A talk about LLM reasoning, covering various methods, core problems, and future research directions!

www.youtube.com/watch?v=S5l5...

Taming Transformers for High-Resolution Image Synthesis Designed to learn long-range interactions on sequential data, transformers continue to show state-of-the-art results on a wide variety of tasks. In contrast to CNNs, they contain no inductive bias tha...

IMO VQGAN is why GANs deserve the NeurIPS test of time award. Suddenly our image representations were an order of magnitude more compact. Absolute game changer for generative modelling at scale, and the basis for latent diffusion models.

Really simple principle that would go a long way: assume good faith. Most people are trying to do the right thing.

The toxicity towards that HF employee for sharing that Bluesky post dataset was really bad, and reminded me of 2022 anti-LLM sentiment.

Not a great look for 🦋.

I personally vote text is self supervised learning (SSL) for the same reason image is SSL.

Text tokenization is a modification but so are patches in vision. If you want you can go all the way down to bits or raw audio samples. We know text bits work as (again modded) it's used in LLM compression.

Is it right to call large-scale pre-training on text self-supervised learning?

Seems more like large-scale, multi-task supervised learning. Labels are present as semantic units (tokens) in the corpus, unlike SSL for vision and other modalities?

Giving this place a go…