dexconstruct

Are all journalists addicted to passive voice?

I often get the sense that people think that LLMs are the product of arrogance, but the truth is the opposite. Researchers arrogantly clung to the idea that human knowledge is special for many decades even as it became clear that general methods work better.

Hopefully it sheds some light on how LLMs came to dominate AI. The lesson is so "bitter" because most AI researchers wanted domain specific human knowledge to be more important, but empirically general methods leveraging compute scaling have outperformed them.

A good follow up to this is Rich Hutton’s article about The Bitter Lesson, which is that AI models that encode human knowledge beyond statistics keep losing to more computation.

www.incompleteideas.net/IncIdeas/Bit...