FleetingBits

kjv?

An example is state health insurance - if you move states, it feels complicated and confusing to fix your health insurance - no one wants this. But, hasn't changed in my lifetime.

And insurance has become the scapegoat (not even recently) - like over the last 20-30 years maybe.

I think people underestimate popular anger (whether or not it is justified - leaving that to another side) - there are a bunch of issues around healthcare that everyone seems to agree exist but don't seem to change (not even political stuff) - and I think that fuels the anger more

feels like the Apollo Research review of o1 was a bit adversarial - just getting that vibe from the description in the system card

YouTube video by Practical Engineering The Wild Story of the Taum Sauk Dam Failure

It's interesting how many disasters come from a collection of small failures - often because people are not sufficiently motivated to coordinate.

www.youtube.com/watch?v=zRM2...

Interesting thread on what social media rewards in academic articles. I think overbroad claims but, you know, you take what you can get.

x.com/0xredJ/statu...

hmm I feel it would be fun

inspiration

YouTube video by FAR․AI Buck Shlegeris - AI Control [Alignment Workshop]

Another interesting video - I think the idea that providers should have to stop deployment of their models if the models attempt to escape is reasonable.

Probably the starting point is actually a set of reporting requirements, but I digress...

I joke I joke lol

I think they are called GPUs ~~

lauren’s views on The Curve conference

x.com/typewriters/...

I can't really tell whether he is saying something other than -> "the lisp syntax generally looks like lisp lists and it is easy to parse those into trees using the tools of the language".

Richard Ngo – Reframing AGI Threat Models [Alignment Workshop] YouTube video by FAR․AI

claimed - AI misuse risk and AI misalignment risk are the same thing form a policy and technical perspective

I think changing the link behavior and encouraging people to go back to linking their substacks would really bring a lot of a academics back to twitter - he doesn't have good "monetization" for academics who don't want $ but want reputation with peers.

I feel like it hit a bunch of touchstones of the current twitter convo - a lot of which make Twitter more boring now?

But - just a few tweaks from Elon could take some of the wind out of Bluesky's sails...

I mean the goal of Anthropic & co is to figure out how to get a small number of extremely high quality labels, and then use synthetic data to get great coverage from them.

So, it is the "average data labeler's answer" but that average data labeler is an IMO winner.

I think some of this is aimed at "in domain" style questions - which are likely to be much closer to the preference labeled prompts.

I think he's making a valuable point there, aimed more at people who expect LLM answers to be magic to a very regular question.

going through more of the references - feels like "many such cases" type stuff

But - AI Safety would be much improved if the optimization was more around ||clearly communicating real findings|| rather than ||getting a great paper title||

I mean - all of this just shows how the OpenAI superalignment team really was asking the right questions - (1) how do we survive with bad labels? (2) how do we give labelers tools to make better labels? (3) how do we help models give answers that labelers can more easily discriminate between?

or, the data labelers don't check the 20 line code fragment that the model spit out

hmmm

And, then the finding ends up being something like "our data labelers don't check the references" so we get bad labels : X

uhh, ok

this paper is referenced, which should have examples of reward hacking and the authors are high quality authors (Ethan Perez!)

Reward Hacking in Reinforcement Learning Reward hacking occurs when a reinforcement learning (RL) agent exploits flaws or ambiguities in the reward function to achieve high rewards, without genuinely learning or completing the intended task....

like this blog post feels this way to me - very few real examples that I can inspect

lilianweng.github.io/posts/2024-1...