Sebastian Farquhar

By default, LLM agents with long action sequences use early steps to undermine your evaluation of later steps; a big alignment risk.

Our new paper mitigates this, keeps the ability for long-term planning, and doesnt assume you can detect the undermining strategy. 👇

Reducing unnecessary action *does* drive growth. We are all more productive when we achieve the same things with fewer inputs, wasting citizens' time makes the whole country less productive. Create slack in people's lives and watch what they create with it!

Interesting analogy, because of course the Dreadnoughts were mostly militarily useless and were obsoleted by changing strategic considerations before they were ever deployed.

I desperately want to know what experience made you try out this prompt. Who hurt you?

Interesting. I guess I'm surprised that oil prices would have such a big effect on total fossil fuel CO2 emissions (presumably mostly coal over the period?). But maybe substitutability links them enough.

Actually just zoomed in on the data viewer. It does look like 1973 is the break point. Still curious about why the effect was so persistent.

Why did land use emissions shrink lots between 196-70 and then stop shrinking?

Why did the oil price shock lead to sustained flat per capita fossil fuel emissions? It was short. Also it started after the trend breaks.

I'm surprised that the per capita global emissions look like they are trending pretty flat from 1950ish, much earlier than I would have guessed. Presumably many people greatly increased their energy consumption after then? Do you know what is driving this?

Updated! Keep em coming.

@maosbot.bsky.social what do you think, do you belong on this list? I think most of your research isn't quite in this area but not sure how you self-identify on research focus at the moment.

Weak signal perhaps, but you are one of two accounts on Twitter that I genuinely miss here. If you did make the leap that would be lovely :D

Help me grow this starter pack for technical researchers working on AGI safety! go.bsky.app/D6P44sC Some flex, but aiming for mostly technical research rather than governance/strategy. Who am I missing?

Agreed. I basically don't believe the result at all. Seems like the memetic strength is it lets you feel well informed.

You too! Just DMed you :D

Strongly agree. On a cold winter day they are basically a pure comfort upgrade. Also great for hayfever.

The fact that every field that has tried to have a reproducibility crisis has been able to suggests that the way journals have done it for decades underinvests in finding critical flaws in papers and that retractions are too rare and late to depend on.

I've seen at least a couple cases where a very high effort public review identified a significant flaw that the reviewers had missed. Losing that would be a real cost.

Little hat-tip to www.jakobfoerster.com/how-to-ml-pa... from Jakob Foerster and jsteinhardt.stat.berkeley.edu/blog/advice-... from Jacob Steinhardt who have excellent advice as well.

How to Write ML Papers This doc is aimed at students learning to write ML papers as well as more experienced writers. It isn’t about how to do the research itself, but about how to present it in a way that makes it impactfu...

Starting to prepare yourself to submit to ICML? Here are my tips on how to write well for an ML research audience. sebastianfarquhar.com/on-research/...

Entertaining essay about how the decline in practical engineering education has been devastating for *checks notes* professional criminal safe crackers. (Ok, mostly just a fun history of safe cracking.) www.timhunkin.com/94_illegal_e...

And for readers! Twitter has been getting gradually more boring. Turns out this whole hyperlink thing is a big deal for the internet.

Something I loved most about the internet in the 2000s was the idiosyncratic personal webpages that some people had put a crazy amount of time and effort into.

These pages must still exist right? What are the best ones you know of?