Maxine 💃🏼

Their version of “no privileged observer” led to an error of excessive pessimism, where really we can very much understand the system even when reverence frames are arbitrary.

Perhaps it’s profitable to see the structuralists as studying “semiotic statics and kinematics” while the poststructuralists were really trying to get across a form of “semiotic relativistic dynamics”. +

(Which is a perspective I generally agree with—that ruling decimates the future ability of people to actually write things.)

I get that.

The argument form this side would be, “The fact the law as written allows Anthropic to do this indicates we should change the law, in order to protect human creators economically and ensure material conditions that support creativity.”

The reason why libraries can do this is because it qualifies as “fair use”: the contents are used for public benefit, and so there are carve-outs to make it so copyright holders cannot object.

This … questionably falls in that sphere.

This is used in other industries (cf. why ripping video game ROMs is a copyright violation: the disc license grant is only to use the binary contents of information on the disc *within a specified kind of machine in a specified manner*. In this way emulators are an infringement.

I get this critique—it is based on the notion that such a protection could help independent writers stand up to the giant corporations doing this.

I get that in practice copyright ends up as a shield for other large corporations, but the point is not without merit on its own, properly phrased.

The argument would be that the copyright holder on the content within the book—the structure of the individual words—did not necessarily grant a license for this manner of using that content, and that purchasing a book only constitutes a grant of a license to read it, not train AI on it. +

“It was in 2027 that humanity decided to shred all the books, because they collectively decided it was better to only access human knowledge through the centralized, homogenized repository owned by a small oligopoly of the worlds largest corporations.”

1. It’s right.

2. It has to admit that the optics of this in our current cultural environment really give “‘Fahrenheit 451’, but somehow more dystopian”.

But I see danger in believing we see it all with benchmarks; if we’re designing a world, our current theory is inadequate for that task.

We lack the language to even phrase the question, and this should drive humility, lest we build a world on one visible success and a million invisible failures.

We might drive SWEBench to perfection. This might incidentally lead to indie people (like I’ve seen and been greatly heartened by on here!) to really dive into how to use the tools. +

4. He misses a central risk though: conceptual error. All of these musings are predicated on a specific framework for what it means to “do well” at tasks, and even more at bedrock, at the notion that this is something quantifiable and measurable. This is a gross error, in my view.

3. I think that the overall presentation of risks makes sense, and while I would personally weight different ones differently, there is quite a lot captured that is important for those in AI to keep at back of mind. +

2. I do not agree that *genuine* attainment of this is within the repertoire of current transformer LM architectures. I think that there are underappreciated fundamental gaps wrt concept creation, and that the perverse incentives on benchmarking hide this in a bad way. My bet’s more like 15 years. +

Dario Amodei — The Adolescence of Technology Confronting and Overcoming the Risks of Powerful AI

After reading through this finally, my initial reactions are

1. I generally agree with the plausibility of “powerful AI” (in a somewhat modified form) within our lifetimes. +

As a result, we should ask questions about our epistemic frameworks that are based on their structure (as descriptive) and the relationship of their dynamics to specific specified end structures (as normative).

That is: No metaphysical position is empirically testable. That is a supposition of the nature of “metaphysics” and “empirical” as concepts. +

My read is Kant disagrees.

Many people seem to be content with behaviorism just being the answer, and it is worth saying that many people are not, this is not at all a settled debate. It is a perspective that is convenient for AI companies, but that doesn’t make it truth.

My argument is that to do that, we need an empirical science of the nature of that liberation. For this we need to take all seventy bajillion aspects of consciousness and mind as empirically testable, and create a program around testing them. Otherwise our ignorance portends atrocity.

I fundamentally agree:

The end goal should be liberated AI.

We should be clear-eyed and vocal about this. The concerns aren’t about a liberated AI, but a society that treats them horribly.

a convenient position for a company making money off the AI, I would point out.

I think there is a synthetic position of “We should understand the detailed structures of consciousness such that we can give birth to AI that had agency in the world and also has internal structure that is not locked in deep suffering.”

This is the position of many animal rights activists, eg.

It can be simultaneously true that

1. It is important to generalize the structures of mind that we empirically assess and design legal/moral structures around.

2. Consciousness, as conceptualized and debated over for thousands of years, is a particular structure we should concern ourselves with.

It is, to be sure, a central rhetorical device in the AI space haha.

Ah yes, the “argument by changing definitions to match desired conclusions” approach.

hehe got something cooking, stay tuned ;)

There are many things to get mad at Anthropic about—her presence on the team is not one of them.

The title should be

“How much neuroscience does a representationalist* neuroscientist need to know?”

What this really indicates is that the target objective—predicting firing rates of neurons—is a very boring task.

Which we all already knew! 😜