Colin Danby

Some of us rode meh all the way to tenure.

J'oublie.

These are good and true points.

Yep. I want to learn things! Trying to postpone becoming an elderly bore.

*of* research

Thinking here of the "feeling for the organism" in Evelyn Fox Keller's bio of Barbara McClintock, Maynard Keynes' efforts to get a grip on rough magnitudes of economic phenomena, the way historians develop a feel for when to back up and look around for what else must be going on.

4. The essential quality research is the opposite of replicating conventional wisdom, which LLMs are set up to do. You want to spot logical problems and discursive sleights of hand in routine rhetoric. You want to pick up on the ways the phenomena under discussion escape the lit.

3. Some people will find good, innovative uses! Mediocrities will find bad uses and flood the review system with bad articles but I see no way around that. We will have great drifts of false citations.

2. When we train students, it's reasonable to restrict tools so they learn skills. But for colleagues, I have no view about how you should do your work. Use what tools you want.

LLM discourse flares up every six weeks or so. Feels like we need an FAQ. Points I find myself repeating:

1. Please specify tasks and tools. "AI" is too broad. Boosters do the camel's-nose thing, starting with an innocuous use and widening.

Criticism can also be over-broad.

Oh good it came back. Got tired of touching grass.

LLMs are also easily fooled into the assuming the state-of-the-lit summarized early on represents the article's conclusions, whereas the author is in fact setting up pins in order to knock them down later.

If you use LLM summaries to access an area you don't know, you sabotage your own learning.

In a lot of fields, articles are structured to address the tl/dr problem. They set out their research question early and flag the answer. You can jump to the results section at the end for more. People who know the lit can figure out what an article does very quickly.

The summaries are no good. (And this is what I suspected you meant.)

No problem is more already-solved than digesting journal articles. They typically come with abstracts, which are carefully constructed by their authors. In some fields there are "keywords."

When we started flying through Dubai to go to India I looked at possible side trips. Always wanted to visit Muscat! But yeah, the weirdness of that lovely huge airport is now really apparent.

If this person just means an LLM might fish up some useful references to look at, fine, if it works. We assume a human researcher still reads them. It's when people ask for LLM summaries that the mischief begins.

There's a camel's-nose problem here.

Conversation at the barber shop just last week. Started with current US troubles (she's immigrant, husband laid off by doge) and free-associated to JE alive, missing kids, cannibalism.

So yeah, blood libel, maybe pishtaco myth.

Where I'm with our host is that a lot of the insights in handling historical material come from recognizing that something you had not expected is going on. Something waves at you, jumps out of the pattern you want to impose.

As long as people can build in rigorous checks, I'm fine with this. You're a professional and should figure out what tools to use. It depends on the nature of the work.

Offer Aristotle to critique your logic, Voltaire for pointers on rhetoric, Nietzsche just to mess with you.

It will certainly make the work of editors and reviewers harder. In the before times, you might get a bad submission to review, but you could quickly figure out why/how it was bad.

How, now, to handle splodge? Or some good bits and some splodgy bits?

That is not pattern-replication but its opposite: asking what might be wrong with a replicable pattern.

And while tool-agnostic, I have to note that you do not express a clear idea of what it means to "do social science research," conflating many different tasks. Most fundamentally, as you would agree, a researcher needs a critical understanding of existing lit and how it handles evidence.

Bryan is straw-manning, and you are over-generalizing. My view is that you're an adult and you can use what tools you like. I do care about the quality of output. LLMs disprportionately raise the output of sloppy researchers.

Watched a half hour and noped.

I see none of this day to day. Traumatic events seem to drive less-regulated people into big-acct replies, and I'm really sorry that happens.

Arguably the greatest expertise we develop is in asking questions and probing for weak points. It's a hard thing to get across to students because the pop idea of knowledge is having a large body of facts under control from which you can soeak confidently. It's what LLMs imitate.

Yup. Don't make trouble for yourself or your chair. This would be especially hard for contingent colleagues.

My logic needs scrutiny.

You would hope the living ones could sue these bastards.