Yanai Elazar

Excited to have the Big Picture workshop back for another iteration this year at @aclmeeting.bsky.social
Submit your big picture ideas, consolidation work, phd thesis distillation, etc. by March 5th!

www.bigpictureworkshop.com
w/ Allyson Ettinger, @norakassner.bsky.social, @sebruder.bsky.social

But I guess this is where the "position" of our paper gets in. At least it wasn't rejected from arXiv 😂

a) but not a reasonable one. there are many such papers, many good ones, and it's not like arXiv is this sanctuary of high-quality papers from the first place. I don't see why should we enforce this quasi appearance of quality suddenly.
b) I don't think this is true anymore.

correct! I combined them together here for brevity (some may say laziness. but in fact arXiv treats them the same, and so do we (at least we try, in our evaluations and instructions).

Please read the paper. Percentages are far from the only metric to look at.

Job application process in 2026:
Applicant: I have written 3 papers
Interviewer: where?
Applicant: arXiv
Interviewer: hired!

No, we unfortunately couldn't get our hands on such data. But from our understanding such data (in the worse case scenario) shouldn't affect the trends we found in our results.

CS ArXiv recently banned “review and position” papers, but what are those? Do they include more generated content? Who is most affected by this change? @yanai.bsky.social and I dug into the data to find out!

Nearly 50% of Computers & Society papers might be censored, vs 3% of Computer Vision ‼️

and lastly, a huge thanks to @pangram.com who provided us with credits to detect AI slop in scientific papers!

LLM-Generated or Human-Written? Comparing Review and Non-Review Papers on ArXiv ArXiv recently prohibited the upload of unpublished review papers to its servers in the Computer Science domain, citing a high prevalence of LLM-generated content in these categories. However, this de...

It's been an absolute pleasure to work on this with @mariaa.bsky.social ! and I hope you find this interesting.
Feedback is welcome!
arxiv.org/abs/2601.17036

In addition, when considering different subfields, we find striking differences in how we classify 'position' papers, leading to huge differences in how this policy affects different CS subfields.

However! when considering the number of papers from each type, we find there are significantly more non-review papers that are LLM-generated in all categories (cs and physics in the first tweet, math and stats here).
These results question the main motivation of this new policy.

We then calculate the percentage of LLM-generated papers in each category, and find that indeed, 'position' papers have higher percentages of LLM-generated papers, across the past three years.

We establish two classifiers that predict based on papers' abstracts* whether they are 'position' vs non-position papers, and whether they are LLM-generated, using two existing classifiers.

We also experiment with classifying the full papers, and reach similar conclusions.

In this new paper (w/ @mariaa.bsky.social ) we study these questions, and try to provide some answers.
Overall, we find evidence that this decision does not make much sense, and that it should be reconsidered!

However, they provided no empirical evidence to support this decision. How many LLM-generated papers are being posted on arXiv? How many position papers are posted on arXiv? Do these ratios justify this abrupt solution? What are 'review' papers anyway?

🚨 New Study 🚨

@arxiv.bsky.social has recently decided to prohibit any 'position' paper from being submitted to its CS servers.
Why? Because of the "AI slop", and allegedly higher ratios of LLM-generated content in review papers, compared to non-review papers.

Microsoft word wizard

"at least it's not Linkedin!" - if that's not shitposting idk what is

We'll see about that...

I think I'm in a secret competition with my co-author about who uses more footnotes in the paper.

Thanks! I've actually read it, and I feel like it falls mostly under the "for people who don't know how LLMs work" category, for which it can be super useful! But I'm looking for something else.

Any good tutorials/guides on how to code with LLMs?
What I found so far targeted folks who don't know what they are, mostly setting expectations and saying what LLMs can or cannot do.
I'm beyond that. I'm looking for best practices. Kind of design patterns for vibe coding.

Estimating the Causal Effect of Early ArXiving on Paper Acceptance What is the effect of releasing a preprint of a paper before it is submitted for peer review? No randomized controlled trial has been conducted, so we turn to observational data to answer this questio...

A data point in favor is from an observational data study we performed on the effect of early arxiving (= potentially breaking anonymity) on paper acceptance. We measured very small effects, and perhaps more importantly, no difference in the effect on different groups.
arxiv.org/abs/2306.13891

Call for Applications for the Azrieli International Postdoctoral Fellowship-2026-27 - The Azrieli Foundation

azrielifoundation.org/azrieli-fell...

Interested in interpretability, data attribution, evaluation, and similar topics?
Interested in doing a postdoc with me?
Apply to the prestigious Azrieli program!
Link below 👇

DMs are open (email is good too!)

You had one job!

would you like to have a pop tart?

cool, let's have a chat then!

Detection and Measurement of Syntactic Templates in Generated Text Recent work on evaluating the diversity of text generated by LLMs has focused on word-level features. Here we offer an analysis of syntactic features to characterize general repetition in models, beyo...

and in another work led by @chantalsh.bsky.social we also found a convergence of LLMs to the same patterns (syntactic templates)
arxiv.org/abs/2407.00211