Rohan Padhye

My current conjecture is that the examples of broken JSON syntax in the paper somehow accidentally caused something like second-order prompt injection in Google Scholar's indexing pipeline. But there's only one way to find out ^^^

Hilarious! It looks like Google Scholar is pulling citations for a different Kirschner et al. paper from 2006.

I wonder if one could use the technique proposed in "Debugging inputs" to identify what causes this anomaly. Gonna need Lukas to publish a *lot* of papers.

🚨 Our amazing #FUZZING'25 keynotes are online!

"Constraining Fuzzing without Paying Too Much" by Miryung Kim
youtu.be/L90MBb6NLBE

"Are you sure you belong in academia?" by Will Wilson
youtu.be/qQGuQ_4V6WI

// @mboehme.bsky.social, László Szekeres, @rohan.padhye.org, @ruijiemeng.bsky.social

The limit applies to references too? Preposterous!

Very cool: ‪@aoli.al‬ uncovered a deadlock in OpenJDK that can be triggered with a tiny test case and Fray's deterministic concurrency testing & debugging support. Read his blog post here: aoli.al/blogs/jdk-bug/

If you write Java/Scala/Kotlin, try Fray yourself: github.com/cmu-pasta/fray

Just Accepted to ACM TOSEM!

The "Havoc Paradox" is about the relationship between byte-level fuzzer mutations and their effect on the inputs produced by generators for structured strings (e.g. XML/SQL). Can disruptive mutations be controlled? Should they be? Find out.

📄 dl.acm.org/doi/pdf/10.1...

🖊️ Register here: ntnu.eventsair.com/fse2025-isst...
(FUZZING is a co-located workshop)

List of Accepted Papers at the FUZZING Workshop

We also have an excellent program of research talks and *fuzzing nuggets*. Detailed schedule coming soon.

conf.researchr.org/home/issta-2...

We're excited to announce two keynote speakers for the #FUZZING'25 workshop (part of @issta_conf at Trondheim, Norway):

[*] Will Wilson, CEO and Co-Founder of Antithesis
[*] Miryung Kim, Professor and Vice Chair of Graduate Studies at UCLA

conf.researchr.org/home/issta-2...

The JQF repo is now both popular enough (700+ stars) and contains enough buggy/vulnerable code as sample fuzz targets that we're getting occasionally spammed with crappy AI-generated patches.

I can't imagine what bigger OSS projects are dealing with right now.

Congratulations!

Delighted to receive an ACM SIGSOFT Distinguished Award for this work... It's about time!

Proud of the PASTA Lab students, including our visiting undergrads :-)

Text highlighted from a research paper that says "To the best of our knowledge, there is no existing search-based testing approach for productiongrade AV software, including [20], [21], [41]–[55] that: (i) uses our novel gene representation"

Love this argument: prior work does not use our novel idea.

Submission deadline for the Fuzzing workshop is tonight (AoE)! Send us those nuggets and research ideas.

Rohan

Paper titled "It’s About Time: An Empirical Study of Date and Time Bugs in Open-Source Python Software". Authors List: Shrey Tiwari Carnegie Mellon University Pittsburgh, PA, USA shrey@cmu.edu Peter Vandervelde∗ University of California, Santa Barbara Santa Barbara, CA, USA pvandervelde@ucsb.edu Serena Chen∗ University of California, San Diego San Diego, CA, USA sec022@ucsd.edu Ao Li Carnegie Mellon University Pittsburgh, PA, USA aoli@cmu.edu Alexander Joukov∗ Stony Brook University Stony Brook, NY, USA ajoukov@cs.stonybrook.edu Rohan Padhye Carnegie Mellon University Pittsburgh, PA, USA rohanpadhye@cmu.edu

Happy Daylight Savings Time to everyone in the US! A few more weeks for European Summer Time.

If you notice some of your apps glitching, don't be alarmed. Even ChatGPT can't write correct date/time code!!!

See more in our upcoming paper: rohan.padhye.org/files/dateti... (MSR'25 preprint)

It's always been a "response" for me.

The only time it was a "rebuttal" was when I explicitly thanked Reviewers A, C, and D for their valuable feedback.

I'm super excited about this new track at the #FUZZING'25 workshop. It's the academic version of thoughtful blog posts, but with a paper and talk for wider reach!

Submission deadline is in a month (March 20th)!

fuzzingworkshop.github.io

#FUZZING'25 CALL FOR PAPERS
──────
✨ New OC members:
* Ruijie Meng (@ruijiemeng.bsky.social; NUS)
* Rohan Padhye (@rohan.padhye.org; CMU).
✨ New paper type: Fuzzing Nuggets (short papers).

🔗 fuzzingworkshop.github.io
📅 20.March (Submission)
📅 17.April (Notification)
📅 28.June (Workshop)

Blog post titled "Concurrency bugs in Lucene: How to fix optimistic concurrency failures" By Benjamin Trent and Ao Li (February 7, 2025) Text reads: "Thanks to Fray, a deterministic concurrency testing framework from CMU’s PASTA Lab, we tracked down a tricky Lucene bug and squashed it"

Back to basics: Concurrency testing in Java!

Our new tool *Fray* correctly solves a 25+ year old problem for real-world software. See this feature from Elastic Labs about Fray's contributions to Lucene.

📰: www.elastic.co/search-labs/...

🔧: github.com/cmu-pasta/fray

📝: arxiv.org/pdf/2501.12618

Cite only the paper title for now, and submit a PDF without embedded fonts. They'll probably tell you to fix it and re-submit in 48 hours. Plenty of time to get an arxiv identifier :-)

Takeaway: Don't just fuzz and wait for bugs to show up. Measure what your inputs look like based on user-defined predicates, and things start making a lot more sense.

A screenshot with two columns. On the left, a Python property-based test annotated with `event(<predicate>)` statements. On the right, a visualization of how frequently event predicates have been excercised by random tests, plotted as histograms and tree-maps.

[3/3] Another similarly cool idea is "events" in Tyche (by @harrisongoldste.in et al.), which is a PBT visualization extension for VSCode. We've actually integrated Tyche into JQF now so it works with Java fuzzing! Check it out.

(Ref: github.com/tyche-pbt/ty..., github.com/rohanpadhye/...)

You may be less familiar with the second type of assertion, which at Antithesis we call Sometimes Assertions. Just as an always assertion asserts that something is always true, a Sometimes Assertion asserts that something is sometimes true! Here are some examples: ``` assertSometimes x < 1; assertSometimes y == 1; assertSometimes(condition); ```

[2/3] I like the "Sometimes Assertions" abstraction recommended by Antithesis, which generalizes code coverage to user-defined predicates, possibly interleaved with application logic. Maybe we should support these in JQF too.

(Ref: antithesis.com/docs/best_pr...)

``` @Fuzz public void testMap2Trie(Map<String, Integer> map, String key) { assumeTrue(map.containsKey(key)); // Create new trie with input `map` Trie trie = new PatriciaTrie(map); // The key should exist in the trie as well assertTrue(trie.containsKey(key)); } ```

How do you know whether random testing is working as expected?

[1/3] Long ago in JQF, we used `assumeTrue` to bias fuzzing towards *valid* inputs. This is powerful, but the abstraction is quite coarse if you have many properties.

(Refs: github.com/rohanpadhye/..., rohan.padhye.org/files/zest-i...)

And now that we’re all here, some work!🚨 Are Large Language Models Memorizing Bug Benchmarks? 🚨
There’s growing concern that LLMs for SE are prone to data leakage, but no one has quantified it... until now. 🕵️‍♂️ 1/

It was expected to happen once this year, because there are 366 days.

Now to find everyone else again.

Alright, I'm here!