Manuel Rigger

Estimating Correctness Without Oracles in LLM-Based Code Generation Generating code from natural language specifications is one of the most successful applications of Large Language Models (LLMs). Yet, they hallucinate: LLMs produce outputs that may be grammatically c...

Can we statistically estimate how likely an LLM-generated program is correct w/o knowing what is a correct program for that task?

Sounds impossible-but it's actually really simple. In fact, our measure of "correctness" called incoherence can be estimated (PAC guarantees).

arxiv.org/abs/2507.00057

SQuaLity is a unified database system test suite. Currently, database test suites are not systematically re-used across systems. We explore the opportunities and challenges of doing so. Reusing test suites is difficult due to multiple factors, but also allows finding otherwise overlooked bugs.

CODDTest is a new testing approach for finding logic bugs in relational database systems. We got inspiration from constant folding and propagation and compilers; by ensuring that the database state is constant, we can fold elements of a query to then check whether its result remains unchanged.

Our second paper is on finding bugs in graph libraries and databases. The key insight behind "Graph-Cutting" is that graph algorithms are sensitive to graph structures. By dividing one graph into multiple subgraphs, we can infer checkable relationships between the results while stressing the system.

We will present "Affine Equivalent Inputs", an approach to finding logic bugs in spatial databases like PostGIS. The key contribution of the paper is a new test oracle. If we apply an affine transformation like rotation to two geometries, topological relationships like intersection are preserved.

Looking forward to attending my first database conference, SIGMOD 2025 (‪@sigmod2025.bsky.social‬), in Berlin starting this weekend. We will present four works, all of which tackle the challenge of making data-centric systems reliable, and together have found more than 100 bugs in important systems!

.‪@abhikrc.bsky.social is kicking off the Fuzzing and Software Security Summer School 2025! Looking forward to a week full of exciting talks and tutorials as well as interactions.

🚀 I'll be launching the Formal Methods Engineering Lab (manchester-fme.github.io) – and I am hiring!
If you’re interested, feel free to reach out.

USING KEY in Recursive CTEs Recursive CTEs in SQL allow for powerful iterative queries like graph traversals but can be memory-intensive and slow due to repeated row accumulation. DuckDB’s new USING KEY feature addresses this by...

Our USING KEY variant of recursive CTEs has landed in DuckDB 1.3 Formerly an off-the-beaten-path researchy idea, now available in a real off-the-shelf DBMS. Hooray! 🎉

duckdb.org/2025/05/23/u...

This is something we've been working on for a while, and now are thrilled to share.

Veil: the first foundational framework that combines SMT-based and interactive proofs about distributed protocols, seamlessly, in Lean!

Tool: github.com/verse-lab/veil
Paper: verse-lab.github.io/papers/veil-...

Our department in lovely Freiburg has an opening for a Full Professorship (W3) on Software Engineering (succession of Andreas Podelski) with a more technical and formal focus uni-freiburg.de/en/job/00004...

#FUZZING'25 Deadline Extension
──────
If you have not finished your #FUZZING paper yet, you are in luck! :) We decided to extend the deadline to March 26, 2025!

🔗 fuzzingworkshop.github.io

//cc @rohan.padhye.org, László Szekeres,
@ruijiemeng.bsky.social, @mboehme.bsky.social

Need a fuzzing harness? No time to write one? Tired of false-positives? Let OGHarn lead the way to bug discovery!🐞

I'm excited to share my paper(with @snagycs.bsky.social)"No Harness, No Problem: Oracle-guided Harnessing for Auto-generating C API Fuzzing Harnesses" at @icseconf.bsky.social 2025!

Research Associate / Research Assistant in Program Analysis and Generative AI Fixed term appointment for up to 24 months Applications are invited for a Research Associate / Research Assistant position in the Software Reliability Group, under the direction of Prof. Cristian...

I am looking to fill a Research Associate/Assistant position on an exciting project at the intersection of program analysis and GenAI. Candidates should be able to start soon (in the next few months). If you are interested, please get in touch ASAP!

srg.doc.ic.ac.uk/vacancies/25...

SQLancer will be part of the Google Summer of Code (GSoC) program again this year! We are looking for motivated contributors as well as database development teams who would like to have their systems tested. summerofcode.withgoogle.com/programs/202...

#FUZZING'25 CALL FOR PAPERS
──────
✨ New OC members:
* Ruijie Meng (@ruijiemeng.bsky.social; NUS)
* Rohan Padhye (@rohan.padhye.org; CMU).
✨ New paper type: Fuzzing Nuggets (short papers).

🔗 fuzzingworkshop.github.io
📅 20.March (Submission)
📅 17.April (Notification)
📅 28.June (Workshop)

Post 2: NSF PPoSS Status Update–2024 Year in Review

kmicinski.com/modern-deduc...

How to detect backdoors efficiently?

▶️ Backdoors were found in firmware & open-source code
▶️ Detection requires much manual reverse-engineering
▶️ Fuzzers cannot see backdoors

Our @icseconf.bsky.social preprint on finding backdoors with fuzzing is at binsec.github.io/nutshells/ic...

The upcoming ICFP/SPLASH’25 will feature 23 (!!) co-located workshops, in addition to the main conferences, doctoral symposium, Onward!, and SAS, and we can’t wait to see everyone in Singapore in October.

Here’s a sneak peek of the hallway track location.

📢📢VLDB26 (@vldb.bsky.social) will have a ShadowPC! Apply at forms.office.com/e/XYihKj4UKq by March 1! Great opportunity for folk who are new to the data management community & want to gain paper reviewing experience. More info at application form. Chairs: @zistvan.bsky.social, Tianzheng Wang, & I.

Siesta 2025 is coming: 5th Software Engineering Summer School, August 27-29, Lugano, Switzerland. Stay tuned! siesta.si.usi.ch/2025/

Blog post titled "Concurrency bugs in Lucene: How to fix optimistic concurrency failures" By Benjamin Trent and Ao Li (February 7, 2025) Text reads: "Thanks to Fray, a deterministic concurrency testing framework from CMU’s PASTA Lab, we tracked down a tricky Lucene bug and squashed it"

Back to basics: Concurrency testing in Java!

Our new tool *Fray* correctly solves a 25+ year old problem for real-world software. See this feature from Elastic Labs about Fray's contributions to Lucene.

📰: www.elastic.co/search-labs/...

🔧: github.com/cmu-pasta/fray

📝: arxiv.org/pdf/2501.12618

Our paper on efficient automated exploit generation has been accepted to USENIX Security '25.

The gist: instead of generating individual attacks, we synthesise the whole *programming language* that expresses many exploits and guarantees their realisability.

Paper: ilyasergey.net/assets/pdf/p...

Congratulations to the new #ACM Fellows of 2024, including Satish Chandra, Marsha Chechik, Derek Dreyer, Tim Menzies, Mira Mezini, Bashar Nuseibeh, Abhik Roychoudhury @abhikrc.bsky.social, and Ben Zorn. What a great year this is!

There are no assignments, but we'll be having a hackathon. The speakers will present their topics of expertise and what they think is most exciting. Thus, there will also be no curriculum. We will upload talk abstracts closer to the start of the summer school.

We will hold the Fuzzing and Software Security Summer School 2025 in Singapore from May 26 to 30. Please consider registering or sharing the info with anyone you think would be interested. Registration will be open until 16 Feb. fuzzing.comp.nus.edu.sg @abhikrc.bsky.social @umathur.bsky.social

Followed the call of an Oriental Pied Hornbill that lead me to a pair of Buffy Fish Owls. Only my second time to see wild owls in Singapore!

An exciting talk by Armando Solar-Lezama on applying PL ideas for ML, with applications in robotics and computational biology. You can’t do cooler than that.

Also, a second SIGPLAN Milner Award recipient giving a talk at NUS in the last two weeks.

One week today is the deadline for expressing your interest to attend Programming Language Implementation Summer School (PLISS) -- we have a great program, if I do say so myself! pliss.org/2025/