Brian Wu ✈️ #NeurIPS2024

If an abstraction cannot effectively hide underlying details and allow users to focus solely on high-level functionality, it risks making the system unnecessarily complex.
*Reference: fhur.me/posts/2024/t...

𝑪𝒐𝒏𝒄𝒍𝒖𝒔𝒊𝒐𝒏
Before introducing an abstraction, we must carefully consider: does it truly simplify the system, or is it merely adding another layer of indirection?

If an abstraction fails to effectively hide implementation details and simplify usage, it instead increases two critical resource demands: (1) the cognitive resources required for humans to understand the code and (2) the computational resources needed to execute additional layers of abstraction.

3. 𝐀𝐬𝐲𝐦𝐦𝐞𝐭𝐫𝐲 𝐨𝐟 𝐀𝐛𝐬𝐭𝐫𝐚𝐜𝐭𝐢𝐨𝐧 𝐂𝐨𝐬𝐭𝐬
The asymmetry here refers to the fact that abstracted code may be easier for the creator to understand and use but often imposes higher costs on others who need to comprehend, modify, or maintain it.

That is, there will be scenarios where users are forced to confront the underlying complexities. For instance, disk space management is typically abstracted at the software level, but when the disk space runs low, system performance may degrade, exposing the physical limitations of the hardware.

2. 𝐀𝐥𝐥 𝐀𝐛𝐬𝐭𝐫𝐚𝐜𝐭𝐢𝐨𝐧𝐬 𝐋𝐞𝐚𝐤
While abstracted code allows users to interact with high-level functionality without concerning themselves with lower-level details most of the time, abstractions inevitably “leak.”

However, if the underlying details frequently require modification, abstraction may not be the best approach.

1. 𝐖𝐡𝐚𝐭 𝐌𝐚𝐤𝐞𝐬 𝐚 𝐆𝐨𝐨𝐝 𝐀𝐛𝐬𝐭𝐫𝐚𝐜𝐭𝐢𝐨𝐧?
A key feature of a good abstraction is its ability to hide underlying implementation details. One of the benefits of abstraction is that it allows us to focus on experimental logic and functionality rather than the details hidden beneath.

This morning, I came across a friend’s insightful post* on this topic. Below, I summarize their points and add my own thoughts:

However, abstraction is not inherently beneficial—excessive or improper abstraction can make a codebase harder to understand or modify, ultimately hindering experimentation. So, are there any guiding principles for effective abstraction?

In experimental computer science research, particularly when working on projects with similar themes, one often discovers opportunities to reuse code. In such cases, introducing appropriate levels of abstraction can accelerate future experiments.

Let Me Speak Freely? A Study on the Impact of Format Restrictions on Performance of Large Language Models Structured generation, the process of producing content in standardized formats like JSON and XML, is widely utilized in real-world applications to extract key output information from large language m...

* Published papers:
[1] Let Me Speak Freely? A Study on the Impact of Format Restrictions on Performance of Large Language Models: arxiv.org/abs/2408.02442
[2] I Need Help! Evaluating LLM's Ability to Ask for Users' Support: A Case Study on Text-to-SQL Generation: arxiv.org/abs/2407.14767

While reading papers on arXiv is a good way to keep up with the latest advancements, the intensity and depth of face-to-face discussions with different individuals are unparalleled. This experience has encouraged me to keep exposing myself to the world’s best scholars.

3. The Value of Intellectual Exchange
For research, engaging in discussions and exchanging ideas with outstanding scholars from diverse backgrounds is invaluable. It’s easy to become too comfortable in our daily routines or lack diversity in intellectual inputs.

I’m grateful for his swift implementation of this idea, which validated our hypothesis and culminated in this work. Moving forward, I aspire to continually refine my research intuition, improving my ability to predict the potential impact of our studies.

Reflecting back, when I first conceived this idea over six months ago, I confidently told my colleague Zhi Rui Tam: “I’m sure this paper will be the most impactful among our recent research outputs.”

During the conference, numerous attendees mentioned, “I’ve read your paper!” One professor even told me after my poster presentation how much she appreciated our findings and wanted to share them with her students.

In one of our papers [1], we posed a research question: “Do format restrictions affect the performance of LLMs’ generated content?” Answering this didn’t require highly advanced techniques, but it addressed a question that both academia and industry care deeply about.

2. The Importance of Asking the Right Scientific Questions:
“Asking the right question” is far more important than “how technically challenging” a problem is. While this might seem obvious, this conference reinforced its truth.

I feel incredibly fortunate to be a Research Scientist in Taiwan, truly engaging in academic research—a role that is rare and invaluable in the local industry. During Taiwan Night, many attendees expressed how unique it is for Taiwan’s industry to have such positions.

1. Gratitude to My Company - Appier:

First and foremost, I’m deeply thankful to my company Appier for fully sponsoring our AI Research Team to attend this conference and for hosting Taiwan Night, which brought together outstanding Taiwanese AI researchers from around the globe.

# Reflections on hashtag #EMNLP2024
I recently attended hashtag #EMNLP2024 in hashtag #Miami and presented our research findings with Zhi Rui Tam [1][2]. Here are a few key takeaways from the experience: