Flora Salim

Bisecle: Binding and Separation in Continual Learning for Video Language Understanding Frontier vision-language models (VLMs) have made remarkable improvements in video understanding tasks. However, real-world videos typically exist as continuously evolving data streams (e.g., dynamic s...

Finally, we compared Bisecle to frontier models like GPT-4o and Gemini 2.5. Note that we can only use APIs to test them. We show that these frontier LLMs still struggle with temporal reasoning and dealing with non-stationary, evolving video tasks. In some tasks, Bisecle show superior performance

- Bisecle exhibits remarkable resistance to forgetting
- Bisecle is compatible with LLMs from 1B to 13B, introducing only a small number of additional parameters and computational cost.
- Bisecle can achieve superior performance even in low-resource settings.

The results show that:
- Bisecle establishes a new SOTA results surpassing others in both accuracy (+15.79%) and forgetting reduction (8.49% lower Forgetting rate).
- Our method Bisecle consistently outperforms others, indicating strong robustness even when training data is limited.

The two components of Bisecle with complementary angles:
- multi-directional supervision mechanism improves knowledge preservation.
- contrastive prompt learning scheme is designed to isolate task-specific knowledge to facilitate efficient memory storage, and to explicitly mitigate update conflict.

Excited to share that Bisecle is accepted at #NeurIPS2025.
🧠 Bisecle: Binding and Separation in Continual Learning for Video Language Understanding.
Preprint: arxiv.org/abs/2507.00469
Code: github.com/cruiseresear...
Inspired by the rapid binding and pattern separation mechanisms in the hippocampus

Award-winning tech enhances information retrieval in wearable AI devices - ADM+S Centre A team of researchers from ADM+S at UNSW has been awarded third place for single-source augmentation in the highly competitive KDD Cup 2025 Meta CRAG-MM Challenge, ranking alongside top institutions s...

Congrats to team members Prof Flora Salim, Breeze Chen & Wilson Wongso from ADM+S and other team members Xiaoqian Hu & Yue Tan from UNSW who placed 3rd in the highly competitive KDD Cup 2025 Meta CRAG-MM Challenge @florasalim.bsky.social
#KDD2025 #WearableAI #VLLMs admscentre.org/4mu7lFx

Multimodal CL #postdocjob #jobs 🦘🎓

* Potential for LLM steering:  The research explored the potential to manipulate ToM-related information within the LLMs to generate more aligned and contextually appropriate responses.

The first author - 1st year student Mehdi Jafari is attending his first academic conference #ACL2025.

* ToM-informed Alignment Improves Response Quality: Empirical evaluations of LLaMA-3 models (3B and 8B) demonstrated that incorporating ToM principles into the conversational agents improved response quality significantly, achieving win rates of 63% and 67% respectively.

Key Findings:
* LLMs Can Represent and Retain ToM-related Constructs: The study investigated whether LLMs could represent and retain ToM-related constructs and found evidence supporting this ability.

* ToM-informed Alignment Improves Response Quality:

In Beyond Words, we explore:
a) The extent to which the activation space of LLMs represents ToM of interlocutors,
b) Whether these representations form a consistent model of ToM,
and
c) How can we leverage ToM-related features to generate more aligned responses?

Current LLMs often generate contextually appropriate responses, but they don’t truly understand the user's goals, beliefs, or misunderstandings. 
Using ToM, we can analyse interlocutor behaviours based on the understanding of their mental and emotional states.

Beyond Words: Integrating Theory of Mind into Conversational Agents for Human-Like Belief, Desire, and Intention Alignment (#ACL2025 Findings)

aclanthology.org/2025.finding...

Codes: github.com/cruiseresear...

Findings in the thread below.

I’m recruiting a 1.5 year postdoc in multimodal continual learning external-careers.jobs.unsw.edu.au/cw/en/job/53...

Check our recent work on this topic. Bisecle: Binding and Separation in Continual Learning for Video Language Understanding
arxiv.org/abs/2507.00469

Know anyone suitable? Pls repost

Overleaf is down. Ah well... theoretically, only 30,000+ people hammering it 2 days before NeurIPS deadline.

NVIDIA Dynamo. Exciting announcement by Jensen Huang #gtc2025. A new open source VMWare-like inference framework for reasoning. It breaks up prefill and decode steps efficiently. DeepSeek R1 requests can be boosted by 30x. Already used by Perplexity, Meta, etc
developer.nvidia.com/blog/introdu...

I’m in San Jose this week for NVIDIA GTC. I’m a panelist for “The Role of AI and Accelerated Computing in Understanding and Mitigating Urban Climate Change” session.
We'll discuss how AI transforms climate modeling, weather forecasting, and high-resolution urban simulations.
Anyone else attending?

Francis Collins, the NIH Director for 12 years, led the Human Genome Project and other NIH efforts for 32 years, resigned today. Key words from his resignation letter
www.nytimes.com/2025/03/01/u...

DOGE Is Working on Software That Automates the Firing of Government Workers Operatives working for Elon Musk’s DOGE appear to be editing the code of AutoRIF—software designed by the Defense Department that could assist in mass firings of federal workers, sources tell WIRED.

We know many cases of automated hiring gone wrong in the past. Automated firing? Doesn’t look good. No guarantee on transparency. The agency will be left to an unnamed LLM: “info would be fed into an unspecified LLM that would assess whether an employee was necessary” www.wired.com/story/doge-a...

How to find climate data and science the Trump administration doesn’t want you to see Several groups are working to preserve webpages, tools and data – some of which have already gone missing from government webpages since the start of the Trump administration.

Public Environmental Data Partners archives climate datasets, uploading copies to public repositories and cataloging where and how to find them if they go missing from government websites. Scholars explain:

😔😢

AIcrowd | Brick by Brick 2024 | Challenges Automating Building Data Classification

The Brick-by-Brick 2024 challenge focuses only on the multi-label classification problem, which we consider to be harder, and the holy grail for automation and management of net-zero and sustainable buildings.
Round 2 of Brick by Brick 2024 has commenced! To join: www.aicrowd.com/challenges/b...

The task also tackles issues like imbalanced data and sparse labels, all while addressing real-world problems like building optimization and sustainability.
Our NeurIPS 2024 paper includes both a multi-label classification benchmark and a zero-shot forecasting benchmark.
neurips.cc/virtual/2024...

BTS is more challenging than existing TS datasets and a lot more interesting because BTS captures the complexities of real-world operations: 1) Temporal Irregularity; 2) Spatial Heterogeneity; 3) Long-tail Distribution.
-- requires models to manage hierarchical dependencies and ensure consistency.

** Knowledge Graph (KG)
BTS also includes a KG that captures the relationships between TS and their physical, logical, and virtual entities.
Making it a great case for Hierarchical Multi-Label Classification. The TS are to be classified across nested categories (e.g. Point>Sensor>Air Quality>CO2).

The Building TimeSeries (BTS) dataset offers a rich, hierarchical structure of a large collection of diverse timeseries. The dataset include the following components:
* Multivariate TimeSeries: From 2021 to 2024, contains over 15,000 timeseries (TS) across 300 unique classes.
* Knowledge Graph
🧵

Earth breaches 1.5 °C climate limit for the first time: what does it mean? The threshold has been exceeded for only one year so far, but humanity is nearing the end of what many thought was a ‘safe zone’ as climate change worsens.

A new stark reality to hit us and the entire planet ecosystem. And it’s only going to get worse unless we do something about it
www.nature.com/articles/d41...

International World Wide Web Conference 2025 ( WWW2025 | The Web Conf 2025 ) 28 April - 2 May 2025

Brick-by-Brick is a #WebConf2025 competition.
www2025.thewebconf.org/accepted-com...

Round 1 is well underway with hundreds of submissions.
Check it:
www.aicrowd.com/challenges/b...

Not too late to join. We’re looking for winners across Round 1 and 2. Win cash prizes and travel grants to WebConf.

#IJCAI2025 Call for Tutorials
📆Submission deadline: 28 March 2025
➡️ 2025.ijcai.org/call-for-tut...
📣Chair: Flora Salim and Quanming Yao

#AI #Research #Innovation #ArtificialIntelligence #tutorial #cfp

Another one down
BTS: Buildings Time Series
+ CRUISE in NeurIPS team photo
#neurips2024
@neuripsconf.bsky.social