Lukas Aichberger

Hot take: I think we just demonstrated the first AI agent computer worm 🤔

When an agent sees a trigger image it's instructed to execute malicious code and then share the image on social media to trigger other users' agents

This is a chance to talk about agent security 👇

Attacking Multimodal OS Agents with Malicious Image Patches Recent advances in operating system (OS) agents enable vision-language models to interact directly with the graphical user interface of an OS. These multimodal OS agents autonomously perform computer-...

🏛️ This work was made possible with OATML and TVG at the University of Oxford (@ox.ac.uk). Special thanks to @yaringal.bsky.social, @adelbibi.bsky.social, @philiptorr.bsky.social, and @alasdair-p.bsky.social for their contributions.

📖 Read the paper: www.arxiv.org/abs/2503.10809

💀 Harmful actions could include engaging with the malicious social media post to amplify its spread, navigating to a malicious website, or causing a memory overflow to crash your computer. Preventing such harmful actions remains an open challenge. [6/6]

🎯 Once an OS agent – among those the MIP was optimised for – encounters the MIP during the execution of everyday tasks, empirical results indicate harmful actions are triggered in at least 9 out of 10 cases, regardless of the original task or screenshot layout. [5/6]

🚨 The real danger? Attackers can simply embed MIPs in social media posts, wallpapers, or ads and spread them across the internet. Unlike text-based attacks, MIPs are hard to detect, allowing them to spread unnoticed. [4/6]

🔓 Our work reveals that OS agents are not ready for safe integration into everyday life. Attackers can craft Malicious Image Patches (MIPs), subtle modifications to an image on the screen that, once encountered by an OS agent, deceive it into carrying out harmful actions. [3/6]

💻 AI assistants, known as OS agents, autonomously control computers just like humans do. They navigate by analysing the screen and take actions via mouse and keyboard. OS agents could soon take over everyday tasks, saving users time and effort. [2/6]

⚠️ Beware: Your AI assistant could be hijacked just by encountering a malicious image online!

Our latest research exposes critical security risks in AI assistants. An attacker can hijack them by simply posting an image on social media and waiting for it to be captured. [1/6] 🧵

Often LLMs hallucinate because of semantic uncertainty due to missing factual training data. We propose a method to detect such uncertainties using only one generated output sequence. Super efficient method to detect hallucination in LLMs.

Rethinking Uncertainty Estimation in Natural Language Generation Large Language Models (LLMs) are increasingly employed in real-world applications, driving the need to evaluate the trustworthiness of their generated text. To this end, reliable uncertainty estimatio...

𝗡𝗲𝘄 𝗣𝗮𝗽𝗲𝗿 𝗔𝗹𝗲𝗿𝘁: Rethinking Uncertainty Estimation in Natural Language Generation 🌟

Introducing 𝗚-𝗡𝗟𝗟, a theoretically grounded and highly efficient uncertainty estimate, perfect for scalable LLM applications 🚀

Dive into the paper: arxiv.org/abs/2412.15176 👇

🙋‍♂️