Illustration of the framework’s structure. Users can specify primary classes and secondary classes across both static and motion video streams (a–f). The right panel shows examples of static and color-from-motion frames from case studies. The color-from-motion trails create characteristic patterns that reveal movement and behavior. The grass moth (Crambidae) is nearly undetectable from still frames due to its color, and motion blur, but it is highly salient in the motion stream (g). The semaphore fly (Poecilobothrus nobilitatus) example shows how motion information can easily disentangle behaviors that are often identical from static frames (e.g., “fly” and “display” appear identical in the static frame, but different in motion). This example also showcases hierarchical classification, with secondary classifiers determining the sex of the flies (h). Sea slaters (Ligia oceanica) are highly camouflaged when static, and salient when moving, resulting in motion models that make far fewer errors (but cannot detect stationary individuals) (i). Human sperm have been classified based on their swimming movement with either symmetric (typically resulting in fast, straight movement), asymmetric (typically resulting in slow, circling, exploratory movement), or weak (twitching, vibrating etc…) strategies. These swimming strategies can be determined without tracking individuals, which is difficult in complex, debris-filled videos (j).
Despite advances, quantifying complex motion info remains challenging. @jtroscianko.bsky.social @kevinjgaston.bsky.social &co present BehaveAI, a #video analysis tool that sees motion as color, tracking animals & classifying #behavior in complex natural scenes @plosbiology.org 🧪 plos.io/4kTF1wX
23.02.2026 09:58 — 👍 12 🔁 6 💬 0 📌 0
