Biota Buzz

The Importance of Video in Animal Behaviour Research

Video archives are an invaluable resource in behavioural research. They allow for non-contact studies across different species and environments, providing long-term datasets that can be reanalysed for new insights. For species like primates, whose populations are rapidly declining (Estrada et al., 2017), these digital archives can act as “data arks,” preserving critical behavioural records that might otherwise be lost.

Whether from camera traps, CCTV, or focal shots, video material generates large data sets which require extensive post processing for analysis. For anyone who’s spent time decoding videos, the challenge is obvious: collecting detailed behavioural data is time-consuming and labour-intensive. Typically, this involves using annotation interfaces that rely on predefined ethograms with categorical labels such as grooming, sleeping, or foraging. These are then applied manually to video footage to generate metrics on behavioural frequency and duration. Without some form of automation, this workflow creates a major bottleneck in data extraction, one that AI is increasingly being used to address.

Machine learning and pose estimation tools like DeepLabCut and SLEAP have been some of the first used in behavioural research (Mathis et al., 2018; Pereira et al., 2022; Saad Saoud et al., 2024). Originally developed for model species like mice and fruit flies, pose estimation tools use deep learning to track key points on an animal’s body, mapping movement across frames. Recent advances in deep learning (DL) have provided new tools for analysing animal movements and cognition with greater accuracy and efficiency (Couzin and Heins, 2023; Tuia et al., 2022). DL is a subset of machine learning that uses neural networks with multiple layers to automatically learn models and patterns from datasets.

Deep learning models, such as convolutional neural networks (CNNs), excel at identifying spatial hierarchies in images, making them ideal for pose estimation. These models process visual data by breaking it down into increasingly abstract features, allowing them to recognize key body parts even in challenging conditions. By leveraging CNNs, pose estimation tools can differentiate between visually similar behaviours and provide more refined movement tracking. This enables researchers to distinguish subtle variations in body postures that may indicate different emotional or social states (Achour et al., 2020).

Moreover, deep learning models continuously improve as they are exposed to more data. Transfer learning, a technique where pre-trained models are used with new datasets, allows researchers to adapt pose estimation tools for different species and environments without having to build models from scratch.

Applying AI in Research
Installing and running AI-based detection tools can be quite challenging, especially for those of us without a coding background. In my undergraduate research on maternal Grey seals, I explored using DeepLabCut to track fidgeting behaviour as a potential indicator of reactivity. While I was excited by the possibilities, I ultimately had to rely on manual measurements due to time constraints and limited computing resources.

One of the main challenges I faced was the lack of access to a GPU. While DeepLabCut can run on a CPU, training models and processing frames take significantly longer without GPU acceleration, making large-scale analysis impractical within the scope of my project. Grey seals posed a unique challenge for pose estimation due to their morphology. Unlike animals with distinct limb structures, most of their joints are obscured within their body mass, making it difficult to identify standard anatomical landmarks. This meant that pre-existing models trained on more conventional quadrupeds were not easily adaptable to seal behaviour tracking. To overcome this, I used the Napari labelling interface, which allows for customizable key points and precise manual annotation. All labelled key points were then exported as a CSV file, enabling further quantitative analysis of movement patterns. While this approach allowed for the fine-scale tracking of micro-movements, it was way too time-consuming to be rolled out across a wider data set.

Other studies have already demonstrated the wide application of AI-assisted automation. For example, DeepWild successfully applied pose estimation to wild chimpanzees and bonobos, tracking individuals across diverse environments and behaviours (Wiltshire et al., 2023). Even in visually complex settings, such as dense forests or close aggregations of conspecifics, machine learning models have proven capable of identifying and following individuals.

Captive Animal Studies
Pose estimation isn’t just for wild animal research. In captive settings, it’s being used to monitor nocturnal behaviours with minimal human intervention. A study on captive African elephants (Lund et al., 2024) demonstrated how DeepLabCut and Create ML could track behaviours recorded on CCTV footage, reducing the need for constant observation. While the models excelled at identifying simple behaviours like lying down or standing, they struggled with more complex, repetitive behaviours such as stereotypic swaying. However, ongoing refinements suggest that AI models will soon be capable of detecting even these subtle behavioural nuances, which are crucial for welfare assessments.

The Future of AI in Behavioural Research
Machine learning is not without its limitations. Issues like visually similar behaviours, environmental variability, and the need for large annotated datasets pose persistent challenges. However, initiatives like the ManyPrimates consortium aim to mitigate these by encouraging data sharing between research groups (Wiltshire et al., 2023). Moreover, the scalability of AI-driven pose estimation is unparalleled. As algorithms become more robust and adaptable, they could be used to monitor long-term behavioural changes in both wild and captive animals.

The development of platforms such as BEHAVE, an open-source, web-based annotation interface, reflects a broader shift towards more user-friendly behavioural tools (Elhorst et al., 2025). BEHAVE, for instance, incorporates AI-assisted video navigation and programmable ethograms, exporting structured outputs that can be easily integrated with downstream analytical workflows. Such tools are designed not to replace human judgement but to improve reproducibility and transparency in behavioural science.

The capacity to automate behavioural annotation represents a methodological turning point in animal behaviour research. For species where observation is logistically or ethically challenging, pose estimation provides a means to extract high-resolution data at scale. For those of us who have spent hours annotating frame-by-frame, these developments offer a more precise way of understanding how animals move through and interact with their environments.

References 

Elhorst, R., Syposz, M. and Wojczulanis-Jakubas, K. (2025). BEHAVE – facilitating behaviour coding from videos with AI-detected animals. Ecological Informatics, [online] 87, p.103106. doi:https://doi.org/10.1016/j.ecoinf.2025.103106.

Lund, S. M., Nielsen, J., Gammelgård, F., Nielsen, M. G., Jensen, T. H., & Pertoldi, C. (2024). behavioural Coding of Captive African Elephants (Loxodonta africana): Utilizing DeepLabCut and Create ML for Nocturnal Activity Tracking. Animals, 14(19), 2820. https://doi.org/10.3390/ani14192820.

Mathis, A., et al. (2018). Deeplabcut: Markerless pose estimation of user-defined body parts with deep learning. Nat. Neurosci., 21, 1281–1289.

Pereira, T.D., Tabris, N., Matsliah, A., Turner, D.M., Li, J., Ravindranath, S., Papadoyannis, E.S., Normand, E., Deutsch, D.S., Wang, Z.Y., McKenzie-Smith, G.C., Mitelut, C.C., Castro, M.D., D’Uva, J., Kislin, M., Sanes, D.H., Kocher, S.D., Wang, S.S.-H. ., Falkner, A.L. and Shaevitz, J.W. (2022). SLEAP: A deep learning system for multi-animal pose tracking. Nature Methods, 19(4), pp.486–495. doi:https://doi.org/10.1038/s41592-022-01426-1.

Saad Saoud, L., Sultan, A., Elmezain, M., Heshmat, M., Seneviratne, L. and Hussain, I. (2024). Beyond observation: Deep learning for animal behaviour and ecological conservation. Ecological Informatics, 84, p.102893. doi:https://doi.org/10.1016/j.ecoinf.2024.102893.

Wiltshire, C., et al. (2023). DeepWild: Application of the pose estimation tool DeepLabCut for behaviour tracking in wild chimpanzees and bonobos. Journal of Animal Ecology, 92, 1560–1574. https://doi.org/10.1111/1365-265