Automated Mouse Pose Estimation and Social Behaviour Modelling

Social behaviour plays a significant role in both the neurological testing of animal models used in behavioural neuroscience and psychiatric studies. Laboratory mice provide a valuable platform to study psychiatric and neurological disorders such as Parkinson’s disease (PD). The quantification of interactions between mice is necessary for the study of their naturalistic social behaviour. Despite efforts made within the research community, automatically and accurately recognising complex social behaviours remains an open and challenging problem. In this thesis, a fully automated and trainable computer vision system is proposed to estimate 2D mouse pose and analyze complex mouse social interactions using video data recorded by calibrated cameras. To achieve this goal, a novel Graphical Model based Structured Context Enhancement Network (GM-SCENet) is firstly proposed for mouse pose estimation, which simultaneously models the differences and spatial correlations between mouse body parts Secondly, a novel Cross-Skeleton Interaction Graph Aggregation Network (CS-IGANet) is proposed to learn mouse social behavior representation based on pose information, where dense and sparse skeletons cooperatively explore the spatio-temporal dynamics of social interactions. Finally, to uncover the complex behavioral correlations of mice in long videos, a general semi-supervised learning framework called Semantic-guided Multi-level Contrast with Neighbourhood-Consistency-Aware unit (SMC-NCA) is presented for temporal action segmentation, aiming to model temporal dependencies using only a few labeled videos. The proposed methods have been evaluated by conducting extensive experiments on the public and proposed datasets. Experimental results demonstrate the effectiveness of the proposed methods for 2D mouse pose estimation, skeleton-based representation learning, and semi-supervised action segmentation.