Muti-view Mouse Social Behaviour Recognition with Deep Graphic Model

Home-cage social behaviour analysis of mice is aninvaluable tool to assess therapeutic efficacy of neurodegenerativediseases. Despite tremendous efforts made within the researchcommunity, single-camera video recordings are mainly used forsuch analysis. Because of the potential to create rich descriptionsfor mouse social behaviors, the use of multi-view video recordingsfor rodent observations is increasingly receiving much attention.However, identifying social behaviours from various views isstill challenging due to the lack of correspondence across datasources. To address this problem, we here propose a novel multi-view latent-attention and dynamic discriminative model thatjointly learns view-specific and view-shared sub-structures, wherethe former captures unique dynamics of each view whilst thelatter encodes the interaction between the views. Furthermore, anovel multi-view latent-attention variational autoencoder modelis introduced in learning the acquired features, enabling us tolearn discriminative features in each view. Experimental resultson the standard CRMI13 and our multi-view Parkinson’s DiseaseMouse Behaviour (PDMB) datasets demonstrate that our pro-posed model outperforms the other state of the arts technologies,has lower computational cost than the other graphical modelsand effectively deals with the imbalanced data problem.