Long-short diffeomorphism memory network for weakly-supervised ultrasound landmark tracking

Ultrasound is a promising medical imaging modality benefiting from low-cost and real-time acquisition. Accurate tracking of an anatomical landmark has been of high interest for various clinical workflows such as minimally invasive surgery and ultrasound-guided radiation therapy. However, tracking an anatomical landmark accurately in ul-trasound video is very challenging, due to landmark deformation, visual ambiguity andpartial observation. In this paper, we propose a long-short diffeomorphism memory net-work (LSDM), which is a multi-task framework with an auxiliary learnable deformationprior to supporting accurate landmark tracking. Specifically, we design a novel diffeo-morphic representation, which contains both long and short temporal information storedin separate memory banks for delineating motion margins and reducing cumulative er-rors. We further propose an expectation maximization memory alignment (EMMA)algorithm to iteratively optimize both the long and short deformation memory, updatingthe memory queue for mitigating local anatomical ambiguity. The proposed multi-task system can be trained in a weakly-supervised manner, which only requires fewlandmark annotations for tracking and zero annotation for deformation learning. Weconduct extensive experiments on both public and private ultrasound landmark track-ing datasets. Experimental results show that LSDM can achieve better or competitivelandmark tracking performance with a strong generalization capability across differentscanner types and different ultrasound modalities, compared with other state-of-the-artmethods.