Sampling-Tree Model: Efficient Implementation of Distributed Bayesian Inference in Neural Networks

Experimental observations from neuroscience have suggested that the cognitive process of human brain is realized as probabilistic reasoning and further modelled as Bayesian inference. However, it remains unclear how Bayesian inference could be implemented by network of neurons in the brain. Here a novel implementation of neural circuit, named sampling-tree model, is proposed to fulfill this aim. By using a deep tree structure to implement sampling with simple and stackable basic neural network motifs for any given Bayesian networks, one can perform local inference while guaranteeing the accuracy of global inference. We show that these task-independent motifs can be used in parallel for fast inference without intensive iteration and scale-limitation. As a result, this model utilizes the structure benefit of neuronal system, i.e., neuronal abundance and multi-hierarchy, to perform fast inference in an extendable way.